On architecture

I have been thinking that there should be a generic way to express the role of an architect in an organization and thus also helping people be better architects. So here it comes.

The layers
Architects know that system architecture is not a thing in itself, it is strongly bound to how an organization works in general. There are authors who have thought about that to some extent (like Christophe Longépé) but they usually only look at the technical or functional side of the systems. It seems to be much wider than that, let's build a stack here:

1. Organizational architecture. This layer answers questions like what business are we in? How do different parts of organization relate to each other? What are the core principles the organization lives upon? What business processes do we have?
   
2. Functional architecture. Every organization performs a certain set of functions in its everyday life. This layer deals with what clumps of functionality (like accounting, HR, customer service etc.) there are and how they interact.
3. Technical architecture. This is the domain usually attributed to technical architects and deals with logical components, integration methodologies, middleware and so forth.
   
4. Support architecture. The layer consists of processes tools, and organization that works underneath the technical architecture. Version control setup, release and testing procedures, coding conventions, team interaction, development methodology etc.
   

An example
Let's explore this layering using an example. Let's say a system architect has three projects on the table for evaluation, each of them approaching the concept of a corporate customer from three different angles creating a technically unsolvable puzzle: although the architect sees that it would make sense to have the three systems talking to each other this can not be done because one treats companies as phone numbers with some fields attached, the second one as a source for authentication information (several customers can operate under the same legal agreement) and the third as an entity that has to pay VAT.

How did it come to this? Apparently all of the layers were affected. Firstly, the organization was set up as a set of silos which do not talk to each other, which leads to the three different unsynchronized projects being initiated.

Secondly, the functional architecture consists of a phonebook, a list of warrants describing who can do what business with the company and an accounting system. This poor setup gives the business units no support in their quest for solving their business problems and getting money pouring in.

The technical architecture, the third layer, now has to face the situation and figure out a way to solve this. As forcing the business people back to the drawing board does not work (remember, there are silos and nobody has ever spoken to them about the concept of a unified customer view) and re-writing all existing systems would blow the budgets of all three projects all there is to do is to build three separate systems. Which, as the architect knows, will lead to an integration nightmare the moment somebody discovers it would make sense to have the ability to have the company phone number displayed along the agreement information so the legal people can conveniently place calls.

The support layer now faces even more trouble: instead of having one or two applications to deploy there is three, all of which have different availability requirements, different release cycles, different versioning schemes and so on.

In short, a problem in an upper architecture layer leads to a growing amount of issues on its way down. Actually, the same applies to the opposite direction. Imagine a setup where two different outsourcing partners (one using RUP and the other XP) are forced to work together to tightly integrate their applications as there has been a change in the organizational structure. The very least that will happen is that the applications remain separated and the business guys suffer acute schizophrenia while participating in a planning game in the morning and compiling requirement artifacts in the afternoon.

The conclusions
Following conclusions can be drawn from the layering concept:

1. A poorly set up layer "radiates" issues to the neighboring layers. For example, a poor technical architecture leads to problems in deployment because of unmanageable dependencies and a weird organizational structure leads to a weird functional architecture.
   
2. A major change in any of the layers causes changes in every layer with the impact getting weaker the further you get from the source of the change. What is interesting about this one is that it usually only applies downwards. If changes start filtering upwards, something is very badly broker somewhere as there should not be any technical change without a business cause.


3. The model shows a weak spot most of the companies happily ignore: the fact that nobody is really responsible for the functional architecture layer. However, it acts as a filter between top and bottom layers and thus, if undeveloped, hinders all cooperation between them. The only conclusion to draw from here is that a system architect needs to make darn sure that part of the stack is under control taking it to him or herself if necessary.

Comments from Sergei added as a separate post

Chaos and agility

The organization I work for has grown tremendously over the last couple of years and I am talking more than 100% per annum in terms of headcount. This is not something one can deal easily with from the standpoint of development processes and how people think of them.

In the very beginning we had the Agile Manifesto on our fridge door. We sure as hell did not follow it but at least we knew it was there. Nowadays nobody really remembers or talks about it any more. There are committees and processes and papers and things. The funny thing is that none of it really seems to improve anything that much: we are doing pretty well as we were, the levels of frustration have not changed (the kind of them has, however), and we do not stick to deadlines not more and not less than before. There are, however, massively more people around and more projects going on which means more mistakes will be made and more deadlines missed and the _number_ of people who get bruised is bigger. Which makes the problems more obvious and hence leads to more processes.

The reason I am writing this is that I mentioned how most people mix up chaos and agility (as much as they tend to mix up bad intent with stupidity, but that's another story) and people started laughing. I dropped the same line a couple of hours later and got the same reaction. Clearly, this was not obvious to the people around me and that got me thinking. And lead to this post. Anyway.

The reason we have not managed to increase our development throughput (not that it is bad, vice versa: we spit out more innovation at higher speed than most) or keep levels of frustration under control is thinking that replacing agility with a more stringent set of regulations will help to create order and make us more predictable. Wrong. What we are actually doing is replacing development chaos with regulated chaos. Think of a bucket full of cockroaches. Now imagine you insert the internals of a wine crate (you know, the thing dividing a box into 6 sections comfortable for a bottle) into it. Do you get more order? No. You get an illusion of order but within the 6 sections, the very same fizzling and buzzing goes on.

The attitude of people does not change when you introduce processes or try to enforce more rules. They still (as a group, not necessarily as individuals) prefer a chaotic approach because they do not know better and it has served them well. Coming back to my point about chaos and agility: agile development processes have some of the most complex rules systems I have known. Look a http://www.extremeprogramming.org/. The stuff is complicated! This is why it is so difficult to pull off successfully. It sure looks chaotic to the naked eye used to clean lines of a waterfall. This is why every company who for whatever reason has not implemented a software development process can hide behind the shield of agility and say "We did agile, it did not work and now we have RUP and everybody is happy. Well, at least they know why their lives suck now."

An interception with regards to Toivo: we really did not hide behind that one back in the days. We just figured getting cool stuff out the door is more important than the ability to express exactly how we are doing it by implementing a particular methodology. And your method sure worked!

Well, I guess what I am trying to say is that as long as one replaces an un-organized chaos with an organized one under the sign of implementing a process one can not hope any major improvements. One should focus on the people instead (Individuals and interactions over processes and tools) and see that their needs are taken care for by maybe taking the Manifesto close to ones heart. Managing processes and business is easy. Managing people is hard.

How to become an arhcitect? Huh?

I was so outraged by an article referenced by TSS that I just could not help put create this post. Would you believe it, that somebody actually goes out and defines how to become an architect. What the heck is that? It's like trying to define how to become an artist or how to compose music or how to become a leader. Sure, there are technical skills to gather and you must read and have experience and all that but at the end of the day it comes down to sheer talent. Of having a vision and being able to execute on it. Anybody claiming they have guidelines on how to become one or in fact who an architect is and what it does clearly does not understand organizations and the software development process.

There, I said it. Luckily enough there is still hope as some comments on the page seem to reflect the same viewpoint.

Will go and try to calm down, promise.