In the very beginning we had the Agile Manifesto on our fridge door. We sure as hell did not follow it but at least we knew it was there. Nowadays nobody really remembers or talks about it any more. There are committees and processes and papers and things. The funny thing is that none of it really seems to improve anything that much: we are doing pretty well as we were, the levels of frustration have not changed (the kind of them has, however), and we do not stick to deadlines not more and not less than before. There are, however, massively more people around and more projects going on which means more mistakes will be made and more deadlines missed and the _number_ of people who get bruised is bigger. Which makes the problems more obvious and hence leads to more processes.
The reason I am writing this is that I mentioned how most people mix up chaos and agility (as much as they tend to mix up bad intent with stupidity, but that's another story) and people started laughing. I dropped the same line a couple of hours later and got the same reaction. Clearly, this was not obvious to the people around me and that got me thinking. And lead to this post. Anyway.
The reason we have not managed to increase our development throughput (not that it is bad, vice versa: we spit out more innovation at higher speed than most) or keep levels of frustration under control is thinking that replacing agility with a more stringent set of regulations will help to create order and make us more predictable. Wrong. What we are actually doing is replacing development chaos with regulated chaos. Think of a bucket full of cockroaches. Now imagine you insert the internals of a wine crate (you know, the thing dividing a box into 6 sections comfortable for a bottle) into it. Do you get more order? No. You get an illusion of order but within the 6 sections, the very same fizzling and buzzing goes on.
The attitude of people does not change when you introduce processes or try to enforce more rules. They still (as a group, not necessarily as individuals) prefer a chaotic approach because they do not know better and it has served them well. Coming back to my point about chaos and agility: agile development processes have some of the most complex rules systems I have known. Look a http://www.extremeprogramming.org/. The stuff is complicated! This is why it is so difficult to pull off successfully. It sure looks chaotic to the naked eye used to clean lines of a waterfall. This is why every company who for whatever reason has not implemented a software development process can hide behind the shield of agility and say "We did agile, it did not work and now we have RUP and everybody is happy. Well, at least they know why their lives suck now."
An interception with regards to Toivo: we really did not hide behind that one back in the days. We just figured getting cool stuff out the door is more important than the ability to express exactly how we are doing it by implementing a particular methodology. And your method sure worked!
Well, I guess what I am trying to say is that as long as one replaces an un-organized chaos with an organized one under the sign of implementing a process one can not hope any major improvements. One should focus on the people instead (Individuals and interactions over processes and tools) and see that their needs are taken care for by maybe taking the Manifesto close to ones heart. Managing processes and business is easy. Managing people is hard.