Considering Agile, deciding against it

Some time ago, while managing a distributed software project, I spoke to a developer who was thinking about joining our team. This individual was a big proponent of Agile development. When he found out that in our particular project we were not using an Agile development model, he began to lose interest. He compared my position there to the position of the Tom Smykowski character from the movie Office Space. When I recently then came across this blog posting by Shannon JJ Behrens, I had to remember that particular incident...

In the movie Office Space, Tom is an older employee at Initech, whose job it is to “take requirement documents from the customers and bring them to the developers”. Not surprisingly, he is being characterized as 'useless' and let go. Similarly, in Agile development, direct communication between customers and developers is seen as the way to ensure that the deliverables align with customer requirements. No middle man is needed. Instead of extensive up-front design, requirements are captured in short and concise user stories.

I have to admit, I was somewhat taken aback by the comparison to the Smykowski caricature from the movie, but this episode certainly made me think about why we had chosen a non-Agile style for our project.

Let's try to answer this. Take a look at the graphic here (I don't remember where I saw it first, but it can be found on many different sites, click to see the larger version):

What do we not see anywhere? Design and architecture. The definition of the 'backlog' and the 'release plan' is probably the closest we get, but even that is feature focused. And in Agile, features always seem to be driven by customers.

For me it boils down to this: I believe that Agile is applicable in projects where not only a clear vision of the project has been communicated to the developers, but also where certain conventions and structures have already been established and where we are dealing mostly with explicit requirements, rather than implicit ones. Projects, where the 'how' is already known, and we are just dealing with the 'what'.

For example, to achieve a consistent UI you require some up-front planning, possibly a UI czar: One person - or at most a small number of people - who establish and downright dictate what the UI shall look and feel like. It may stifle the creativity of the developer, but it will ensure a much more consistent look for the UI. Imagine you have user stories describing the requirements for various dialogs in the application's interface. Different developers take on the implementation of those user stories. Unless there is some established, over-arching strategy and - dare I say it? - upfront design of the UI then the results are likely to appear disjointed or piecemeal.

Another example is error handling in the application. I remember one of my software engineering instructors a long time ago, who insisted that a project should have an error czar: Someone who establishes the standard by which errors should be handled within the code and who then diligently ensures that all new code complies with this standard. If this is not done and established ahead of time, how are the different developers supposed to produce consistent code that knows how to adhere to those standards and handles errors in a unified manner?

It appears to me that if you are developing an application for an existing framework then many of these up-front 'infrastructure' decisions have been made for you already. Consequently, you can focus more readily on explicit customer requirements and Agile is a much more worthwhile development strategy.

For example, if you develop a web-app in a framework such as Django or Rails then there are specific ways in which your database is going to be laid out. You also have specific conventions for how errors are handled, and so on and so forth. Thus, the design and architecture of much of your application is pre-determined by your choice of framework. You don't have to make those decisions any more. And later then, the more of the application has been written, the more such underlying decisions will have been made already. Agile works well in those situations. Since big-picture architectural decisions don't have to be made any more, we can focus on the addition of small features (or larger features broken down into many smaller ones), while all developers benefit from the established infrastructure of the project. The 'grab-bag' approach with focus on the individual user story becomes feasible.

However, if a vision for the project still has to be conjured up and the basic architecture has to be created and the UI look and feel needs to be designed and error handling or other standards have to be established... well, then Agile does not seem to be ideally suited for that. I have seen design and architecture by committee and design by chaos (everyone going off to implement things before anything was established). Neither one is a pretty sight. The best results are achieved if the architectural direction and basic design can be established by one or just a few, good individuals. Up-front. Well, somewhere after establishing the vision and collecting initial requirements, of course.

Realistically, I'm sure that most Agile projects will include some sort of up-front design and architecture, whether the participants call it that or not. Heck, even questions like: “Should we use Java or Python for this?” need to be answered ahead of time and are part of these implied up-front activities.

That particular project for which the developer from the beginning of this article interviewed was not at the stage yet where an Agile approach really would have worked. There were still too many unknowns at that time, which required up-front consideration. Thus, our project wasn't run as an Agile project. There is a time and place for everything, and our project was not at a time or place for Agile, yet.

However, projects can be transitioned from a more conventional approach to an Agile approach over time. The more we know, the more suitable Agile becomes. The less architecture and design decisions have to be made the better for Agile. But at the start of a brand new project, without anything established, some quite conventional up-front design and architecture is needed.


You should follow me on twitter here.