As has been mentioned before, many of these Agile ideas are just common sense. This is often pointed out by its critics, but is that really a bad thing? Since common sense is not as common as we would like it to be, Agile frameworks and methodologies codify their practices in rather specific rules on how to do things. Basically, these rules are designed to help (force?) a team to adopt a lot of common sense practices, which in traditional software engineering may have been forgotten. I have to say, I feel like I actually 'get' many of them and I might talk about some specifics here in future blog entries.
In the meantime, though, I'm still trying to wrap my head around one particular issue: Agile seems to work best - in fact it requires - that there are no 'silos' in the engineering organization: Particular knowledge should not be locked up in the head of one developer or a small group of them. Instead, knowledge should be shared across the team and some Agile practices, such as pair programming, for example, are specifically designed to address this. But in same cases those silos are unavoidable. Read on for an example of that. And in the moment you have knowledge silos in your team, many of the benefits of Agile seem to just evaporate.
Let me explain what I mean. For truly Agile development it must be possible to assign user stories and their related tasks to almost any team member. At least, there should be more than one team member who's able to take on a specific task.
If you have a team like this it allows a number of interesting things to happen:
- Once a developer is done with a particular task they can merely grab the next highest priority item from the backlog. This makes it possible for the team to really deliver higher-priority items in order, thus producing most value to the product owner in each sprint.
- You can use story point estimating, maybe utilizing surprising techniques such as planning poker, and it will actually make sense: Over the course of several sprints you get a feeling for the velocity of the team and even each individual developer. The nice thing about estimating in story points is that it is self-tuning and doesn't depend on the experience level of the individual developer. After a while you will know that John - a senior developer - can do about 20 story points per sprint, while Bob - fresh from university - may be able to do only 10. So, you can re-assign user stories without having to re-estimate them - well, you need to re-esitmate the associated tasks, but that is easier with these metrics - and still quickly get a good idea of how your team will be able to do in the sprint.
- Every developer can just jump in and help others in the team. That is of course a given and is helpful in every kind of team, Agile or not. If a staff member goes on vacation or is sick the work can be picked up by whoever is available (see also point 1). Likewise, if a developer is held up with a previous task for longer than expected then the next task that he was supposed to tackle doesn't have to wait. If someone else is free or doesn't have a higher priority item to work on, that is.
But now imagine a different team in a different company and suddenly we encounter trouble: Assume you are in a small startup, which doesn't just devlop a web-app, or work with a single framework or environment. Assume you are working on a project that hacks the Linux kernel for some innovative network monitoring solution, produces a lot of data that needs to be sliced and diced and requires a web-app to drive an interactive front end. You are a small startup with some angel investment, so you get yourself the smallest team you need to get the job done: An experienced networking/kernel hacker, a database expert, someone who knows Django or TurboGears (or a similar web-app framework), and a user-interface person taking care of the HTML/JS/CSS for the front-end.
So, that's your team. Now imagine a planning poker session for a specific feature: "Ok, on the count of three: What's your estimate for hacking the ipchains kernel module to add feature XYZ?" Guess who is going to put down a card with a number and who is going to put down a card with a question mark (meaning: "Don't know...")? Well, the kernel person may have a decent estimate, while everyone else - those who never hacked around in the kernel before - realistically will have no idea how complex that task is! You have just one person on the team who can do the job and can provide estimates. You may just as well estimate in hours then, and you will be back at much more traditional planning. You need to define exactly ahead of time who will do which tasks, since each person is so specialized that in effect they are the only ones being able to estimate and do it. Say hello to your Gantt charts.
I think this is an example where some of the best aspects of Agile seem to fall flat.
Maybe you will say: Well, you shouldn't have teams like this anyway, you should pair-program to spread the knowledge, you should assemble your team so that you don't have this formation of knowledge silos.
To those arguments I will have to say: VCs don't have a lot of patience and they don't hand you a lot of money to start with. You can only get a small team, and you can't turn an HTML/CSS/JS person into a Linux kernel expert just like that. Especially not when you deal with a very tight deadline and budget. You can try, but then who's doing the front end work, while you are sending that person through the Linux kernel 'apprenticeship'? No, you need subject matter experts that are as efficient as they can be and can crank out a piece of functionality with the least amount of fuss and delay. And all you can afford at that stage is one expert for each subject area.
If you have a team of experts, each expertly knowledgeabe in their own area with very little overlap: Does Agile still make sense?
As I said at the beginning, I can see a lot of value in many of the Agile practices and ideas. Yes, it's basically common sense, but I understand how by codifying it you can bring a lot of this common sense back into engineering organizations. But while I have learned to appreciate a lot about Agile since my [previous post], I have to stand by the assertion I made then: Agile works in some teams and environments, but there are also some cases where it simply makes much less sense. Some of its aspects might still be useful, but under some circumstances it loses a lot of value to the point where you have to ask yourself: Why bother with Agile at all then?
Agile seems to make sense only once your team can grow in size - or alternatively in the spread of knowledge across the team - to a point where most tasks could be effectively tackled by more than just one person on your team. If your work team consists of generalists, and your user stories and tasks are suitable to be tackled by those generalists. Note, with 'generalist' here I mean someone who is able to deal with all the aspects of your project. I'm not advocating that developers shouldn't have speciality areas.
This is just my impression. If you have experience working in Agile teams that just consist of a small group of specific and distinct subject area experts, please comment and let me know how you handled this problem. I would love to know.
In the meantime, I can imagine that non-homogeneous teams, with too many knowledge silos, may be the reason for many failed Agile projects: The team thinks it's agile, but when push comes to shove, they realize that actually they have been hand-cuffed by the insufficiently spread knowledge in the team. After a few sprints the situation may improve, but for starters?
Agile is about common sense, and there seems to be a very important common sense item that needs to be considered: Does Agile even make sense for my team and project? The answer is not always yes, no matter what Agile-enthusiasts may say. In reality (clue common sense) the answer shouldn't come as a surprise: It depends!
You should follow me on twitter here.
You can't teach an old dog new tricks.
ReplyDeleteIn the end, this has been the root cause of impediments to agile adoption that I've personally dealt with (and really, any other major procedural change in an organization.)
You simply can't take a developer who has been doing what she does (successfully, even if only in her own mind) for a decade, turn the process on its ear overnight, and expect her to just fall in line.
Some of the agile ideas still work, like iterative development, even with highly specialized development.
ReplyDeleteBut yeah, specialists like me don't fit too well into this.
The solution I've seen is for people not hire a db expert until the database grinds to a halt or data starts mysteriously disappearing. Not a great path to successes.
While it is ideal that over time that more people on the team have the flexibility and knowledge to take on whatever is of highest value, agile can still (and has shown) to work.
ReplyDeleteIf you look at Scrum, it recommends that teams are 7 +/- 2 people. It doesn't say that each of those people are generalists. In fact, chances are that there is much specialization. The key is that you have the right combination of people on the team that can accomplish the work.
Those are the dedicated resources, you may also have shared resources that have high level of specialization that their time is spread across more than one team (like architect, or usability expert). Those people should be rare, and you should strive towards most team members focused on one team.
Regarding Planning Poker, I find that while the sizing is important, it's the sharing of knowledge through conversation that is much more crucial. Over time, both subject domain and technical knowledge can be gained. With pairing, that can extend to specific technical skills.
For any size company, it is extremely risky to have specialists. Should those people leave the company, you lose a lot of knowledge. Agile reduces risks by minimizing the impact of such a loss over time.
I think the key is to use common sense. If I've never hacked the linux kernel I'm going to defer to the kernel specialist on estimates and I'm not going to take a kernel related task without pairing with the specialist. As time goes on, I'll be able to do more and more of the kernel work myself.
ReplyDeleteSame with database related tasks. I'll design schemas, optimize queries, etc., but I'll defer to the DB admin when it comes to figuring out db segments, replication etc.
From a business perspective, I'd want to know that the success of the business is NOT hanging on one guy who might just win the lottery, get hit by a bus, or just might not be that good anyway.
I agree with your original comment that you shouldnt have only 1 person who can do a task.
ReplyDeleteIf you have 1 DB guy, 1 kernal guy, and 1 UI guy or something you have no oversite.
Everything will work well at first until someones contract is up and they leave and you think they are "awesome" and the new guy comes in and goes WTF and throws away all of his code.
If you want maintainability you need more than 1 dev who can understand a section. Most devs can learn quickly.
The key features of Agile are Continuous Intergration, Work Items in order, and Unit Test Coverage.
You can do this even if people are specialized but letting people code something because they are highly paid and "experts" doesnt mean that they dont get lazy or code well.
The best code can be picked up and edited by anyone with enough unit test coverage to know if you broke anything.
Here is a good response to your post in form of a comment on Hacker News.
ReplyDeletehttp://news.ycombinator.com/item?id=729107
Nice article!
ReplyDeleteI would say having silos of knowledge driving projects introduces higher levels of risk (what if the person leaves or is ill?). You would have to deal with these problems in any methodology but the nice thing about agile is it helps to bring down the silos. So your team is not as effective as cowboy specialists but the business gets a bit of safety in the broadening of knowledge. I would consider XP instead of scrum in the scenario suggested as Pair programming will help to bring down the silos faster. Waterfall would be a disaster.
So Agile may not be suited but what would be better? Sometimes projects are just risky!
@Anonymous: I think we shouldn't make the mistake and assume that 'waterfall' is the only other option here. So far, my study of Agile processes is purely theoretical, but on the other hand, I've never been in a 'waterfall' project in my entire life. And I've been in a lot of projects.
ReplyDeleteAnd I don't know how one can chose XP over Scrum, considering that the former is a development methodology and the latter is a project management framework. I don't think one of them replaces the other, since they are different things. One can combine the two, however.
The problem I am describing is that it reduces some of the core values that are proposed by Agile methods.
@Skip Angel: But aren't the Agile proponents themselves saying that you can't just pick and chose: If you only implement a little of Agile then you aren't really Agile? I've read that more than once.
ReplyDeleteYou are saying that "Agile reduces risk", even if you can't do the Agile planning aspect of things. But you say that "the sharing of knowledge through conversation that is much more crucial". While I agree that the sharing-practices in Agile are some of the most attractive things, it doesn't mean that you can just call yourself Agile because you communicate well. It may result in a lot of agility (lower case 'a'), but if you look at the Agile descriptions, you need to implement many more of the 'Agile practices', before you are 'allowed' to call yourself 'Agile'.
Brendel,
ReplyDeleteAgile is not an end state, but a journey. While I believe that you should try to apply Scrum as it is a framework, you need to expand on it as you inspect and adapt. For many, that includes applying the XP practices, Lean and other methods as the organization is ready for the change. There is no such thing as a big bang approach without a lot of problems.
In the end, I find myself going back to the Agile manifesto's value and principles as the test of agility. If the team is figuring out the best way to adhere to these values and principles, which may be a combination of "methodologies" or even some they come up with themselves (I am a firm believer that there are new agile techniques being learned daily) -- then they are moving in the right direction.