Why knowing what you want in agile development may be an impediment to getting it.

It all started with one of those weird trains of thought that come to you in the wee hours of the morning when you’re half way between asleep and awake. The first lines of the Sex Pistol’s Anarchy in the UK song were playing in my head. (This may be a hint at both my age, and the type and volume of music I listen to.) On that morning, Johnny Rotten’s words seemed particularly wise – and seemed to precisely describe a recurring problem I’ve had helping people really grok Agile development. Shortly after declaring himself an antichrist, Johnny says:

“Don’t know what I want, but I know how to get it.”

And, why this is relevant is because I constantly run into a couple problems that make my spider sense tingle. In software development, have you ever heard something like this?

“We know what we want. Can you estimate how long it will take to build?”

If you felt a shiver, that was your spider sense tingling. The other problem goes a little like this:

“We need to get these requirements nailed down before we can start development.”

In short strokes, I run into situations where people on the specifying side of software development, “customers” or “product owners” in Agile terms, either believe they know what they need, or feel they need to know before we can start development. What’s more, I still run into a number of developers in Agile environments with the same old annoying complaint about “customers not knowing what they want” or “customers always changing their mind.”

All these sentiments for me seem to spring from not knowing what “iteration” means, and is used for in Agile development.

Iterating and incrementing are separate ideas

I most often see people in Agile development use the term iteration, but really they mean increment.

By incremental development, I mean to incrementally add software a time. Each increment adds more software – sorta like adding bricks to a wall. After lots of increments, you’ve got a big wall.

By iterative development I mean that we build something, then evaluate whether it’ll work for us, then we make changes to it. We building expecting to change it. We never expected it to be right. If it was, it’s a happy accident. Because we don’t expect it to be right, we often build the least we have to to then validate whether it was the right thing to build.

I’ve used the two figures above for a number of years now to help illustrate the concept. Artists work iteratively. They often create sketches, decide to create a painting, create an under-painting showing colors and form, then eventually begin finishing the painting. They stop when it’s “good enough” or they run out of time or interest.

Paint-by-number artists work incrementally. When I was a kid I did some pretty good paint-by-number art. The problem with paint-by-number art was that some real artist had to actually paint the thing, figure out what all the colors were, then draw all the lines and number the areas – which takes more time than just creating the painting. Using a strategy of only incrementing means you more or less have to get it right the first time.

We Iterate for multiple reasons

After talking about iteration during a talk at XP Day 2007, someone correctly pointed out to me that it wasn’t as simple as “changing things” each iteration. He pointed out that:

• We iterate to find the right solution.
• Then given some good candidate solution, we might then iterate to improve a candidate solution.

We Increment for Multiple Reasons

We add to software incrementally for lots of reasons as well.

• We use incrementing to gradually build up functionality so if development takes longer than we expect, we can release what we’ve incrementally built so far. (“If” is in italics because I honestly can’t remember a project I’ve been on where development took less time than expected.)

• We release incrementally so that we actually get that business value we’re chasing. Because, we don’t really get return on investment till people begin to use the software we’ve built. Until then, the expected business value is just an estimate. And, if you think software development estimation is tough, try estimating return on investment.

We Conjoin Iteration and Incrementing

In Agile development we actually conjoin these two tactics. During a development “iteration” where we build several user stories some may be adding new functionality incrementally, others may be iterating to improve, change, or remove existing functionality.

Where things really fall apart in Agile development is when no one plans to iterate.

The Gun-shy Customer

Perhaps you’ve been on this Agile project:

Customers meet with the team and successfully write a number of user stories. After a lot of conversation between developers and customers, developers estimate the stories. Customers prioritize them, highest value first, and choose the most important stories for the first release scheduled after six iterations.

Development starts, and things seem to go very well. In the fantasy world this story occurs in, all the development estimates were accurate. In the first couple iterations all scheduled stories are finished. But, that’s where things go wrong.

After looking at the resulting software the customer says “Now that I see this, we’re missing a few things. And, although the things you’ve built meet the acceptance criteria, we, well.. uh… weren’t really sure about that acceptance criteria and now that we see it, it needs to change.”

“No problem” says the team. “Just write more stories. But, you’ll have to remove some of the others from this release in order to get them done on time.”

The customer’s shocked and angry. “What you’re saying is that I needed to get the requirements right up front! This smells just like waterfall – except without the up front time I’d need to even try to get the requirements right in the first place.”

I’ve worked with these teams and customers many times. I know of many organizations where “Agile Development” has been labeled a process that simply doesn’t work and ejected from the organization.

I know of other customers who’ve adapted by spending more and more time on requirements. They’ve introduced prolonged “Iteration 0” or “Sprint 0” phases where they actually write those big requirements. They work 1, 2, or 3 iterations ahead to really craft the details of their stories before they get introduced. They try hard to get them right. And, when inevitably they fail to get them right, they’re deflated, disillusioned, disappointed – and any other “dis” you can think of.

It’s not their fault. They were mislead.

It Doesn’t Mean What You Think It Means

There’s a nasty little phrase Agile people often use. They often say, “at the end of every iteration you’ll have potentially shippable software.” The commonly used Scrum Snowman model that all the tens of thousands of certified Scrum Masters saw clearly says that.

In the movie the Princess Bride, one of the villains exclaims “Inconceivable!” each time one of his plans is thwarted by the hero. It happens so often that one of his sidekicks says “You keep saying that word. I do not think it means what you think it means.”

“Shippable. You keep saying that word. I do not think it means what you think it means.”

For a customer, someone who intends to sell or use the software, shippable means they could actually sell and use the software. This means the minimal number of features all need to be present. The software needs to be useful for it’s intended purpose – at least as useful as the old software or paper process it replaces. The software needs look and behave well – have a high quality of fit and finish – particularly if this is commercial software and you’ve got competitors breathing down your back.

Shippable means done. Completely done and dusted. There’s no need to iterate on something done – really shippable done.

Saying “shippable” to people in the customer role means implies they’d better get the requirements right because that’s the way Agile development works.

Now, I believe Agile people had something else in mind when they said it. I think they mean keep code quality very high. Keep the code supported with unit and acceptance tests. Take steps to validate each and every user story. It tells testers to get involved earlier and more continuously. It tells developers to develop with a higher attention to quality. (Apparently developers would be developing crap otherwise?)

YAGRI: You Ain’t Gunna Release It

I propose we, the Agile community, clarify what we mean by iterative and incremental. We need to explain to those in customer and product owner role that it’s important to write user stories that they don’t intend to release. To write stories that they intend to evaluate, learn from, improve, or toss out as a failed experiment.

In conversations with my friend Alistair, he proposed writing three story cards instead of just one. The first story card has the actual story on it. The second one is a placeholder for the inevitable changes to the story after we see it. The third for the fine-tuning after we see those changes.

This is an example of planning to iterate. It would relieve a lot of stress from the hand-wringing gun-shy customer worried about getting it right because the story needs to be “shippable.”

You can always get what you want, but is it what you need?

Where we can apply Sex Pistols lyrics to software development, we can’t necessarily apply the Rolling Stones.

“You can’t always get what you want. But if you try sometime, you just might find, you get what you need.”

In software development, sadly if you specify something, and everyone is doing their best, you’ll get what you want – at least what you specified. But, is it what you need? You’ll only know after you look at it and try it.

Don’t listen to Mick.

In fact, try hard to not be too sure about what you want. If you leverage iteration, you’ll get it even if you didn’t know what it was to start with. Johnny’s got it right.

“Don’t know what I want, but I know how to get it.”

Please leverage the explanation if you’d like

This is a bit of the story I told during my XP Day 2007 talk Embrace Uncertainty. It’s rare when you get to quote Johnny Rotten, Roger Waters, Paul Simon, Pete Townsend, John Lennon, and the Spice Girls in the same talk.

Feel free to download the talk.

Feel free to use the examples using a creative commons license. Let people know you borrowed them from me.

If you’d just like the Mona Lisa slides, you can grab those here.

The general ideas here are written in a StickyMinds.com article with a little less ranting. You might share that version with your boss.

Stay tuned

If you want to get more about specific strategies for iterating sensibly in Agile development, please visit me at a tutorial I’ll be teaching at a conference. Also pay attention to this site and blog as I resurrect my long overdue book from it’s current purgatory.

Finally, if you’ve read this blog in ThoughtBlogs (and my web analytics tell me most of you do) this may be the last time my blog appears there. Please subscribe directly, or look for me on ThoughtWorks alumni blogs. I’ve had a great time at ThoughtWorks for the last several years, but it’s time to set out on my own.

Thanks for reading.

What Now?

Thanks for reading this blog/essay/rant! I’d love to hear from you, so please rate this entry, and consider leaving a comment. Or, send email to me directly.

If this blog entry was interesting, please take a look at all blog entries

If you find my brand of thinking valuable, please consider visiting me at a conference, or public class. Or feel free to contact me directly for agile coaching and training.