How the push to reduce story size pulls us farther from user goals and forces more up-front-design into Agile processes

It’s not so much the decrease in size that’s concerning, but the lack of practices and techniques to support this approach. And the misunderstanding from Agile practitioners dead-set against big-design-up-front that getting those little stories forces lots of up-front design.

In Agile development the user story as originally described in extreme programming is “one thing the customer wants the system to do.” Ideally it would be written on an index card. It’s definitely not a requirement – contrary to the wikipedia definition and many practitioner’s misconceptions. I see it as an item in a to-do list – a discussion to have about a need or desire a user or stakeholder of the system has. What we really intend to build isn’t written on the card. It comes from the conversation we have as a team about the story. Ideally, we’d write acceptance tests to serve as specifications and to validate that we got what we wanted.

Stories have engineering constraints built in

That user story idea sounds simple enough. Now, enter the villain in this story: engineering constraints. I started with XP back in 2000. At that time the user story had a definition like this:

Notice the engineering constraint there: “estimable between one and five ideal programming weeks.” It’s not a tough constraint – since an ideal week usually has a ratio of between 2:1 and 3:1 with a calendar week. That means we’re talking about work that a pair of developers may take as long as 15 weeks to complete. That’s up to 75 man days for those doing the math – 150 if you double it for pairing.

It’s really quite easy for someone to imagine something useful the system could do for them that could be built by a developer in that amount of time. And, it’s enough time for developers and customers to collaborate together to figure out exactly what to build, even enough time to make and reverse a few mistakes. To get that much built will likely take a fair bit of collaboration during the development iteration to decide on all the details.

Now when I cite this definition of a user story in an Agile context today, most developers are in shock. Many think I’m making it up. That’s partly why I’ve included the image of the glossary definition of Story from Beck’s 1st edition Extreme Programming Explained. It seems to be conventional wisdom today that a user story shouldn’t be bigger than about 3 ideal days. That might translate to as much as 9 man days by a pair of developers.

Size based on the time for developers to complete is an interesting constraint that may not have much to do with what the user of the software hopes to achieve. One project I worked on in the past was developing medical software. The heavy regulatory environment and large number of dependent legacy systems to integrate with meant that we couldn’t have gotten much done in a 3 day user story. So tougher development environments or more complex domains might mean that a small story gets less functional software for your customers.

Five weeks is now three days! That’s a big drop over the past 6 or 7 years. Why did that happen? And, what are the consequences?

There’s benefit in keeping stories small

I agree that a small story size is a good idea. There are lots of benefits from having smaller stories.

• From a user and usage perspective, reducing the story size forces me to understand users and usage better. It forces me to separate what really is valuable or useful to the users from what’s merely interesting.

• From a release management perspective having lots of smaller stories allows us more flexibility with planning, prioritizing, and cutting scope.

• From a development and detailed design perspective, a smaller story often implies we know more about what to build – there’s less unknown, less risk – less risk of the development work mushrooming into something much larger.

Is it these sorts of benefits that pushed stories down so much in size?

Did you catch the up front design there? Statements like “knowing more about what to build” implies that someone spent whatever time was necessary to figure out what to build.

When stories were large – estimated in weeks, and up to 5 weeks at that, not many details were resolved to give those estimates. Those details were deferred for resolution later – immediately before or during the development iteration. Developers were expected to collaborate with those in a customer role to arrive at what those details were.

Today, if we want story estimates to come in smaller, we need to figure out those details beforehand, and compartmentalize each of those details into their own user stories. That work is the design-up-front. And the push to smaller stories forces it to happen earlier – ahead of the development iteration.

By itself that’s not good or bad. What often is bad is that all the figuring out of what to build and how to decompose it often happens inside the customer team and outside of earshot of and collaboration with development. That means those in our customer role, often people unfamiliar with building software, struggle through it without practices to support them. And, the developers who get these user stories miss out on a lot of context they might have had back when stories were large and conversation was more mandatory.

The goal-task-tool dependency – how stories help users do work to reach goals

There’s a simple hierarchy I keep in mind when designing software – the goal-task-tool hierarchy. People or businesses want software to achieve some goal or objective or solve some problem. For them the software is a tool that makes the actual work done to achieve that goal or solve that problem go better. Other tools might be paper forms, telephones, or other physical things used by people in support of a process that helps them achieve their goals or eliminate their pain.

The software we’re building is a tool. The quality of a tool, from the user’s perspective, is measured based on its ability to effectively help reach their objectives.

The user story format described in Cohn’s User Stories Applied is a good one.

That story format has user, task, and goal elements in it that make me happy. And it defers deciding exactly what the tool/software is till later.

To make a story smaller, we’ve got two choices:

1. decomposing big tasks or business processes into smaller tasks or business processes
2. deciding what the software should look like to support user’s tasks or business process – designing the outside of the software – then cutting that finished design up into pieces to implement.

Both take time and expertise. And, there are risks with both approaches.

Decomposing the work

Decomposing tasks into smaller tasks means we’ve got to keep some idea of what the big picture is – what the real goals of the user are, and how the little task in this story fits into a the context of a bigger task or process. A model that relates stories together and helps people visualize that workflow is helpful here. I’ve had good luck doing that in the past using big workflow models like this and this, or writing user scenarios that spin smaller user stories into bigger stories. We could use use-cases here, or other forms of workflow modeling.

Decomposing the tool

The second approach, deciding what the application looks and behaves like early, I think is the most risky – and involves the most upfront work. In this approach we’ve had to decide up-front – and hastily at that – what tool our users need to solve their problems – then slice that tool up the way a butcher slices a side of beef. We then build each cut a piece at a time.

Unlike the butcher whose goal is to cut up meat for a lot of people’s dinner… we’re trying to reassemble a cow from its parts and hope it looks like one when we’re done. We’ve also decided the solution is a cow… not a chicken, pig, or eggplant. Our user stories become about building a cow and not about what the user’s goals are and what he’s doing. Is the user’s goal to get milk to make cheese? to eat a healthy meal at some later date? or to pull a plow? We don’t know anymore. All our stories are about chuck roasts and flank steaks. All our design was done up front – the design that’s done to figure out what product could best meet (or meat) the user’s needs.

Reducing story size benefits from adding additional practices to mitigate risk

At some time we do need to figure out what to build. But ideally we defer that time till the latest responsible moment, and we never lose sight of our users, their goals, and the work they will use this software for to meet their needs.

There are a lot of good reasons to have smaller stories.

• Do they need to all be small up front? Can we start with bigger stories – stories estimated in weeks, then decompose them as needed?

• Can we build big models to connect our stories into an understandable workflow?

• Can we build user models to help us not forget who the users are and why they use this software?

You’ll notice if you adopt these strategies you can start projects quicker because you can write fewer, bigger story cards initially and decompose as you get closer to building the software. It helps to keep track of the relationship between those big stories and the smaller stories they were decomposed to. In Cohn’s User Stories Applied and Agile Estimating and Planning he describes sort of a story decomposition hierarchy of theme -> epic -> story -> task. The important idea here is that big stories are useful sometimes, and less useful other times.

You might also notice that you won’t have to write as much on your cards since if you have a good user model with goals on it and a good task model; you won’t have to keep rewriting those same users and goals over and over on each card.

Take the approach I recommend and model users and their workflow as tasks, then design user interface support tasks as needed. But, don’t fall in love with your user interface. As you layer in tasks your user needs to accomplish, the user interface will need to change along with it.

These strategies help mitigate the risks we take on by moving to smaller stories, and help us avoid the temptation of deciding too early how the software should look and behave.