What the product owner needs to worry about isn’t in the product backlog

If you’ve read one of my blog essays before, you know this isn’t going to be a quick thought, but a bit of a long discussion (read “discussion” as “rant”).

This particular discussion describes what I see as a problem in agile development – sort of a practice vacuum. One of the most difficult roles in an agile process is the product owner or agile customer. In this essay I describe the problem as I see it, and sketch what I see as a mental model and some practices that product owners can use to begin to fill that skills vacuum.

Warning, it’s a bit of a long rant. Go grab a cup of coffee and get settled in.

It’s also a thinly veiled attempt to explain things in a way that gets people interested in attending one of my classes, workshops or talks at conferences this year. My mission this year is to improve software development practice for those in the product owner role.

The XP customer is always right

I’ve been involved with agile development since my first XP project in 2000 where a dream team supported by Kent Beck helped get XP going in a San Francisco startup. I started at the company as a member of the XP customer team. I struggled with that role – felt lost. I had deep subject matter expertise, so I guess I belonged there, but the lack of clear and compelling vision for the product drove me crazy. I left work every day ready to scream.

In my past jobs I had been a product manager – been able to declare what the product should be. I was a prima donna, and the company I worked for supported me well since my products were successful. In this XP project I was part of a collaborative team of prima donnas. If that sounds like an oxymoron, it’s because it is. The process for customer seemed to be: Prima donnas fight, alpha prima donna wins, others shut up, then customer(s), speaking with one voice, tell the developers what to build.

My feelings were mixed. I loved XP – loved the productivity and energy I saw in this approach, but felt the customer side of things was a mess. The team of customers, although they had deep subject matter expertise and experience, had no clear vision or discipline in their approach to figuring out what to build. In truth, as a customer on that team, neither did I. I completely sucked. I saw that the alpha prima donna process approach didn’t work. There had to be something better.

In my past life I was also an engineer and UI designer. (note to readers, engineer with graphic arts background = UI designer.) I hated that XP didn’t let me be everything – subject matter expert, product manager, UI designer, and engineer. Happily for me, just before leaving frustrated, Rob – the XP coach, and one of the smartest people I’ve ever met, let me change my role to developer. I loved being a developer on the team. I loved the new skills I was learning, the increased discipline, and rigor I’d not known to apply to engineering before. I’d been using the loudest prima donna approach there as well – mostly.

I happily served my time on this XP project as a developer. Although I felt the product concept and release strategy was flawed, it wasn’t my problem, the hole was in their side of the boat.

That boat eventually sunk – or more accurately took on lots of water and limped to shore in Greenland with most employees thrown overboard, and the last remaining employees stock options in ruin.

I learned some fabulous lessons, and was left with a few problems to solve.

1. Keep doing agile. XP and Agile methods brought a level of rigor, expertise, and predictability to software development that I’d never seen before – that along with a more satisfying collaborative environment for the engineers at least. Embedded in XP-style engineering was a culture of craft, of continuously striving to increase skill.
2. The XP customer role needed help. I now understand that role to be the most critical – like the captain of a ship. That captain can drive it into an iceberg if he wants – happily smiling as it takes on water. The team does what he asks, sometimes oblivious to the fact that they’re in the boat too. After all, the customer is always right.
3. Find effective agile customer practice. I must figure out something to put the same level of rigor into that customer practice as I saw in XP engineering practice – at least if I was going to stay in an agile world.

Glad I got all that off my chest. Thanks for listening.

It’s the product, stupid

For those who didn’t know, there’s been a slow moving and quiet agile process brand war. Scrum has overtaken XP as the dominant approach. And much like Christianity absorbed Pagan holidays to help it spread through Europe, Scrum has relaxed its one calendar month sprint rule I’d originally learned for shorter XP-style 2-week sprints, and adopted XP’s user stories to fill its backlog. Scrum trainers have begun to gather and teach some of the best practices originally found in many other agile approaches at the beginning of this decade. Luminaries from other approaches (like Alistair Cockburn and Ron Jeffries) are joining everyone under the ever widening Scrumbrella. Me too. I want to be like Mike.

One of the concepts I particularly like in Scrum is the product owner role.

The role name “Customer” has always troubled me. “Customer” is the relationship between people or groups. To the engineers, people asking for software to be built are “customers.” To a company who builds a product, the people who buy the software are their customers. And for many products, the people who buy the product aren’t actually the people who use it.

In software development there are people who make decisions about what the product should do while taking into account what people who make buying decisions want, what people who use the product what, what stakeholders who pay for the software to be built want, and what others who maintain and support the product want. Sound tough? It is.

But it’s all about the product, and the very tough decisions someone needs to make – decisions rife with uncomfortable tradeoffs, risk, and uncertainty. By using the term product owner Scrum acknowledges that there is a product here, a piece of software, and someone with the tough job of “owning” decisions about that product. It’s a more accurate than customer.

Product owners learn lots of agile process but scant product design and product management practice

I often quote the simple distillation my friend Alistair made when talking about requirements.

Product owners have a lot of design decisions to make. They’re responsible for the success of the product. It seems appropriate that we leverage valuable design practice that’s already out there.

It’s unfortunate that much of the training and support available for those in the product ownership role doesn’t teach much in the way of design practice. It’s getting a bit better, but it has a ways to go. Until then, what passes as product owner indoctrination is lots of instruction on working inside a Scrum and agile process. That’s important – but it leaves product owners with the same problem I first encountered in 2000 with XP.

We need a hero?

I find lots of product owners leading with pure intuition. Not that intuition doesn’t work – it’s just unpredictable. It works fabulously well for a select few with strong experience in and understanding of their domain, their market, and a sixth sense about what that market needs.

The ideal Scrum product owner or agile customer is often described as super-human. They have a strong understanding of their customer, their users, deep domain expertise, and the time and willingness to work directly with the development team and write user stories that describe the vision of the product they hold in their mind. They can motivate others to see that vision, and to work hard to help achieve it.

I don’t know about you, but I’m imagining someone wearing spandex and a cape.

Those in the engineering and testing team aren’t expected to have such super powers. Instead they rely on learning lots of new skills. Armed with those skills, they build ever more capable tools that help them write better code, and validate it quickly and repeatably. They respond quickly, roll with change, and produce high quality software. When you see a good agile team at work, they actually look superhuman – but they’re not. They’re just good.

Batman is armed with brains, skills, and a really cool utility belt. Superman just came from another planet where compared to earthlings, he has superpowers.

So for those in the product ownership role, and for companies wishing to use agile development, you have a choice. You need a hero. You can search for that Superman product owner, or you can take the smart people you have, give them the skills, and help them identify the tools that make them look and act superhuman.

That’s my point.

There’s a frame of mind I bring to agile and Scrum product ownership -’ a mental model that I borrow from the design community that gives me the thinking structure to learn new skills and build new tools that allow me to look superhuman at times. This is a toolset built over the last eight years working in agile development and collecting techniques and experience from lots of other smart folks.

So here’s the mental model.

You’ll need more than a backlog

As a product owner, you’re often pushed into battle armed with only a backlog full of stuff to build. While that’s a necessary tool needed to communicate with the team, you’ll need more.

Building the backlog from XP style user stories became a Scrum practice early in this decade. Mike Cohn wrote the user story bible, and it’s a good one. In it he described the now popular user story format:

Writing stories in that format gives us some clues as to what’s important to know: our users, what they want to do, and what goals they have or benefits they home to receive.

A nasty thing happens when you write a bunch of these stories. You end up repeating the user part over and over – many people get tired of thinking about it and just start writing “as a user” – which is as good as writing nothing at all. Many folks have a hard time with the “so that” part – in that at times it seems obvious or unimportant – especially for very small stories about pushing buttons, or ensuring a phone number contains digits and not letters. Really – do we need to justify why phone numbers should be numbers and not letters?

The database guy in me gets twitchy writing these sorts of backlogs. They’re denormalized – which is a geeky way of saying we end up repeating the same stuff over and over and run the risk of being inconsistent as we do.

And, while each individual story makes some sense, when confronted with a backlog full of stories written this way I can’t make sense of what the software is supposed to about. It feels a bit like the story of the blind men and the elephant – this user story is about a rope, and this one is about a tree trunk, this one’s about a snake, and that one about a wall. How does the reader get an elephant out of that?

So let’s start from the beginning.

Identify your users.

No really. Name the kinds of people who use your system. And don’t stop there – gather details about those users relevant to your product design, details like how much subject matter expertise they have, how much they know about computers, what they’re doing when they’re using the system – if they’re likely to be interrupted or distracted, and most importantly why they use the system. What benefit do they gain for using your software? How do they define success? There’s lots more valuable information – stuff relevant to how you build your system.

And, if you don’t know the answers to some of these question, go out and meet your users, spend time with them, and make good notes about what you hear and observe. Take that information back and use it to fatten up your user information. Distill all that into something like a persona to keep it straight in your head, and help explain it to others. It’s important to remember that you’re not doing this for fun. You should be able to easily identify characteristics of your users, their goals, and use that directly affect the decisions made in your product.

But, don’t stop there.

For each type of user, describe their use

You’ll need to understand your user’s likely “use” of the tool you’re designing. Start with the way they do things currently today using some other product, your old product, or some combination of other tools – some electronic and some manual. You’ll use this to understand how to create a product that meets their goals, works within their knowledge and skill constraints, and lets them accomplish what they need to.

Identify your customers

Users are great… but let’s face it, it’s about the money. If this is a commercial product, someone will likely buy it. These are your customers. Don’t mix them up with your users – even if they’re the same people. The thought process we go through to make a buying or product adoption decision is a different thing than our day-to-day use. I ranted about this in my old entry designing software for two headed people.

You’ll need to distill the most relevant bits about your customers into profiles or personas. Understanding your different types of customers and what motivates them to buy is often called market segmentation. If you’re building a commercial product, and don’t segment your market, you run the risk of satisfying no one. Alan Cooper describes the approach of honoring and acting on all customers input equally as “customer driven death spiral.”

Identify your organization’s goals and your resulting product goals

At the end of the day, if you’re a product owner, you work for a company that’s paying your salary, and for the construction of the software. Why are they doing this? What do they hope to gain? What organizational goals does building this software help realize?

The trick when setting goals for your product is to understand the difference between output, and outcome. Good product goals point to the intended outcome.

• Output is the software built, the stuff you ship.
• Outcome is what you get after the software ships, sometimes quite a while after. It’s the financial benefit, the increase in customer satisfaction, the decrease in customer service calls, or the improvement in efficiency.

No one really wants a few hundred more megabytes of code on a hard drive some place. What we really want is the benefit. It’s the product owner’s job to know what that benefit is, make sure everyone else does, and make sure all the software we build somehow increases the odds of getting it.

Pause and answer this question: For the product you’re working on right now, how will your company benefit after the product has shipped? If you understand how, do others?

Prioritize before you prioritize

There’s an important dependency chain here that product owners need to understand.

This is the sing-a-long part of the essay. Sing the following to the melody of “There was an old lady who swallowed a fly”:

• We wrote the story to build the software,
• We built the software to support the use,
• We supported the use to help the user,
• We helped the user to earn benefit for the customer,
• We benefited the customer to get the sale,
• We wanted the sale to support the business strategy,
• But we don’t know why we adopted the strategy, perhaps we should.

OK, it’s a bit of a leap, but I want you to see the dependency here. The minute you lose sight of it, you’ve lost the context you need to make good decisions as a product owner.

Given a business strategy, it helps us choose the most important customer segments to focus on. That’s not trivial – and takes understanding of who possible customer segments are, what they need, what our organization capabilities are, and what we as an organization want to be. I smell a hedgehog here someplace.

Given most important customer segments, we can look at their possible users and uses – and select the most critical users to support to help us earn and keep customers.

For those important users, we can look at their goals and possible uses and prioritize high the uses that help them meet their goals.

In an ideal world the user stories we write are about users and their use.

As a product owner, you need to understand all these things, before you can effectively build and prioritize a user story backlog. And that’s not all. Depending on your industry there may be regulatory concerns that come into play, SOx requirements, architectural concerns, or the CEOs latest passion – all of these factors may affect priority.

I find this isn’t commonly understood by product owners, or those who teach them. I find that even when understood, product owners lack the skills to capture, distill, leverage, and communicate this information.

The product-shaped hole

The interesting thing about business strategy, customers, users, use, and all those other possible constraints are that they don’t actually say what software to build – not directly anyway.

When I think of all this contextual information, in my head I see pieces coming together and filling in – building sort of a fabric… and in that fabric is a hole – a missing piece shaped exactly like the product I need to build – a product that fits perfectly into that context.

For me, my job as a product owner is to find that product-shaped hole.

Finding the product-shaped hole is: understanding all of the relevant context that helps me shape the product to build. The trick becomes not trying to understand too much – just the stuff that’s most relevant to the decisions we need to make so we can begin making them confidently. (It’s easy to get lost in irrelevant context. Try not to do that.)

Some get frustrated talking about users, and customers. For them it’s more interesting to leap straight to talking about the software we’d like to build. I don’t blame them. Talking about software is fun. And for me at least, building it is even more fun. But, when we don’t know what features to include, because we’re not sure who our users are, how they intend to use the product, or if they’re part of our target market or not, the easiest thing to do is put everything into the backlog that anyone asks for. Then we’re left with the daunting job of prioritizing this mess.

Sketching the solution using release planning and envisioning

For me the problem shaped hole starts by understanding business strategy as some concisely distilled product goals. Good goals describe what the desired benefit is after the product ships. Even better goals are supported by metrics that help to measure that benefit.

Simple personas are a good way to distill users, details about them that affect your product design, and the goals that motivate them to use your software and consider that use a success. You can do the same thing for you customers as well – those who make the buying decisions.

Finally, I model use a bit abstractly, thinking both about what users are trying to accomplish, and some details about how they might do it. This work is done using user stories shaped into a story map that helps me, as a product owner, understand the products use – the full business process – and easily explain it to others to boot.

Now I can begin to fill the hole by planning incremental releases of a product that fits tightly within that context, no more, no less. I can slice a story map with confidence that I’m considering all relevant users, the whole business process, and the entire value stream.

To be more confident my thin slice is a good one, I write user scenarios or “blockbuster stories” – longer narratives that cut through the entire release and demonstrate what a day in the life of my users and business will look like when the software is released. It’s a good way to be sure I’m building a release that really delivers value.

And, since a picture is worth a thousand words, now’s a good time to sketch the user experience as a simple sketched story board that describes the users’ journeys through the system from beginning to end, showing a bit of what the UI could look like along the way. Users, business people, testers, and engineers, all begin to “see” what the software could look like. All begin to develop a common shared vision of what the product could be.

Each planned incremental release is a prospective solution. Releasing it, getting to users is a chance to really be sure it’s a genuine solution, as well as to begin the flow of value received from using the product.

But, if you’ve worked in agile development before, you know the tough work has just begun. You’ve got a lot of detailed decisions to make to fill in your vision. There is a lot to learn along the way. It’ll take skill and bag of good techniques and tricks to help you sprint to delivery.

Sprint to release to fill the hole

The Scrum model starts the sprint with a sprint planning meeting. But, for a product owner, your sprint planning started days before – maybe longer.

As a product owner you’ve been working with your backlog identifying the most important stories to work on first. For those most important stories, you’ve split them down into the smallest valuable pieces. You’ve discussed them with users, UI designers, subject matter experts and others that can help you fill in the details – supply you with the context you need to make detailed decisions about the software.

If you’re an interaction designer you’ve leveraged your story map to sketch high level interaction design and navigation, and you’ll leverage that high-level design thinking to do more detailed UI design that supports the stories in the next sprint. Ideally you’ll find time to prototype and test the UI. (See the salesforce.com example for black belt design and testing.)

When the sprint planning meeting does come you’ll need to be ready to discuss the story, acceptance criteria and support the team as the plan the details of how to build software during the next sprint.

You’ll need to be available to answer questions and help keep development and testing moving smoothly. You’re doing this while planning and doing more detailed research to support upcoming sprints.

After a few sprints go by, you’ll have a the beginning of a product that if you can’t ship it, you’ll ideally be taking out to customers to test it – to validate it with them, to help gain confidence that product you’re building will get used.

 

 

 

 

 

Sprinting towards release isn’t simple. There’s a bit of strategy to determining what stories help you see the whole shape of the software as soon as possible – the functional walking skeleton. This first earliest version helps you confirm early that you’ve found all the necessary functionality for the software – early enough to course-correct if you haven’t.

As your grow the software iteratively and incrementally, you’ll see it begin to tighten up – move closer to releasable. While it’s true that the Scrum calls for “potentially shippable software” at the end of each release, it’ll be your job to assess releasability each sprint. You’ll need to communicate to the team where the product is, how close to releasable it is, and if it isn’t what areas of the software need the most attention.

I, and others in the scrum community, have begun to use a chess metaphor to describe an opening, midgame, and endgame when sprinting towards release. Different strategies are applicable at different times. Start by laying functional foundation; continue filling in and building to full functionality while vetting with users; finish by refining the software while validating tougher stuff like performance and scalability. While each sprint is important, as a product owner, you’ll need to keep your eye on the product release.

Land and expand

Once the product has released, that existing product becomes part of your context. You’ll need to begin paying attention to what your customers and users do with the product. Do they behave as you expected? Are you getting that outcome, that business value you hoped for?

You’ll need to begin balancing inevitable requests from users for changes to the product with your future plans for releases that further your company’s strategy or help you pursue new target markets. And, I’m sure it won’t happen to you, but sometimes products have bugs that need to be fixed with smaller short releases that interrupt longer running releases that add more functionality.

If it seemed challenging getting to that first release, you’re just getting started.

Telescope and Microscope – strategic and tactical product owners

For larger products and in larger companies I often see product owners that play a more strategic role. At a workshop for product managers, I heard Luke Hohman say:

“the only true thing we can say about strategic and tactical in a business context is that one refers to the long term, and one to the near term.”

Strategic product owners tend to be looking forward into the future, forward towards new markets, next problems, and next releases. Strategic product owners tend to focus outwards in the company towards the executive team.

Tactical product owners tend to focus on the current release in progress, working hard with the team to make sure the product goes out on time, is as valuable as it can be, and is aligned with the business outcomes targeted for the release – or better outcomes if they’ve been identified.

Guy Kawasaki has written and spoken about using microscope thinking to focus on important details today, and telescope thinking to look into the future – to bring the future close in. Some product owners move nimbly between telescope and microscope. In some organizations the responsibility is split among multiple people.

Rethinking product ownership as product design

Effective product ownership requires understanding and synthesizing lots of context. A good product owner is a business advocate, a customer advocate, a subject matter expert, a user experience expert, an engineer, a top notch communicator, and a visionary leader. That sounds like a bit of super-hero to me. In the absence of a super hero, you’ll need to build skills, to build a team with skills.

Product owners are decision makers. Design is a discipline focused on making decisions based relevant context, iteratively refining decisions, and validating product decisions with those that will actually use the product.

This is my wrap-up, and it’s sounding a bit like a call to arms. And, possibly it is. I started this story with concerns about agile customering or product ownership – concerns that common agile practice didn’t offer much for the unlucky person saddled with the role. But, the truth is it’s getting better. As agile development and Scrum grow in adoption, more practitioners, smart practitioners, are learning more, are evolving useful practice for product owners.

With this essay, I hoped to point out the problem, and lay down a bit of structure for the solution. Through the year this year, I’ll be working to co-teach Certified Scrum Product Ownership courses with friends who currently teach certified Scrum courses. I’ll also be teaching dedicated Agile Product Design courses for those who already feel they’ve got the basics of agile development under their belt, and wish to add some skills to those in product ownership, business analysis, and user experience design. These courses will help those in agile coaching roles to add more tactics to their portfolios to better help their customers. I’d love to see you at a class.

If you’re currently a product owner, and you’ve developed practice that’s working effectively for you, I’d love to hear from you. My personal goal is to help improve software development generally. I’m actively seeking practices that I, and others, could use to improve life for those in the product ownership role.

Quoting Fred Brooks: “The hardest single part of building a software system is deciding precisely what to build.” In agile development, much of that deciding responsibility falls to product owners. It’s the essential difficulty that Fred describes in his No Silver Bullet essay. This is an area ripe for innovation and practice improvement.