Chapter 5: Requirements and User Stories

This chapter discusses how Scrum projects handle requirements, nonfunctional requirements, and knowledge-acquisition work. It also describes the role of user stories and how they can represent business value at multiple levels of abstraction, how to determine when user stories are good, and two techniques for gathering user stories.

Contents Overview

A Scrum project treats requirements very differently than sequential product development does, as shown in the following table.

In place of detailed upfront requirements, Scrum uses placeholders for requirements called product backlog items (PBIs), which are discussed and progressively refined throughout the project. Each product backlog item represents desirable business value. While there is no standard format for a PBI, the most common approach is user stories. 

Using Conversations

Requirements are intended to facilitate a shared understanding of what needs to be built. Though written requirements can be helpful, they are easily misunderstood. Timely conversations between the people who understand what should be created (stakeholders and product owner) and the people who have to create it (the development team) are a much more effective way to ensure that the desired features are being built. Scrum projects leverages real-time, in-person conversation as an essential tool for ensuring that requirements are properly discussed and communicated. Conversation, however, is only a tool. It doesn't replace documents completely. Instead, the product backlog is a living artifact, available at all time during product development. Those who still want or must have a requirements specification document can create one at any time, simply by collecting the product backlog items and all of their associated details into a formal document.

Progressive Refinement

With sequential product development, all requirements must be at the same level of detail at the same time. This has several disadvantages:

• Errors. Details are predicted at a time when the least amount is known about the product
• Waste. Because all requirements must be equally detailed, regardless of their relative priority, projects might overinvest in creating some requirements that will never get built.
• Expense. A large inventory of requirements is created upfront that will likely need to be discarded or reworked later when things change.
• Rework. The mistaken belief that the requirements are complete curtails future discussions, leading to unnecessary misunderstandings and rework.

With Scrum, the requirements that are higher in priority will be built sooner, and so these soon-to-be built features are broken down into a collection of smaller and more detailed items compared with features that won't be worked on for some time, which are left larger and less detailed. Scrum projects employ a strategy of progressive refinement to disaggregate, in a just-in-time fashion, large lightly detailed requirements into a set of smaller, more detailed items.

What Are User Stories?

User stories are a convenient format for expressing the desired business value for many types of product backlog items, especially features. The typical format is to specify a class of users (the user role), what the class of users wants to achieve (the goal), and why they want to achieve it (the benefit). 

I like user stories and use them as a central placeholder to which I will attach any other information that I think is relevant and helpful for detailing a requirement. I do not, however, always write user stories, especially in situations where it seems forced, such as representing certain defects. 

Ron Jeffries offers a simple yet effective way to think about user stories. He calls it the three Cs: card, conversation, and confirmation.

• Card
  People originally wrote user stories on 3X5-inch index cards or sticky notes. Nowadays, though many still use physical cards, others prefer an electronic tool. Either way, keep the card deliberately small so as to limit the amount of written detail. Remember that they are only intended to capture the essence of the requirement, with the details to emerge in future conversations. 

• Conversation 
  One of the benefits of user stories is that they shift some of the focus away from writing and onto conversations. These conversations enable a richer form of exchanging information and collaborating to ensure that the correct requirements are expressed and understood by everyone. Though these are often verbal discussions, they can be supplemented with documents, such as user interface sketches or an elaboration of business rules. 

• Confirmation
  
  A user story also contains confirmation information in the form of conditions of satisfaction, which are often listed on the back of the physical card. Conditions of satisfaction are the acceptance criteria that clarify the desired behavior. They can be expressed as high-level acceptance tests—but are by no means the full battery of tests that a team might need to run to verify a story. The conditions of satisfaction are used by the development team to better understand what to build and test and by the product owner to confirm that the user story has been implemented correctly. 

Level of Detail

User stories can (and should) be written to capture customer and user needs at various levels of abstraction. The figure below depicts these multiple levels, which range from epics to features to stories (see Chapter 17). In general, epics give a big picture, high-level overview of functionality, features are more detailed than epics but still too large to be implemented, and stories are small enough to be completed in a sprint. Some teams also use the term theme to refer to a collection of related stories, such as stories that all belong in the same functional area. Though not included in a product backlog, tasks are a layer below stories. Tasks are typically worked on only by one person or pair, which can often be measured in hours. Tasks differ from stories in that they describe how to build something instead of what to build. As such, stories should avoid task-level details. 

Note that these are not universally accepted terms, but instead general descriptions of the various story sizes included in a typical product backlog. For more information, check out my blog entry, “Demystifying Product Backlog Concepts.”

INVEST in Good Stories

Bill Wake offers six criteria, summarized by the acronym INVEST, that can help when evaluating user stories.

• Independent
  As much as practical, user stories should be independent of other user stories. In other words, stories should be written in such a way as to minimize dependencies.

• Negotiable
  Good stories capture the essence of the desired functionality but leave room for the product owner, stakeholders, and the team to negotiate the details.

• Valuable
  Good stories explain how they are of value to the customer, the user, or to both. Some user stories, which I call technical stories, might only be valuable to the developers. In general, those types of items are better expressed as tasks and included in the team's sprint backlog.

• Estimatable
  Teams should be able to determine how large a story is so that the product owner understands the relative cost and effort to build each story. A story's cost will affect its priority.

• Sized Appropriately
  A story's size and priority together will influence the timing for refining that particular story. In general, higher priority stories need to be smaller and more detailed than lower priority stories. NOTE: in Wake's original description, “S” stood for Small. Here it has been changed to Sized Appropriately to indicate that just because something is large doesn't mean it is sized wrong. For example, an epic item that we don't plan to work on for a couple of years is arguably sized appropriately (based on when we plan to work on it).

• Testable
  Stories should be testable in a binary way—they either pass or fail their associated tests. Being testable also means having good acceptance criteria (related to the conditions of satisfaction discussed earlier).

Nonfunctional Requirements

Nonfunctional requirements represent system-level constraints. These are important because they affect the design and testing of most or all stories in the product backlog (e.g., works with a particular set of Web browsers). As such, I recommend that teams try to include as many nonfunctional requirements in their definitions of done as they possibly can. Waiting to test nonfunctional requirements until late in the development effort defers getting fast feedback on critical system performance characteristics. You can read more about this in my blog entry, “Nonfunctional Requirements and Your Definition of Done.”

Knowledge-Acquisition Stories

Sometimes teams need to create a PBI that focuses on knowledge acquisition—in other words buying information. These exploration stories are known by many names: prototype, proof of concept, experiment, study, spike, and so on. Like every other PBI, knowledge-acquisition stories must be estimatable and valuable. For instance, suppose a prototype is estimated to take a whole team one sprint to build, which tranlates to a cost of around $20K. Is it worth doing? That depends. How much would it cost if the team does the work without the prototype and turns out to be the wrong approach? $5K? $500K? The answer to these questions makes it easy for the product owner to choose whether or not to buy the information or not. Sometimes, failing fast (and at a relatively low cost) is a better strategy than acquiring knowledge through other means.

Gathering Stories

So, with all of this in mind, where do user stories come from? How do they come into being? Simply asking the users to describe what they want has never been a successful approach for me. Instead, I involve the users as part of a team tasked with determining what to build and reviewing what is being built. This level of participation is often achieved through two story-gathering techniques: a user-story-writing workshop and/or story mapping.

• User-Story-Writing Workshop
  The goal of a user-story-writing workshop is to collectively brainstorm desired business value and create user story placeholders for what the product or service is supposed to do. They frequently include the product owner, development team, and stakeholders and typically last from a few hours to a few days, depending on the size of the project. In the first workshop, I typically start by performing user role analysis to populate the user role part of the stories. I might also ask for personas, which are prototypical individuals that represent core characteristics of a role. Teams can choose to begin with large stories, and then create all the stories to accomplish that story, or to begin with small stories for the next release, then creating larger stories for features further out in the future.

• Story Mapping
  Story mapping is a technique popularized by Jeff Patton that takes a user-centric perspective for generating a set of user stories. The basic idea is to decompose high-level user activity into a workflow that can be further decomposed into a set of detailed stories. Story maps combine the concepts of user-centered design with story decomposition. Good story maps show a flow of activities from the users' perspective and provide a context for understanding individual stories and their relationship to larger units of customer value.

Traditional story-writing workshops and story mapping are complementary. For example, during user-story-writing workshops participants are primarily focused on generating stories and not as focused on prioritizing them. The 2-dimensional (table) nature of story mapping (versus the the 1-dimensional list that is the product backlog), on the other hand, offers an excellent way to visualize the work-flow relationship among different stories as well as their relative priorities. Don't be afraid to use both techniques. You might also want to consider user story training from Innolution, specifically Writing Effective User Stories, or in-person coaching/mentoring on running a user-story-writing workshop.

Closing

Scrum projects treat requirements differently, preferring less detail upfront and more conversation throughout the project. The placeholders for requirements on a Scrum project are called product backlog items, which are often expressed as user stories. These user stories can be thought of as card, conversation, and confirmation, and can be evaluated using the six criteria represented by the INVEST acronym. Through conversations, user stories are progressively refined during product backlog grooming until they are small enough and detailed enough to be implemented during a sprint (i.e., they are ready). Nonfunctional requirements should most often be part of a team's definition of done, while knowledge acquisition activities should be balanced against their value. Two excellent ways to gather user stories are workshops and mapping. The next chapter will discuss the product backlog.

To dive deeper into these concepts, consider user story training or a user-story-writing workshop from Innolution.