Essential Scrum
Essential Scrum

Chapter 3: Agile Principles

This chapter describes the underlying agile principles that drive and inform the mechanics of Scrum.

Agile Principles Overview

For years, companies have run software development projects using a plan-driven process (also known as waterfall, traditional, sequential or predictive). This process works well if you are applying it to problems that are well defined, predictable, and unlikely to undergo any significant change. The problem is that most product development efforts are anything but predictable, especially at the beginning. So while a plan-driven, sequential, process gives the impression of an orderly, accountable, and measurable approach, that impression can lead to a false sense of security. After all, developing a product rarely goes as planned, no matter how "perfect" the upfront plans seem. (For more on the fallacy of perfect predictive planning, read the blog post, "Agile Risk Management: The Role of Traditional Risk Management."

Scrum, on the other hand, is based on a different set of beliefs—ones that do map well to problems with enough uncertainty to make high levels of predictability difficult. These beliefs, or agile principles, are described in this chapter. I also encourage you to check out this at-a-glance comparison of the underlying principles of plan-driven process and Scrum.

Variability and Uncertainty

Scrum leverages the variability and uncertainty in product development to create innovative solutions. The four principles described below all relate to variability and uncertainty.

Embrace Helpful Variability

In product manufacturing, companies conform to a defined process that is designed to repeatedly reproduce the same finished product. Conversely, in product development companies need a process that will help them create the unique single instance of a product. To create something new, companies require a process that embraces variability.

Employ Iterative and Incremental Development

Scrum is based on iterative and incremental development, which are two distinct concepts that complement one another. Iterative development is a planned rework strategy, where teams use multiple passes to improve what they are building in order to converge on a good solution. For example, in writing, I often begin with an outline, write several rough drafts, and then create a final draft for publication. The biggest downside to iterative development is that it can be difficult to determine how many passes will be necessary.

In incremental development, teams break the product into smaller pieces so that they can build a small chunk, learn how each piece acts in the environment in which it must exist, adapt, and then build more. For example, when writing a book I often write a chapter then send it out for review, rather than waiting for feedback when the entire book is complete. This gives me the opportunity to apply what I learn into future chapters. The biggest drawback of working in small increments is that the big picture sometimes gets lost.

Scrum's use of timeboxed iterations (sprints) leverages the benefits of both iterative and incremental development, while negating the disadvantages of using them individually. Inside each sprint, teams perform all of the activities necessary to produce a working product increments, allowing them to do any necessary rework inside the sprint timebox. During these sprints, teams also work on a feature at a time (versus a phase at a time), allowing them to receive feedback on new features within the context of already completed features, encouraging a more big-picture view. This learning is then incorporated into future sprints. These timeboxes and the continuous stream of feedback will guide teams to do the appropriate and economically sensible number of iterations while developing the product incrementally.

Innolution offers in-depth training on iterative and incremental development through its corporate "Working on a Scrum Team" course, and Certified ScrumMaster course.

Leverage Variability through Inspection, Adaptation, and Transparency

At the heart of Scrum are the principles of inspection, adaptation, and transparency (referred to collectively by Schwaber and Beedle in Agile Software Development with Scrum as empirical process control. In Scrum, teams inspect and adapt not only what they are building (sprint reviews), but also how they are building it (sprint retrospectives). 

Reduce All Forms of Uncertainty Simultaneously

In his book Simultaneous Management, Laufer divides product uncertainty into two broad categories: end uncertainty, which surrounds the features of the final product (the what), and means uncertainty, which has to do with the process and technologies used to develop the product (the how). In certain cases, I would also add customer uncertainty, which has to do with who the actual customers of the product might be. In Scrum, teams don't try to eliminate uncertainty through comprehensive upfront documentation. Instead, they seek to reduce small bits of all of the forms of uncertainty simultaneously through incremental and iterative development coupled with inspection and adaptation. This approach allows teams to opportunistically probe and explore their environment to identify and learn about the unknown unknowns (the things that they don't yet know that they don't know).

Prediction and Adaptation

When using Scrum, we are constantly balancing the desire for prediction with the need for adaptation. The five principles below focus on this topic.

Keep Options Open

Scrum contends that teams should never make a premature decision just because a generic process would dictate that now is the appointed time to make one. Instead, when using Scrum, teams try to keep their options open until the last responsible moment (LRM). This means they gather information and delay commitment until the cost of not making a decision is greater than the cost of making a decision.

Accept That You Can’t Get It Right Up Front

Scrum acknowledges that teams can't get all of the requirements or the plans right up front. In fact, Scrum contends that trying to do so is dangerous and wasteful, because early in the project teams are likely missing important product knowledge that can only be gained over time. Scrum teams still produce some requirements and plans up front, but just sufficiently, and with the assumption that they will fill in the details of those requirements and plans as their understanding of the product grows.

A great read for those who want to know more about adaptive planning, is "Plan Like an Extreme Skier," an excerpt from Chapter 14 of Essential Scrum.

Favor an Adaptive, Exploratory Approach

In Scrum, when teams have enough knowledge to make an informed, reasonable step forward with a solution, they advance. However, when faced with uncertainty, rather than trying to predict it away, teams use low-cost exploration (e.g., prototype, study, experiment, etc.) to buy relevant information to help them decide how to make a step forward. They then use feedback from this step to decide how best to proceed.

Embrace Change in an Economically Sensible Way

Changes late in the project are expensive, which is why sequential processes seek to carefully control and minimize change by trying to improve the accuracy of predictions. Paradoxically, however, being too predictive too early actually contributes to late, over-budget deliveries. That's because these teams tend to overinvest in each phase and make decisions based on assumptions rather than feedback on working assets, which results in a large inventory of work that will later need to be corrected or discarded.

Scrum teams, on the other hand seeks to minimize waste by embracing economically sensible change. (The blog post, Economically Sensible Change explains the subtle but important difference between embracing change, and embracing economically sensible change.) The goal is to keep the cost-of-change curve flat for as long as possible—making it economically sensible to embrace even late change. To do this, Scrum teams manage the amount of work in process and the flow of work so that the cost of change is less affected by time. For details, see Work in Process later in this chapter.

Balance Predictive Up-Front Work with Adaptive Just-in-Time Work

Scrum teams believe that upfront work should be helpful without being excessive. As such, they look for a balance point between order and chaos, where they can maximize the amount of ongoing adaptation based on fast feedback and minimize the amount of upfront prediction, while still meeting compliance, regulatory, and or corporate objectives.

To help your teams learn how to plan using agile principles, Innolution offers the on-site training course, Agile Estimating and Planning.

Validated Learning

Scrum teams acquire validated learning when they obtain knowledge that confirms or refutes an assumption that they have made. The three principles below focus on this topic.

Validate Important Assumptions Fast

Assumptions represent a significant development risk. Scrum teams try to minimize the overall number of assumptions and reduce the lifespan of any assumptions that do exist. They do this through low-cost exploration and a focus on iterative and incremental development.

Leverage Multiple Concurrent Learning Loops

Scrum accepts that some learning can only be acquired after a feature has been built, integrated, and tested and that constant learning is the key to their success. As a result, Scrum teams look for and maximize multiple, concurrent feedback loops to increase learning. Scrum also includes several built-in learning loops, including the daily scrum and the sprint review. These and other learning loops are discussed in detail in later chapters.

Organize Workflow for Fast Feedback

Scrum is based on the idea that it is better to truncate wrong paths sooner and that it is vital to find and exploit time-sensitive, emergent opportunities. As such, Scrum teams organize the flow of work so that feedback-generating activities occur in close time proximity to the original work, reducing the chance that errors will compound and maximizing their ability to change course toward an advantageous alternative.

Work in Process (WIP)

Limiting work in process is a vital principle too often overlooked by many agile and Scrum teams. The four principles below focus on this topic.

(Innolution offers corporate training in Kanban, an excellent agile method to help teams optimize work flow and work in process).

Use Economically Sensible Batch Sizes

Rather than batch up all of one type of work and perform it in a single phase (an all-before-any approach based on manufacturing's economies of scale), Scrum encourages teams to work in small batches. Economically sensible batch sizes have many benefits, including reduced cycle time, reduced flow variability, accelerated feedback, reduced risk, reduced overhead, increased motivation and urgency, and reduced cost of schedule growth.

Recognize Inventory and Manage It for Good Flow

Though many principles of manufacturing do not apply in product development, the high-cost of inventory does. No competent manufacturer sits on a large quantity of inventory, because to do so would be wasteful and costly. Product development organizations, however, are just as affected by large inventories but are typically not as cognizant of their inventory, or work in process (WIP). After all, knowledge assets aren't physically or financially visible and are far less physically intrusive than a part on a factory floor. Yet they have a tremendous impact on the cost of change. Because WIP is a critical, and frequently ignored, variable to be managed during product development, Scrum teams proactively look for ways to find a balance between just enough inventory and too much inventory.

Focus on Idle Work, Not Idle Workers

Scrum is based on the principle that idle work is far more wasteful and economically wasteful than idle workers. As such, Scrum teams are acutely aware that finding the bottlenecks in the work flow and focusing efforts on eliminating them is a far more economically sensible activity than trying to keep everyone 100% busy. Unfortunately, most organizations are far more interested in keeping people busy (worried about idle workers) than they are interested in making sure the work is flowing smoothly (worried about idle work).

One helpful resource to provide more clarity on these and other principles is "Scrum Teams, Idle Work, and Adaptive Planning."

Consider Cost of Delay

Scrum teams can use the cost of delay, the financial cost associated with delaying work or delaying achievement of a milestone, to make informed tradeoffs. Take for example, the previous discussion on idle work versus idle workers. Assume, for example, that a company assigns a documenter full-time for 12 months to work on a product, even though that person is not needed 100% of the time. Let's say that doing so costs an incremental $75K above what it would cost if the same person were brought on for two months at the end, once the product reaches the state of "all but documented." On the other hand, if that same documenter is assigned at the end, full-time for 2 months, delivery also will be delayed by the same 2 months. Let's say the cost of that delay in delivery is $500K in lifecycle profits. In other words, if the company tries to save $75K by optimizing the utilization of the documenter, they will substantially sub-optimize the economics of the overall product. Companies are presented with these types of trade-offs on a continuous basis; cost of delay is one of the most important variables to consider when making  economically sensible decisions.

To learn more about how these and other agile principles work together and reinforce each other, read the blog post, "This Ain't No Cafeteria!"

Progress

When using Scrum, teams measure progress by what they have delivered and validated, not by how they are proceeding according to the predefined plan or how far they are into a particular phase or stage of development. The three principles below focus on this topic.

Adapt to Real-Time Information and Replan

Scrum teams believe that unbridled faith in an upfront plan frequently blinds people to the fact that the plan might be wrong. Instead, Scrum teams rapidly replan and adapt to the stream of economically important information that is continuously arriving during the development effort.

Measure Progress by Validating Working Assets

Scrum measures progress by building working, validated assets that deliver value and that can be used to validate important assumptions. In Scrum, it's not about how much work teams begin; it's about what customer-valuable work they finish.

Focus on Value-Centric Delivery

Scrum's prioritized, incremental delivery model means that in each iteration, teams build and deliver the highest-value features. As a result, customers get a continous flow of high-value features sooner. Value is generated when working assets are delivered to customers, when teams validate important assumptions, or when they acquire valuable knowledge. The intermediate artifacts that are produced along the way provide no perceived customer value; they are instead a means to an end if they themselves cannot be used to generate important feedback or acquire important knowledge.

Performance

There are specific performance-related characteristics we expect when using Scrum. The three principles below focus on this topic.

Go Fast but Never Hurry

Scrum teams desire to be nimble, adaptable, and speedy. By going fast, teams deliver fast and get feedback fast. This does not, however, mean that teams should rush to get things done. Doing so would violate the Scrum principle of sustainable pace and have a detrimental effect on quality.

Build In Quality

Scrum teams don't "test in" quality at the end; instead, cross-functional Scrum teams own, build-in, and verify quality in each and every sprint. Each increment of value that is created is complete to a high level of confidence and has the potential to be put into production or shipped to customers (See Chapter 4 for a deeper discussion of the definition of done.) As a result, the need for significant late testing is substantially reduced.

Employ Minimally Sufficient Ceremony

A side effect of Scrum's value-centric approach is that very little emphasis is placed on unnecessary formality, or what some might call process for the sake of process. Scrum teams set the ceremonial bar at a low level, one that is minimally sufficient, or good enough—a bar that will vary from organization to organization. Frequently, Scrum's focus on minimal ceremony is misinterpreted to mean "Scrum is anti-documentation." That isn't accurate. Rather, Scrum teams adopt an economic perspective and carefully review which documents they create. For example, they will likely write a document if it is deliverable as part of the product; if the goal is to capture an important discussion, decision or agreement; if it is the high-value way of helping new team members come up to speed quickly; or if it satisfies a regulatory requirement.

Closing

The core agile principles are the fundamental beliefs that drive how teams develop with Scrum. A more indepth discussion and additional examples of these principles can be found in Chapter 3 of the Essential Scrum book. The next chapter begins a series of chapters on the mechanics of Scrum, starting with sprints.

View all Scrum training offerings from Innolution.