Blog

Agile in a Hardware / Firmware Environment – Draw the Cost of Change Curve

Do you believe that agile can be applied to the development of non-software products? Well it can! Take Wikispeed, for instance, which builds a street-legal car every sprint. This one example alone should be enough to convince you that agile outside of software development is not only possible, but also is a great way to achieve competitive advantage.

Over the past couple of years an increasing number of my training and coaching engagements have been for companies where hardware (HW) and/or firmware (FW) was a major focus of their agile efforts.  

Many people I encounter both within and outside of HW/FW environments assume that HW/FW development is better suited to a non-agile, sequential, phase-based approach. In my experience asking “Can agile be used in a HW/FW environment?” is the wrong question. A more appropriate question would be “Where and how can agile be used during HW/FW development?”

One effective way to start answering this question is to examine the cost of change characteristics of the organization’s current HW/FW development approach.  

Cost of Change Is the Driver

The principal argument that I hear against the use of agile in HW/FW development is something like: “Hardware is not software, we aren’t as flexible as you software guys because the cost of change for us is different than it is for you.” I agree that the cost-of-change curve is different when developing HW/FW than when developing software, but that doesn’t mean that agile can’t play a role during HW/FW development. Let’s see why.

A cost-of-change curve has time on the horizontal axis and cost on the vertical axis.  

A goal during product development is to keep the cost-of-change curve flat for as long as possible—making it economically sensible to embrace change for as long as possible.

All cost-of-change curves contain one or more inflection points (where the graph turns sharply upwards). Once you hit an inflection point, the cost of making (at least certain) changes goes up. In other words, making that same change earlier would have been cheaper and now it has gotten much more expensive.

When HW/FW people say, “Our cost of change is different!” what they really mean is that their cost-of-change curve hits its first major inflection point sooner than a software cost-of-change curve. See the following graph.

Don’t get too attached to the exact nature of each curve in this graph—I didn’t base the shape of each curve on a rigorous analysis of historical data. Rather, this graph is meant to be more intuitive. The basic takeaway should be that HW typically inflects up sooner than FW, and FW sooner than software.

Draw the Cost-of-Change Curve for your Development Process

To make any discussion on this topic more real, you need to draw the cost of change curve for your specific development process. Before I do training or coaching for a HW/FW team, I ask team members to help me graph their cost-of-change curve. Below is a recent example of one my client’s cost-of-change curve. 

In this graph I changed the names of the “phases” to make them generic (and not company identifiable), but the cost-of-change curve is the one I drew based on the input my client provided. (Again the fine-grain details are not what’s relevant; what is relevant is the general shape of the curve.)

In this example, the cost of change during definition and prototyping is pretty low. The goals of these phases are, respectively, to come up with a killer high-level product definition and to create prototypes of sufficient fidelity to provide confidence that the product can be cost-effectively manufactured at scale. During these phases the teams can operate (and, in my client’s case, currently are operating) in a very agile-like way.

Notice at Phase3 the cost-of-change curve takes a steep climb upwards. For this particular company, Phase3 is when the teams actually start cutting steel for the injection molds that will be used to produce their product’s plastic housing. Cutting steel (taking a block of steel and cutting out a mold) is expensive in terms of both time and money—about six weeks and tens of thousands of dollars. So, when the teams get to this phase they really don’t want to be making any significant changes in areas where the cost of change has inflected radically upwards.

Cutting steel is just one example of an aggressive inflection in the cost-of-change curve. In other companies, the teams’ cost of change might inflect upward at this point if they are going into the foundry to fabricate custom ASIC chips. Making design changes and having to go back into the foundry to do another spin on the chip is expensive in terms of both time and money.

To further the example in the above graph, when the teams begin Phase5, the cost-of-change curve takes another steep upward turn. In this case, Phase5 is where the company places orders for long-lead components needed to manufacture its product. Order volumes could be in the millions of units. At this time, for example, the teams really don’t want to change their choice of microcontroller after the company has already placed a non-cancellable order for one million units!

To summarize, by drawing the cost-of-change curve, we can see the low-cost-of-change areas where agile might have obvious application in a HW/FW environment. My experience is that companies doing HW/FW are already operating in a very iterative and incremental manner in the areas where their cost of change is low.

Tools and Technologies Can Influence the Cost-of-Change Curve

The tools and technologies employed by teams can significantly affect the shape of the cost-of-change curve. Often when people comment that HW development can’t be agile, they are focusing on fabricating boards or chips. But in the early stages of their development effort, teams are not immediately building boards or chips. Today, teams commonly perform software simulation of electronic and mechanical components and assemblies first. When simulating circuits the cost of change is on par with software change. In other circumstances perhaps the teams are using FPGAs (field programmable gate arrays) either as their target deployment technology or to test-out their logical designs before moving on to more custom development. Changing FGPAs is more like changing software than it is like changing hardware.

As tools and technologies improve they tend to flatten the cost-of-change curve, and equally if not more important, such improvements can reduce the overall product development cycle. Revisit the injection-molding example I used earlier with a particular focus on where the cost-of-change curve first inflects upwards in the graph. That inflection point occurs because the technology (injection molding) requires a specific tool (a steel mold to hold hot plastic) to be fabricated.

Now, fast forward into the future (maybe a decade or so from now). What if by then 3D printers can produce plastic housings at speed, cost, and quality levels that exceed today’s injection molding? If this becomes true, then a major step in the process can be eliminated along with the associated cost of cutting steel and the time required to cut it. More importantly, change at this point in the development cycle becomes economically sensible again (where in the past it was prohibitive). You would be more willing to consider a change to the plastic housing of your product if the majority of the work to create it was performed in a software CAD system and then sent to high-speed 3D printers for fabrication.

So not only do we change the nature of the cost-of-change curve by flattening out an inflection point, we very likely remove weeks or months of overall time for getting a new product to market. Remember, in the earlier example, it takes six weeks to cut steel and the company is planning to cut steel twice because the first cut, after inspecting it and trying out the resulting plastic housings, will likely require some adaptation for the final product.

Advance the Work Forward to Lower Cost-of-Change Areas

There is an additional approach that I use when working with HW/FW organizations to help them better utilize agile. The approach is based on leveraging the principle of “never let an unvalidated assumption live long before you validate it.”

Let me use failure analysis to illustrate the point. Let’s say you are developing a device that can be dropped (e.g., phone, camera, watch) or can fail in a mechanical way (the door handles of a Tesla car automatically present and retract themselves electromechanically). At some point you need to test the various failure modes of your product. For example, can the Tesla door handles present and retract 100,000 times before there is a failure? Or can you drop the new Apple watch 10 times in a row from a height of one meter and not have the band disconnect from the watch?

The obvious (and very late) time to validate whether these characteristics are being met is when you actually have the final product (and before you manufacture at scale). But learning that you have a failure problem this late in the process might put you rather high up on the cost-of-change curve. So, much like we do with software development, we would try to advance forward at least some of the testing (the failure analysis work) to an earlier part of the development process when the cost of change (assuming we find a problem) would be easier to absorb.

The good news is that frequently we can advance such work to an early phase. In the specific case of failure analysis, there are some pretty good software tools that can be used to determine whether or not your design will likely have a problem under certain conditions (e.g., dropping the watch from a height of one meter). Using these tools won’t eliminate the need to do failure testing on physical instances of the product, but it would hopefully raise your confidence level that failure testing done late will confirm the robustness of your product, rather than exposing now-expensive-to-change problems.

Conclusion

Agile can and is being used in HW/FW environments to develop products. To understand where and when your teams can use agile, first draw your cost-of-change curve based on your current process and annotate it with the tools and technologies that are drivers of the costs. Then start looking for opportunities to leverage agile approaches in all of the low-cost-of-change areas. Next, look at the high-cost-of-change areas and see if applying agile principles there can move work to areas of a lower cost of change.