How DevOps with Simulation Can Solve Your Hardware Problem Part 1

By Jeff Gowan

If you develop software for devices and systems that will eventually work on the intelligent edge, you’re already sold on the idea that DevOps is a worthwhile endeavor, and you want to know how you can take your practice to the next level, you are in the right place.

Now, you may already be sold on DevOps, but perhaps your colleagues aren’t. Maybe this will help:

According to the latest “State of DevOps” report published by the Google DevOps, Research, and Assessment (DORA) team, a low-performing DevOps team might deploy software to production less often than every six months. When they do, they probably have a change failure rate of 16%–30%, and they need up to six months’ lead time to go from code to production. Changes are infrequent, there is a relatively high risk of breaking something, and it will take six months for the next update.

Compare this with an elite-performing team that can deploy 973 times more frequently (basically on-demand multiple times per day), has a lead time that is 6,570 times faster from commit to deploy and a faster time for recovering from incidents, and whose changes are 30% less likely to fail.

While the numbers above are impressive, they may be a bit abstract. What happens when you attach a dollar value to failure?

According to a 2018 report published by the Consortium for IT Software Quality (CISQ), the total cost of poor-quality software in the U.S. is $2.84 trillion. That works out to about $50M per failed project, on average. Even if that is off by a factor of 10, think about whether you can really afford to lose that much money.

Why is it so difficult to release high-quality software? It is generally agreed that soliciting and responding to customer input in a timely way is difficult for organizations. Agile and other modern development processes aim to reduce the risk, but how can the rest of your organization keep up and deliver value at the speed of relevance?

The point is, doing DevOps is good, and there is real benefit to doing it better.

What do you need to do and how do you do it?

I’ll assume you’ve proven the basics of DevOps. Your teams have probably become more agile,  are more aligned, and have shown improvement in collaboration. Maybe by now they are able to push higher-quality code to production faster than they were able to before. Good job, especially if you played a role in making that happen!

At this point, though, the improvement benefits may have plateaued, but chances are your executives’ expectations have not. It’s time to think about your next move.

The trouble is that you need to prove ROI. You may be concerned that your next change may not be as profound as those you saw in the early phases. So how do you make a case for incremental change that will accelerate your DevOps effectiveness? You’ll probably run into one or both of the following scenarios:

1.     You need to overcome inertia: This is what I call the “didn’t we just do this?” problem. You go to your team and say you want to introduce new DevOps practices, but they’ve already designed a workflow around the current ones. Change can be hard, especially if people are just becoming comfortable with the last set of changes.

2.     Executives expect too much: Who hasn’t experienced this at some point? Often executive expectations are very high, and sometimes it seems as though they don’t truly appreciate how complex this work really is. There is constant pressure to innovate. Faster. Realistically, this may be the better problem to have — the question is, though, can you keep up?

Addressing both of these problems — resistance and pressure — can be accomplished in one way: identifying the “care abouts” and finding the metric you want to change. Nobody says, “Change your process to make your software worse.” But to simply say, “Make it better and faster” isn’t helpful.

As we consider how to build an ROI, or make the case for increased investment in DevOps, let’s look at some things that are measurable and some that are not.

Most companies can establish how much it costs in dollars to resolve a defect reported by a customer. With increased automated testing, defects are found sooner and fixed at lower cost. If you can reduce the number of shipping defects by 5%–10%, that is an estimable value based on data you likely have (or can find) today. Note that someone on your team will still have to do the work to fix the defect, so we won’t double-count the savings.

There are benefits that are harder to measure but may change over time and are worth tracking. A good example is reputation. There are different ways to measure corporate reputation. At Wind River®, we use net promoter score, or how likely is it that a customer will recommend us to someone. Your company may use a different metric. The point is that you can use this metric to track your reputation in the marketplace. Causality is harder to establish — did your reputation improve because of increased quality or some other market force? — but if you’re tracking the metric over time, you can identify positive or negative trends.

Reputation has an impact on sales, and if you have access to that data, it can also show trends. Positive increases in renewals with happy customers, better penetration into other projects with an existing customer, or even leads that come from positive coverage are all trackable by most sales or marketing teams. Again, causality is hard to prove, but it is worth tracking.

So a good case can be made around cost savings that improve the bottom line. It can be reinforced with harder-to-measure metrics that your company already tracks to show an improvement to the top line.

Now that you’ve determined what you want and how improving DevOps will help you get it, I have some bad news for you.

If you are developing for the intelligent edge, you have a problem with hardware. The good news is that simulation is the answer to that problem.

In the next installment of this series, I’ll share some of the specific ways hardware may be causing a problem, and how simulation can fix it.

We’ve also created some short videos to illustrate the ideas in this series, which you can use to share these ideas with your team and colleagues. To watch, visit https://www.windriver.com/resources/webinars/devops-with-simulation-can-solve-your-hardware-problem.

To learn more about Wind River Simics®, visit www.windriver.com/products/simics or contact us at salesinquiry@windriver.com.