Metrics that Matter: How You Should Be Measuring Agile
Monday, October 21, 2019
Posted by: Gary Hotze
Cornel J. Montano, Partner & Managing Director, Forté Group
Over the past decade, agile has become the cure-all to improve software delivery. Agile adoption is now almost universal: In the 2018 “13th Annual State of Agile Survey,” 97 percent of IT professionals reported that their organizations practice agile development methods.
Even with agile adoption at an all-time high, there’s a value gap: Most organizations aren’t satisfied with the output they’re getting from agile. This is not to say that agile is failing, but most companies have difficulty communicating the value of agile to their organization.
I’ve observed this among many companies and IT organizations I’ve worked with. IT departments feel like they’re doing a great job. Agile teams are delivering every two weeks. Releases are on time. Velocity is high. Agile is a success!
But when business leaders respond with, “How does any of this help us?” IT departments are baffled. Shouldn’t agile metrics be self-evident of the methodology’s effectiveness?
The answer is that it depends what you’re measuring and why. In this article, I outline what agile metrics matter most from a business standpoint and how you can measure agile success, then communicate its value to your organization.
Commonly Used Agile Metrics
There was a time when agile was a huge selling point. However, many “agile” organizations see agile as a set of checklists, sticky notes, time-tracking tools, and procedures. To these organizations, agile is a toolbox rather than a mindset. This is the primary reason “agile” product development often fails.
In the aforementioned “State of Agile Survey,” respondents listed common ways in which they measured agile success:
· On-time or speed of delivery
· Product quality
· Project visibility
Each of the above are great metrics for measuring agile. What they don’t measure is business value. The number of tickets you’ve moved through your project management system is irrelevant if the result is a product no one will use.
Value-Based Agile Metrics
A more productive way to measure agile is to focus on (and measure) the value agile can bring you. This is not to say that metrics like velocity and quality should not be measured—far from it. These data points are useful but shouldn’t be the end goal.
So how do we measure value? The following metrics, also listed in the survey, are closer to the mark:
· Customer/user satisfaction
· Business value
· Product scope (features, functionalities, requirements, etc.)
· Process improvement
To expand on the results of the survey, an effective way to measure the value of agile is to track key performance indicators (KPIs) that align to value. These can include:
· Usage index: What features are people actually using? By analyzing the usage index of different product features, you’re able to measure what’s valuable to your users, an indispensable KPI for business value. By tracking the usage index of features in a production setting, you can prioritize the features that offer the greatest return on investment.
· Innovation rate: Innovation rate measures the capacity of a team to build value-driven features versus non-discretionary work like detect fixes and application support. The more a team is burdened with maintaining existing systems, the less bandwidth it has to innovate, grow, and create competitive advantage. By reducing an application’s support needs, you can increase a team’s innovation rate and focus on creating new, value-driven features instead of worrying about keeping the lights on.
· On-product index: An on-product index is the measure of time a team spends working on the product. This KPI measures the efficiency of how a team is run and how much they’re contributing to a product’s growth. By reducing unnecessary meetings and other distractions, you can increase a team’s on-product index, which will “buy time” to work on valuable things.
· Installed version index: Similar to the innovation rate KPI, the installed version index measures the number of versions of an application that require support. Each supported version increases the required support needs, thus lowering a team’s ability to focus on value-driven features and innovate.
These KPIs allow a product owner to view and control the resources of a team and prioritize value-based features and objectives.
Measuring Agile Success: Value-Driven Development
A simpler approach to measuring the value of agile is to focus on defining it rather than creating it. At Forte Group, we help align the IT enterprise, business stakeholders, and product owners to define and focus on what we call value-driven development, where business value drives priorities.
For example, you could employ the best and most experienced developers to write code and build an application. However, that application itself doesn’t matter unless it offers value to customers or helps them achieve a goal.
Of course, the definition of “value” can differ from person to person and organization to organization. Therefore, it’s important for organizations to get aligned on what value is and what it means to them before they can begin measuring it.
Seeking alignment is a process that requires deep thought and conversations, as well as a leadership culture that facilitates those conversations. Leaders must understand that business plans can evolve and change, and that thought-provoking questions from project teams usually result in better outcomes for everyone involved.
How to Define Value
So, how do IT enterprises and organizations define value? Here are some questions to ask that can help:
● What’s the problem or challenge we’re trying to solve?
● What’s the simplest and quickest solution to solving the problem?
● Is this useful to our customers?
● Are users using it?
● How can we validate that what we’re building is the right thing?
● What’s the shortest path to value, both to our customers and to our organization?
● What’s our minimum viable product (MVP)?
So, what exactly do we mean by MVP in this context? Value-driven development defines an MVP as the smallest change the team can make to a product that offers immediate value to end users. Not only does this process eliminate wasting precious time on developing complex and time-consuming features, it also helps the team focus more on prioritizing and achieving value.
By taking the time to think through and answer these questions, IT organizations and enterprises will find themselves in a better position to plan a release roadmap that maximizes the MVP.
In addition to asking, “Are we delivering value?” you should also ask, “Are we delivering value frequently?”
The Results and Impact of Measuring Agile
Nothing actually measures agility. It’s about identifying and enabling value that’s unique to your organization, then using the right strategy to maximize that value.
If you look at the list of metrics at the beginning of this article as examples of how some IT enterprises and organizations measure agile and adopt your own strategy for measuring agile and value, you’ll begin to see better quality, higher productivity, and happier customers. Measuring agile and delivering value helps IT enterprises and organizations advance, accelerate, and evolve.
If you’re struggling to measure value, it can be helpful to have an outside resource. Speak with one of our practice engineers about ways to measure the value of agile while evolving to a value-driven development model.