25 Agile metrics and measurements - Measure one damn thing!

You and I know it: There are loads of agile key performance indicators, metrics and measurements. But you always have to keep one thing in mind:

Tell me how you measure me, and I’ll tell you how I’ll behave.

Dr. Eli Goldratt

Therefore, one question has to be asked: Are all of these metrics relevant? \n\nAnd then there is this other citation:

Simplicity is the soul of efficiency.

Austin Freeman

So the next question arises: If we would want to measure agile as simply as possible, how would we measure it? What agile key performance indicators, metrics and measurements matter most? Or if we would measure only one damn thing, what would it be?

Overview: Typical agile metrics & KPIs

There are far too many ways to collect agile metrics and KPIs. Here is an overview of the possibilities (not exhaustive):

Agile KPI	Explanation	Based on…	Correlation to customer value	Ease of measuring it	Aims at…
Sprint Burndown Chart	Shows the progress within the sprint to achieve the sprint goal	Subjectively estimated effort	Low 🔴	Partially manual ⚠️	Productivity 🏃
Velocity	Indicator of how much work was accomplished during a sprint	Sum of subjectively estimated story points (complete user stories)	Low 🔴	Partially manual ⚠️	Productivity 🏃
Epic and Release Burndown	Monitor progress across many tasks towards an epic	Subjectively estimated effort	Medium ⚠️	Partially manual ⚠️	Productivity 🏃
Control chart	Time span from status “In Progress” to “Done” of tasks	Objective timekeeping	Low 🔴	Automated ✅	Productivity 🏃
Cumulative flow diagram	Number of tasks in the backlog in relation to the remaining time	Objective number of tasks vs. time	Medium ⚠️	Automated ✅	Productivity 🏃
Lead time	Time period between commissioning a product and delivery	Objective timekeeping	Medium ⚠️	Automated ✅	Productivity 🏃
Value delivered or delivered customer benefit	Customer benefit of a requirement in € or points	Subjectively valued and assigned by the product owner	Medium ⚠️	Partially manual ⚠️	Customer value 🔍
Net Promoter Score	Willingness to recommend the product to others	Subjectively by the customer	High ✅	Partially manual & tools necessary 🔴	Customer value 🔍
Work Item Age	Indication of how long it takes from the start to the end of a task	Objectively through a tool	Low 🔴	Automated ✅	Productivity 🏃
Throughput	Average completed tasks within a given time	Objectively through a tool	Medium ⚠️	Automated ✅	Productivity 🏃
Blocked time	Number and duration of tasks that cannot be processed further due to internal dependencies	Objectively through a tool	Medium ⚠️	Automated ✅	Predictability 🎲
Escaped defects	Indicates the number of software errors when a release is published	Objectively through a tool	Medium ⚠️	Automated ✅	Quality 🏆
Failed deployments	Number of deployments	Objectively through a tool	Medium ⚠️	Automated ✅	Quality 🏆
Code coverage	Extent to which the source code of a program is executed (indicates the quality of the software)	Objectively through a tool	Medium ⚠️	Automated ✅	Quality 🏆
Quality Intelligence	Helps identify recent changes in the code (indicates software quality)	Objectively through a tool	Medium ⚠️	Automated ✅	Quality 🏆
Cycle Time	“Work in progress” divided by average task completion rate (indicates how well the workflow is working)	Objectively through a tool	Medium ⚠️	Automated ✅	Productivity 🏃
Customer satisfaction	Customer or user satisfaction with the product or service	Subjectively by the customer	High ✅	Partially manual & tools necessary 🔴	Customer value 🔍
Planned-to-done-ratio	Ratio between planned and completed user stories	Objectively through a tool	Medium ⚠️	Automated ✅	Predictability 🎲
Usage index	Shows which features are actually used by customers and to what extent	Objectively through a tool	High ✅	Partially manual ⚠️	Customer value 🔍
Innovation rate	Ability of teams to develop valuable features vs. fixing “bad” work (bug fixes and support requests)	Objectively through a tool	Medium ⚠️	Partially manual & tools necessary 🔴	Quality 🏆
Enterprise value	The value of the company (division)	Objectively or subjectively by the stakeholders	Medium ⚠️	Partially manual ⚠️	Customer value 🔍
Psychological safety	Probability that employees openly share their opinions and ideas	Subjective through a (survey) tool	High ✅	Partially manual & tools necessary 🔴	Culture 🧑‍🤝‍🧑
Purpose	An emotional justification for why your company or department exists	Subjective through a (survey) tool	High ✅	Partially manual & tools necessary 🔴	Culture 🧑‍🤝‍🧑
Vision	An emotional picture of what your company or department will look like in the future	Subjective through a (survey) tool	High ✅	Partially manual & tools necessary 🔴	Culture 🧑‍🤝‍🧑
Employee satisfaction / Happiness	Satisfaction of employees with their work	Subjective through a (survey) tool	High ✅	Partially manual & tools necessary 🔴	Culture 🧑‍🤝‍🧑

Don’t get me wrong: Of course you can have more complicated models for agile key performance indicators or metrics, such as the Agilometer. But I was fascinated by the goal to measure one thing only. Therefore, this article investigates this goal. 

The above agile KPIs serve as the basis for this text.

A note in advance: A simple, pragmatic but very helpful agile metric is the one shown in the video here. Agustina even shares the team data publicly from the whole of last year:

Play

Reject agile metrics & KPIs?

Do we even want agile key performance indicators, metrics and measurements?

Before we go deeper, one thing that I regularly hear or read on LinkedIn when I talk to agile coaches, scrum masters or scaled agile frameworks consultants: Do you even want to measure agile?

There are lies, damned lies, and then there are statistics.

Mark Twain

Mark Twain is a little dramatic about it. But he has a point. A point that Albert Einstein put in a Nutshell.

Not everything that can be counted counts.

Albert Einstein

What is “KPIs” in Agile - Velocity, burndown charts, number of failed deployments? Are these agile metrics critical to agile success? I doubt it. 

But abandon all metrics? That would also be a mistake.

Advantages of agile KPIs and metrics

In today’s world, economical circumstances change fast. In good times, decision makers start thinking about an agile transformation because they have the resources and safety to do so. And in bad times?

In bad times, senior executives will fall back into traditional thinking. They will move to old behavioral patterns: e.g., top down decisions that are not compatible with modern agile thinking.

If you would accept not having agile key performance indicators in bad times, you shouldn’t even start an agile transformation. Because at that point, any progress made within that transformation would be destroyed.

Therefore, the only chance that scaled agile frameworks survive difficult times is by beating the system with its own weapons: by providing metrics. Agile key performance indicators that help to manage in times of uncertainty.

And I am sure about it: there are agile metrics that matter. 

In retrospect, I consider it one of my biggest mistakes to always categorically **transformation.

Marcus Raitner

Einstein also says it in his citation: There are things that count. Those are the ones we are looking for in this article.

Measuring agility - What makes a good agile metric?

Let’s say the question you typically ask in your Daily Stand-up is: What did you accomplish today? So that’s how you “measure” progress in your team.

Good question, right? No, not right. This question urges the team to show that it is diligent. The question puts pressure on the team to complete the “to do list” so that it can proudly point to the metric: Yes, I’ve been pretty busy in the last 24 hours!

But completing the “to do list” is good or not? Well, it depends. Something else is much more important: Namely, achieving the team’s goal. This is typically - in the case of agile teams - delivering customer value or, in English, “delivering value”.

Agile KPIs - how to measure agility

A better question (or metric) in your Daily Stand-up would therefore be: “How have you helped your team or organization achieve our (sprint) goal in the last 24 hours?”

Change your question (or the metric), change the way people think and act: first effective, then efficient - in the words of Peter Drucker accept.

Change the metric - change the way people think and act.

To quote Einstein: We have to find the one thing that can be counted - and that really counts.

What does really count in an agile transformation?

How agile metrics and measurements should be viewed

The goal of your agile transformation is definitely not an agile transformation. Why are you doing the agile transformation? Let’s use the “3-Why technique” to understand this:

So the core reason for your agile transformation: you don’t want to end up like Blockbuster, ignoring industry trends and customer needs, unopen to change and finally running out of money.

You want to end up like Netflix, who are constantly evolving their business model around their customers’ core needs. Just look at them Case Study Blockbuster vs. Netflix.

How can you translate that into measuring one thing in an agile transformation? Well that’s tough.

What companies do instead: measuring parameters because they are easy to track. Because tools out there throw them at you. The probability that these are the right metrics is pretty low.

Because for example, it is not about improving sprint velocity. That is a common mistake: monitoring and measuring effort, or efficiency. Instead, it is about solving customer needs. 

Agile KPIs & measuring agility - an important insight

Having laid down all of this, if we want to measure one thing in agile, we have to rate the validity of this metric against one thing.

We need to measure outcomes, not outputs. We need to measure our agile transformation by how much it helps us achieve our goal. We must not measure people by their time spent, but by their contribution to a common vision or goal - we must measure the value created for the customer!

If we want to measure by value, we have to deeply understand the customer’s needs.

For example, a railroad company has to understand that it is not in the railroad business. It has to understand that it is in the transportation business. Customers do not care if they are transported by train or by plane.

One other thing: Of course, it is key to not only measure but also reflect on agile metrics. Like for example about the Spotify Health Check ´ results (through a retrospective). 

You can do exactly that (also across teams if required) via our Health Check & Retrospective Tool. You can find more information at “How it Works”. You can also simply take a look at a Health Check Retrospective here. In this case, it is a retro regarding Scrum.

The measurement of culture metrics (such as psychological safety) can therefore take place in agile retrospectives, where measures can also be derived immediately.

One alternative to this is one-to-one meetings between managers and employees. 

Even in this routine, regular short pulse surveys and reflections can help to collect meaningful culture metrics and at the same time implement a continuous improvement process.

Our Echometer tool can also help you with this. The following template contains some health questions that you can reflect on individually with your team members. You can answer them on a scale from 1 to 7. 

Try it out without logging in using the button:

One other thing about agile metrics that matter - timing

One other thought is important here. At best, metrics depend on the timing and/or at what stage you are in your agile transformation. 

Let’s imagine that you already know that transforming to scaled agile frameworks or other agile working practices is the right move for you (That is by the way the first thing that many companies forget to check).

In that case, at the beginning of your agile transformation your focus should be on one thing: The agile mindset of the leadership team.

Is the leadership team really ready to change? Does it understand the implications of a transformation? Is it ready to seriously introduce agile methods as the first team in the company - including Kanban, agile retrospectives & continuous self-reflection?

Theoretically, an initial agile metric should therefore focus on the “readiness of the management or leadership team”. My colleague Jean gives in his article 7 tips on the role of leaders in agile transformations .

The next most important question to ask in your transformation: Do we have the right processes in place to understand and monitor our customers’ needs continuously? This is where the next agile metric could focus on.

But hold on. That doesn’t help us achieve the goal of this text - only one Thing to measure.

No. It serves to inspire you for your agile transformation.

So let’s take a look at what typical agile metrics exist and to what extent they correlate with the most important outcome that your agile transformation is about: customer benefit.

The most important agile metrics - a ranking

You can find a table above with the agile metrics that matterwithin scaled agile frameworks and agile transformations. I ordered them by five areas, five goals that you generally want to reach through your agile transformation:

• Customer value: Are you meeting the customer’s needs?
• Predictability: Are you delivering on time with smooth processes?
• Productivity: Are you getting more and more done at the same time and with the same resources?
• Quality: Do you deliver a product free of bugs and other issues?
• Culture: Are the people in your organization satisfied, continuously learning and do they feel free to innovate, making sure you can maintain your pace?

Agile KPIs - how to measure agile success

The table above also gives an indication of

• how easy it is to measure the metric (based on my personal experience)
• how much that metric is correlated to our core goal with a long term perspective: future customer value (based on my experience).

So which agile metric from the table above is the one?

Interesting to note is that there does not seem to be an agile metric that is easy to measure and provides the right kind of value. Damn it.

I would say that “simplicity of measurement” is not as important as “correlation to future customer benefit”. Therefore, the most valid metrics seem to be “Usage Index”, “Customer Satisfaction” or the “Net Promoter Score”.

By the way, you can start collecting 11 KPIs in your team today by carrying out the so-called Spotify Health Check with your team. Take a look at this video and the other videos mentioned to see how it works:

Play

Scaled agile Frameworks - the most important agile KPI is…?

It does not matter if you use the Scaled Agile Framework (SAFe®) to measure one thing, or a different framework to do your agile transformation.

Given these three agile key performance indicators, what correlates most with future customer value is probably customer satisfaction. Or, to be more specific, top-box customer satisfaction. You can find more on that topic following this link.

Customer satisfaction probably also has the highest correlation of all of these metrics with the ROI of your agile transformation. It shows you whether to pivot, persevere or perish.

Agile KPIs: Is one simple metric enough - or not?

But stop. By focussing only on customer satisfaction, how do you make sure you are staying ahead of your competitors long term ahead? How do you make it possible for innovative and disruptive ideas to grow and thrive in your company? What is your long-term goal after all…

Given these questions, I think we are getting back to the basics: to continuously provide value and to innovate around it, you need two other things: smooth agile processes and a great company culture.

Your company culture makes sure that employees feel psychologically safe , are open to fail, raise their voice and share their ideas. And your agile processes make sure you implement these ideas faster than your competitors.

**Agile, as a leader, aims for a people-first approach; it puts people over things.**Vikram Verma

The following graphic illustrates this in an easy way. If value is your long term goal, then our input is your working methods times culture.

Culture × Agile Processes = Long-term Customer Benefit

I once read that “Agile transformation requires culture change, not process change.”

I disagree with that. It requires both.

Scaled agile framework - Measure one thing? Measure three things!

So, if you only want to measure one thing in Scaled Agile Frameworks (SAFe®) or other frameworks, it would be customer satisfaction. But honestly, I can’t recommend measuring just one thing - sorry to disappoint you.

If you really want to keep it as simple as possible with your agile key performance indicators and metrics, I recommend that you measure a minimum of three things that will help you thrive long term:

• Measure customer value - through customer satisfaction. 
• Measure Corporate culture - through psychological safety as an indicator for learning and innovation. 
• Measure agile effectiveness - using the “planned-to-done ratio” as an indicator of how well you are able to deliver incremental customer value.

After measuring, build. Then learn. And then iterate on your metrics…

That’s so easy, right? Nope. Are you serious about implementing agile frameworks throughout your department or company?

What is a good KPI in agile and can I do about it?

We are currently surveying dozens of experts - Release Train Engineers, Agile Coaches, Scaled Agile Framework consultants - on the topic of KPIs and metrics in the context of Scaled Agile Frameworks and agile methods.

Based on those interviews, we developed Project Scagile: 7 webinars to help you avoid the 7 core mistakes in agile transformations to avoid. One of the webinars is on the topic of “Agile Metrics”.

By the way, if you are still looking for a suitable retro board, our article can help you with the topic: Comparing the 6 best retrospective boards