Organizations need metrics to understand what is working and what is not, but as the famed father of modern management Peter Drucker points out, what we measure is what gets our attention. To drive real impact, we must focus on metrics that drive value, not vanity. Goodhart's Law comes to mind, “When a measure becomes a target, it ceases to be a good measure.” In other words, when we set a specific goal, we tend to optimize for that objective.
What to Measure and What to Avoid
In design system circles the metric most commonly cited is adoption because of its tenuous relationship with usefulness. While not directly related to delivery, many EDP leaders maintain that adoption is an upstream requirement for downstream value. Possibly an equal number of them think adoption distracts from the true value of design systems. And so the debate rages.
Like so many performance indicators, adoption has been a popular metric because it is easy to measure and easy to understand, making it ideal for reports to executives whose buy-in we need. Adoption is also an easy-to-impact metric because it can be pushed as a mandate, making it an even less reliable indicator of a tool's ability to drive bottom-line value.
“Adoption is the kind of metric you get when you throw up your hands because you can't get the right data or you're more focused on building the design system than on what you can build with it,” says Chris Strahl, CEO of Knapsack. His stance is that adoption metrics are the equivalent of "well, people seem to use it, so it must be great! " If you're using adoption, you're not alone,” notes Strahl. “We've heard time and again on the Design Systems Podcast that folks look at this metric all the time as a measure of success, but it certainly doesn't paint the full picture, nor is it the north star for understanding value.”
Measure What Matters
So what can we use to measure and demonstrate the value of design systems and product ops tools? The unit may differ depending on the stated goal of the design system but for almost any product team -- and certainly for high-performance enterprise teams – the underlying question and focus is how fast the team can build products (not components or designs) using the design system.
At Knapsack, we have chosen to express the speed of moving from idea or design to code as Cycle Time (CT). The CT metric looks at a team's capacity as expressed by how fast an iteration cycle lasts or the number of cycles it takes to get from design to shipped product. Ideally, product teams use both measures.
We tend to focus on capacity for several reasons:
- Most teams know their capacity both for a single cycle and across a historical range.
- CT measures can easily be expressed as a relative percentage increase or decrease.
- CT measures account for a lot of secondary things (e.g. QA time)
When accessing CT in projects with our clients, we’ve found a Goldilocks zone between 20 - 40% greater cycle capacity. For most of our enterprise customers, the ability to improve CT by 20 - 40% translates to literally millions of dollars that can be saved by focusing on a metric that is tied to real output and value creation through their design system.
The Impact of Measuring the Right Thing
Imagine being at your next executive meeting with two metrics, a 10% increase in adoption of an internal design system and a 10% increase in speed-to-market for the product you manage. Which metric has a greater impact on revenue, customer experience and satisfaction, competitive advantage, and operational efficiency? Which metric do you think will win you the buy-in needed to grow and innovate?
Measuring the thing that most directly aligns to value creation is not only the best way to drive continuous improvement for your team and your design system, it is the best way to build a case for the value of what your system does and can do.
Join Richard Banfield on May 22 for a live discussion of how design systems add value and drive ROI, and how to build a business case for design system investment.
* This post was adapted from Design Systems as Innovation & Profit Drivers: Creating Value By Moving Beyond Adoption Metrics by Richard Banfield.
