Measuring What Matters.
When the language of business leans on analytics and the precision of science asks for validation, a critical eye is cast on efforts that place time, resources, and money in the hands of uncertainty. While metrics can be helpful in understanding progress and performance — the trick is to stay focused on the purpose, questions and outcomes, not just the numbers.
Don’t focus on the measure, focus on the impact. Numbers can be misleading. It’s tempting and easy to believe them. While metrics may be created with the best of intentions, they can become a distraction from the true goal. In the end, the goal is not to generate a metric but to have impact. The drive to tangibly and easily show the value of investment is understandable, but we need to avoid the tendency for them to become the default, and loose sight of what matters in the process of meeting a target.
“Precision isn’t significance. When we loose track of what’s important in a rush to keep track of what’s measurable, we fail.” (Seth Godin)
When we put our judgment into a standard of performance, do we really get to excellence? The recent stories of Volkswagen cheating the metrics to pass a standard test, while an extreme case, surfaced how the focus can easily shift from designing for a result to designing to pass a test. Often that test itself can be flawed.
“What counts can’t always be counted; what can be counted doesn’t always count.”
Trying to measure something often reduces it, stripping away all the meaning, and can then occasionally become what is strived for. It can lead to a business chasing profit over building value or a non-profit securing funding instead of advancing a mission. Value is much more complex than prices and income statements would lead us to believe. It’s the difference between looking at calories vs. what they’re made of, how many students were in class vs. what they learned, hours worked vs. the value and impact of those hours, number of likes vs. the quality of the content, and so on.
Balance both the what and the why. There is power in the solidity of a number. It can summon confidence. It feels certain. Even when it might not be. Strict metrics often fail us at the early stage of development, when facing a interdependent challenge, or when considering less measurable aspects of an experience. When operating at the earliest stages of creating something new a metric misplaced can kill potential. When trying to tackle systemic issues, the truth behind a number, the why behind the what, requires digging deeper. When the goal is to define and develop potential value and perhaps not immediately prove scale, efforts to validate can be stifling and misleading. Added to that is the challenge of considering the role things like culture, design, and emotion play. Designing an experience, or shaping the values that drive an organization, may drive significant results when taken in totality, but may not be as easy to capture or prove early on. Take trust, which is a big driver of great teams, organizations and initiatives, yet is quite tricky to simplify and quantify.
The challenge of understanding return on impact and how to effectively measure it is why I’ve been so thrilled to see a closer connection of qualitative and quantitative. It shouldn’t be either/or. There is value in both. Quantitative can establish scale of impact and qualitative can ensure depth of impact. You might look at numbers linked to adoption, volume, speed, usage, and operational efficiency. Meanwhile, you also want to look deeper at things like value, satisfaction (for customers and staff), ease, uniqueness, outcomes, and learning. These are much harder to capture.
While the tendency is to lean toward the easy measure, I would advocate for biasing toward value delivered and outcomes created. The metrics that lead to a clear number are easier to get, but they are not necessarily more important. For instance, it could lead to unintentionally sacrificing quality and efficiency for speed and scale. Striving for a balance of both scale and results, with a clear link to understanding the result of the action (not just the action itself), keeps things honest and each metric embedded with meaning.
How can input-output-outcome-impact stay linked? How do we not loose sight of the important factors that are harder to capture, but significantly important in leading our countries, companies and our own lives? All measures are just a shortcut for learning and insight, yet we use them in all the wrong ways, to defend, to prove, to blame. The true story is always a bit more complicated. Getting to the nuance of qualitative detail, while looking at the precision of numbers can help fill in the unfinished picture. We are beginning to have a host of new ways of capturing and gaining insight from sophisticated (and integrated) data. It should help usher in a new level of understanding impact, progress toward purpose, and identifying leading practices, while also being more easily accessible, simple and shareable.
Lead with goals, questions and outcomes. Perhaps its better to first think of questions to ask and ambitious goals to strive for? No matter what industry or focus, defining what’s important at the outset should dictate what we measure. The intent should guide what matters most in assessing progress toward a goal, impact on a mission, and effective use of resources. Sometimes those can demand entirely new ways of looking at performance. Industries have been rethought by asking a very different kind of question, rather than competing on accepted standards. The small kingdom of Bhutan long ago catalyzed countries to rethink their metrics, by prioritizing wellbeing. They shifted the definition of success by looking at Gross National Happiness rather than GDP.
Often it’s not a matter of one singular goal or element, but striking a balance of a few. For instance, the “triple aim” is a powerful new lens on what defines great health solutions. It means balancing three elements — positive individual care experience for patients and providers, positive health outcomes for communities and operational effectiveness (eg. easy, affordable and accessible). It reflects the overall shift in the health industry moving from volume to value, and from services to outcomes. What if our health systems went even further and truly measured themselves on creating quality of life and enabling a community to thrive? If it’s the stated purpose, it should be a measure of success. What if things like empathy, ease of suffering, prevention, building health capability and supporting people in being their best self, were prioritized? What if it was about ensuring each individual understood and had access to what was most important to build and maximize their own health (new metrics for their own wellbeing)?
When looking at potential vs. (established) performance it’s especially important to embrace multiple dimensions, holding a few different unique criteria in mind. I’ve worked with many internal innovation and design teams, and the pressure is always on to demonstrate value. When combined effectively with other efforts and treated mindfully, having clear criteria, measures and evidence are all strong tools to have on hand. Often it’s a combination of a few elements that work best. For instance an innovation team might look at 3 areas: solution impact (how is the portfolio delivering value and outcomes?), organizational impact (how have we helped the organization achieve impact?), and purpose/mission impact (how have we changed the industry and beyond?). Each may have their own set of unique values, balancing both the numbers and the quality for each area. The portfolio of work alone is big enough, and distinct enough, to require it’s own set of values to drive what good looks like. Looking at number of projects done and relationships built alone says very little about the success of the efforts of the team. The goal isn’t to do more work, but to do more great work and ultimately change the lives of people being served. Staying true to that intent means not getting seduced by the power of the easy to measure.
Measures should be based on the criteria of what great is and could be. How are the questions, investment theses and clear criteria used as fuel for both focusing efforts and evaluating impact? How can we remain tangible and clear without defaulting to the easy, sometimes misleading, vanity metrics (especially when seeing the impact might be years away)? How can we use more sophisticated data tools to deepen understanding and learning? How might evidence of values in practice be used as a measure of success (without leading to unintended consequences in behavior)? How might a vision of the future and experience principles to deliver it be a guidepost? After focusing on the most important factors to consider and learn more about, the qualitative and quantitative indicators simply become a path to that learning.
“I must state clearly that my teaching is a method to experience reality and not reality itself, just as a finger pointing at the moon is not the moon itself. A thinking person makes use of the finger to see the moon. A person who only looks at the finger and mistakes it for the moon will never see the real moon.”
