Wabi-Sabi Your Data: The Success Metrics Layer
Zen and the Art of Measuring Data Success: The Final Layer of Your Vision Board
Hello, data Shokunin-deshi!
Welcome to the final part of our journey through the Data Ecosystem Vision Board. Starting today, you have the complete framework to create your own Data Ecosystem Vision Board! Today, we explore the Success Metrics layer - our compass for measuring growth and nurturing change.
With this final Success Metrics layer, your Data Ecosystem Vision Board becomes complete. To ensure you're getting the most value from this framework:
Review all three layers together: Present state informs future capabilities, which drive your success metrics
Update regularly: Review every six months to ensure alignment
Share widely: Make the board accessible to all stakeholders
Document decisions: Keep track of why certain metrics were chosen or changed
What will we learn today:
How to select and implement the two core KPIs that truly matter
Techniques for creating effective supporting metrics
Framework for establishing clear guiding principles
Practical approach to change management
Steps to complete your Data Ecosystem Vision Board
In our previous newsletters, we established our present state and envisioned our future. Now we focus on measuring progress toward that vision.
The Story of Two States
A student went to his meditation teacher and said, "My meditation is horrible! I feel so distracted, or my legs ache, or I'm constantly falling asleep. It's just horrible!"
"It will pass," the teacher said matter-of-factly.
A week later, the student came back to his teacher. "My meditation is wonderful! I feel so aware, so peaceful, so alive! It's just wonderful!'
"It will pass," the teacher replied matter-of-factly.
Like the meditation student’s experience, our data ecosystem’s performance will fluctuate. This is why we need consistent, unchanging core metrics to guide us through both challenges and successes.
This story illustrates:
Data teams often find themselves in a cycle of firefighting. Focused metrics break this cycle.
ROI measures how data generates value; utilization tracks how much is being used.
Avoid over-utilization—idle or underused data may have hidden potential.
Frameworks help build shared realities and prioritize initiatives collaboratively.
From State to Vision to Metrics
While the present state focuses on setting the “state of the nation” — highlighting the costs of data issues, gaps, and opportunities — the future vision imagines how the data ecosystem will look and the capabilities that will be crucial.
The metrics layer serves as the bridge between these states, keeping us focused on achieving our long-term vision:
Setting a clear baseline for performance
Tracking meaningful progress without distraction
Aligning actions with principles
Why This is the Most Important Layer of Them All
With over 15 years in the data industry and the last 7 years spent in data strategy meetings—and later consulting companies on crafting theirs—I’ve experienced a recurring frustration: the outcomes of these meetings often leave everyone dissatisfied.
The crux of the problem? A lack of long-term vision. Sure, we’d discuss what needed fixing and create plans. But without meaningful metrics to measure progress, the data function always felt like mere enablers. The business would ask us for a dashboard, we’d ask a few clarifying questions (often met with irritation), and ultimately, we’d deliver it—just doing our job.
These strategy meetings typically resulted in a long list of initiatives. Deep down, we all knew we couldn’t execute them all. To bridge this gap, we’d connect them to OKRs, hoping to align priorities. Yet both sides—data and business—knew we were barely scratching the surface, achieving 0.7 at best, and moving on as though that was acceptable.
Worse, our communication lacked monetary framing. For example, we’d say an initiative was essential for management to "trust the conversion data" but fail to assign a financial value to that trust. Product teams, understandably, prioritized revenue-driven initiatives, while we overloaded ourselves with ambitious goals, conflicting tasks, and the unspoken acknowledgment that even OKRs couldn’t resolve the mess.
This is where the Success Metrics layer can change everything. It gives us a way to evaluate initiatives based on their impact, ensuring we focus on what truly matters. By introducing clear indicators of success, we can embrace imperfection, find joy in the journey, and above all, discover the middle way to move forward.
The Wisdom of Focused Measurement
"The master who measures a thousand things measures nothing at all."
By Unkown
This Zen wisdom reminds us that tracking too many metrics can cloud our vision. Like a gardener monitoring soil health, water levels, and plant growth, we must select our measurements with intention and focus.
The Success Metrics layer
The three parts of this layer have a significant impact, they are the reminder for us that we are no longer enablers we are impact creators, meaning that we move from being in reactive mode to more proactive
The KPIs - We have space for only six KPIs in total so you need to be very strategic on which KPI you pick here, bellow I will share with you my way of filling them up
The Principles - This is what guides us when we make decisions, we need to set clear Principles, go behind just a title, and help us have a northern star for data initiative decisions, when we will talk in the next series about the yearly data strategy, they will come very handy.
The change management - This is an overview of how we communicate our vision on the one side, on the other how we ensure everyone speaks the same language when it comes to data, but above all know what is expected of them going forward.
Understanding Success Metrics
The Two Pillars (Data ROI & Utilization)
When we measure anything, consistency of the KPIs is the most important aspect of our success, in past newsletters I spoke many times about the two KPIs I truly believe are the key to any data team's success and for defining a strong impact created by them which are.
But before we jump to them, let me explain why out of the six KPIs you can pick you should pick two that will be a strategic indicator of your overall dashboard success, the idea behind it is to have two main indicators for our data health status I was thinking about them for many years, each time I had some in mind but each time I realized that at some point they lose relevancy or they are not indicating the true nature of our data ecosystem, and so I arrived to two KPIs I truly believe are a reflection of the data ecosystem and are a direct hint to the data team performance.
Data ROI: The measurable financial return on data investments. This means tracking actual cost savings and revenue generation from data initiatives, not just theoretical benefits.
Data Utilization: The extent to which data transforms decision-making across the organization. This isn't about how many people access dashboards – it's about how fundamentally data changes how the business operates.
These core KPIs are like the sun and water of your data garden - essential, unchanging elements that support all growth.
The idea behind the Data ROI is that it gives us a clear indication of how effective the data is used in the organization, if the data team processing engine is weak and not cost-effective it will show increased costs and take down the ROI, it a symbiotic relationship when the infrastructure is not effective you will see it, when the processing is garbage you will see it’s not generating profits, and in this way we can evaluate the impact of our operation.
Data Utilization on the other side is a great indicator of the waste created by the organization, data ingested and never used because no one knows about it or the quality is not good, dashboards created that have never been used, and so on, this indicator tells us how well we use our data, and together with the Data ROI it tells us how well we leveraging the data we have.
📈 Case Study: Zalando's Data ROI Journey - Or how to meassure your revenue
Challenge: Manual campaign management consuming 4-6 hours daily
Solution: Implemented data-driven automation
Results:
- Eliminated daily manual work
- Reduced budget wastage
- Generated positive ROI exceeding infrastructure costs
Key Learning: High utilization isn't always the goal; focus on value generation
Supporting Metrics in Practice
Beyond our core KPIs, we can select up to four additional metrics that align with our current goals. Like seasonal plantings that change with specific needs, these metrics can evolve as objectives are met:
Share of certified data products
Data quality score
User adoption rate
Time to insight
When you set them again think about the future objective you wish to achieve, for example, I worked with a company whose goal was to reduce their 329 dashboard down to 10 dashboards only with no more than 15 KPIs in each dashboard, so they created two KPIs for them to measure number of dashboards and average KPI per dashboard, the thing that I asked them to do was to explain what will they do if the trend goes up or down and what it means to the different teams.
To tell you I loved this KPI? I can’t! However, for them, it was an indicator that they are in the right direction with their future vision for the visualization not to have more than 10 dashboards and have them built with Python to reduce the costs of Looker and ensure KPIs are not being manipulated or changed, so they get also the consistency.
The outcome of their goal would be savings of over 20,000 euros a year, the other capability they wanted to have was a cheaper processing engine which would save them another 12,000 in processing fees, and they wanted to set a KPI to reach its savings, 32K in budget, here is when I stoped them because of a few issues, first of all while reducing the number of dashboard is great and moving out of using Looker will save money, the amount of data will change with time, which will increase the processing costs and the only thing they can do is optimize it and I urge them to look on other KPIs as a measure.
Remember: When one goal is achieved, its metric can be replaced with a new one - but our core KPIs remain constant.
Principles for Success
When we talk about principles we talk about the guides we need to ensure we pick the right initiatives, while the KPIs help us measure whether are we successful or not and whether are we improving our performance, the principles are there to guide us on why we pick initiative X or initiative Y, it’s not money value, it’s not about strategy, it’s about helping us understand how we envision our data ecosystem if it’s sharing your data to ensure we don’t have silos or data as a product which ensures we don’t ingest data that is not following the data product concept.
Like a trellis guides climbing plants, clear principles support your data ecosystem's growth. Each principle should follow the CLEAR framework:
Connected to vision
Limited in number
Explicit in meaning
Actionable in practice
Reinforced regularly
Example Principles:
"Data as a Product": Every dataset must serve a clear purpose, it is treated with a data mindset has clear documentation, releases and changes logs, and is certified to be the true source of data.
"Share by Default": Data should be accessible unless restricted, and all data that is not PII or Financial should be available for consumption through the Data Marketplace, it needs to have a list of producers and users to ensure visibility, and it has to be logged in the catalog for visibility towards the entire organization.
"Quality at Source": Ensure data quality at the point of creation bad data should not be ingested into the warehouse and should already be rejected in our access point.
Change Management Essentials
We need to understand that our vision will have an impact, for example, one example I saw recently in a vision talked about replacing the buying agents with machines, meaning a minimal touch of humans in the buying process to increase efficiency and get the best results, it means you no longer need to have an entire department and it may be one person, but they need to be trained how to use the new system and quality check it.
When we talk for example about having self-serve analytics which everyone likes so much we need to discuss what skills people who wish to have access to it will need to have, is it basic SQL knowledge, or the ability to explore the data catalog?
Another thing we need to think of, we will need to prioritize things, some are problems right now on the table of parts of the organization, how do we communicate to them why we pick capability X and not Y, and also how long-term X will naturalize the issue they have.
This area is for us to design the communication plan, the role and required skills, and learning and support, one thing it’s important to say all the artifacts that will be created during each of the layers won’t fit into this board, they will need to be kept somewhere and open for anyone interested to learn more, we are all about transparency, this is why I urge you to use monetary values when communicating the board because it will reduce the ability of people to resist it or to self ready and understanding the thinking process behind it.
Like introducing new plants to a garden, implementing change requires care and attention:
Communication Plan:
Regular updates on vision progress
Clear channels for questions
Success story sharing
Role Clarity:
What teams can expect from data leadership
What's expected from each department
How to request support
Learning Support:
Training resources
Documentation access
Mentorship opportunities
Principles for a Balanced Data Ecosystem
How can teams build a system that avoids constant firefighting and delivers sustainable impact? Here are some guiding principles:
Every KPI Must Align with Business Objectives: Measure what matters to your strategic goals.
Prioritize Data Quality Over Quantity: It’s better to have fewer, actionable insights than excess, unused data.
Governance is a Shared Responsibility: Break silos by engaging all stakeholders.
Balance ROI and Utilization: Monitor both metrics and ensure they align. Over-utilization can indicate inefficiency.
Create a Shared Reality: Collaborate to understand gaps and prioritize initiatives. Translate problems into monetary terms where possible.
Exercise: Choosing Your Impact Metrics (15 minutes)
Objective: Identify key metrics that best measure your data team's impact on the organization.
Step 1: Current Impact Assessment (5 minutes)
List 3 ways your data team currently creates value
Estimate the monetary impact of each (cost savings or revenue generation)
Note which impacts are easily measurable vs. hard to quantify
Step 2: Metric Selection (5 minutes)
Name your KPIs
Write down specific formulas for how you would calculate each
Write down how will you react or how the data team should react when the trend goes up or down
Identify any gaps in data needed for these calculations
Step 3: Reality Check (5 minutes)
For each metric you've chosen:
Can you measure it today?
What would you need to start tracking it?
How often would you review it?
Share Your Insights Share your chosen metrics and reasoning in the comments below. What challenges do you anticipate in measuring them? How might they help demonstrate your data team's value to leadership?
Reflection Questions
How does the meditation story reflect your own experience with data metrics?
What current metrics might you need to let go of to focus on Data ROI and Data Utilization?
How might your organization's data culture change if you implemented these success metrics?
Before You Go:
Vision Board Completion Checklist
Have you documented your core KPIs (Data ROI and Data Utilization)?
Are your supporting metrics aligned with future capabilities?
Have you established clear principles using the CLEAR framework?
Is your change management plan comprehensive?
Have you scheduled your first 6-month review?
Coming Up Next: Our Data Strategy Journey continues
Mark your calendars! Starting January 8th 2025, we'll begin a new series focusing on transforming your Data Ecosystem Vision Board into an actionable yearly data strategy. We'll explore how to:
Turn your vision into concrete initiatives
Prioritize projects based on impact and feasibility
Create quarterly milestones
Align teams around clear objectives
Don't miss this practical guide to making your vision a reality. Subscribe now to ensure you receive each installment of this essential series.
Call to Action
Try applying this framework within your organization. Start small. If you encounter challenges, I’m here to help refine and tailor it to your needs.
Let’s tackle this together. Book a free, no-obligation call or share your progress. Your feedback will not only improve your outcomes but also help refine this framework for others.
🔗 Book a Workshop Discovery Call
📧 Email me directly by replaying this email
💭 Share your experiences in the comments
Together, we can build a data ecosystem that drives real value for your organization.
May your data flow with purpose!
Lior
