Many teams track NPS, SUS, or CSAT faithfully and still struggle to answer the real question that matters:
“So what should we do differently?”
Acronyms make metrics portable, but they also make them shallow. People remember the letters and forget the question, the behavior, and the decision the metric was supposed to inform.
I think there are a few reasons people align with acronym metrics that are often siloed by function.
Acronyms create authority… three or four capital letters feel scientific. They signal rigor, even when the underlying question is simple.
They compress meaning into a label- acronyms act like containers. Once people learn them, the letters stand in for an entire method, scale, and interpretation. (despite truly knowing what they mean)
They spread inside organizations (easier than a definition)- teams copy what other teams use. Saying “What’s our NPS?” is safer and more familiar than proposing a new metric that needs explanation.
But here’s the downside and uncomfortable part. Acronyms gloss over nuance. Different teams often mean different things by the same metric, but the shared label creates the illusion of alignment.
Decided to lean into this idea, and write a post. It’s the number one thing people talk about when joining the forum. Creating design impact requires aligning these pieces.
Two experiences that changed my impression of Net Promoter Score (NPS).
A. When talking to former colleagues after they had implemented the NPS measure they told me: “if it’s bad we talk about it”. I think this is a great insight: An NPS holds NO information (it shouldn’t), it’s merely an alarm set off if things get bad. But since it holds no information it doesn’t tell people what has happened and they need to talk about what to do to fix it.
The NPS is an alarm, not insights.
B. A large telecom struggled to align their cross-functional team in shared activities to improve the customer experience (specifically broadband fall-out). Using an NPS they could produce a number that didn’t need expertise insight to be understood and didn’t lead to multiple interpretations. It’s a binary number: good or bad. As an alarm the NPS removes ambiguity and doesn’t position blame. The team needs to huddle together to figure out what causes the alarm, how and where to fix it.
My point is that it’s our own over-marketing of metrics that produce problems, not the numbers themselves.
This reminds me of the warning lights on a car’s dashboard. I don’t understand how the sensors work, where they’re located, or even what the light means some of the time. But it does tell me when to get my car into the shop to address a larger issue.
There’s certainly value to those metrics! But, when the dashboard light is going off for Adobe, Salesforce, or BMW, where do they go for the expertise to address it?
Like that. And just like the dummy car dashboard you start to ignore, it becomes less effective over time, and you struggle to trust whether it’s alerting you to an actual problem.
Good points @EricZ and @Bryan . I’m not saying I’m a fan of NPS, but I could see how one “signal” without any information could work as a unifier (because all specialty insight also creates silos). Dumbing it down to a number invited a conversation to figure out what was wrong.
But yes, these were early days in both projects (<1 year in both). Once the leadership moves on I’m going to assume the rest of the people will to.
And also: what do you do when the light goes of? In one project there was a cross-functional team responsible for the collaborative response. In the other project the alarm got the attention of the leadership and then the UX-team was asked to fix it … nobody else wanted to bother (or learn).
So it’s nuanced.
But at the time of me learning about these two examples there was a clear usefulness to the NPS and things were actioned because of it. Sometimes ‘simple’ rallies the support the experts need to get the job done well.
We argue that we can capture nuance with better tools that can be shared across the organization. Do people comprehend this? How much effort is required for the user?
These should be straightforward, based on how we talk about the problem.
@Bryan my experience is that you have to align it with how they already work. If you bring in a tool to change how they work you’re working from the wrong end of the change journey (nobody wants to buy a square peg for their round hole).
They need to want to change how they work first, and then your tool can enable the change.
Or… (maybe more likely) there is a leader who wants change and is looking for a tool to power it through. Still .. if the people aren’t onboard nothing will happen (except frustration and friction).
Your online content is creating a lot of change / mindset material. Are you selling Helio as a tool or does it include the change leadership as well (just a thought) …
We used this framework: mindset - skillset - toolset
Agreed! Design needs to come with the right metrics to influence the user experience, but ultimately, these need to map these outcomes and metrics to how the business sees the problem.
NPS is merely a thermometer being used as an on/off indicator.
I’ve met many companies adding feedback to their NPS. To me this is a mistake. It would stop a proper insights / expertise approach and simply try to categorize / simplify / misinterpret the underlying issue (if you ask anything you always get data, but that doesn’t mean the data is any good).
If the NPS is simply an indicator attention would be put to the problem and proper expertise would be asked to figure out the underlying cause.
Funny story in our company was that the NPS question itself proved no value (asking if physicians would recommend our content to other physicians was not found to have any indication on future revenue or growth), so we changed the question. And with the new question came better responses, but we still kept calling it an NPS because that’s what leadership was expecting and wanted to hear. Nobody complained
@Helge that’s great shows that there’s a disconnect between the value these feedback methods can provide and the expectations stakeholders have for a tried-and-tested data point for evaluating performance.
We think of the 28 UX metrics in Glare as a familiar set of building blocks. Each one helps explain a part of the experience problem. Call that proof. When you combine them, teams have hundreds of millions of ways to measure what’s actually happening.
It’s a bit like ordering coffee at Starbucks. The ingredients are mostly fixed, but you can assemble them in many different ways. The menu gives most people a starting point.
The difference is this: there’s no single “right” combo tied to an outcome. The framework doesn’t make the decision for you. You still have to think, choose, and apply judgment.
There’s no one universal answer for everything. And if I understand your approach your NOT saying: do this like this .. But rather offering a set of ingredients which the client can put together in the best way fit for them .. their own dish perfect for their company, context and customers? To build on your analogy.
What’ you have done is figure out the ingredients and come up with good suggestions and examples of dishes that have worked for others.
Yep, which is why we created a framework that could adjust to the experience problems you were solving. We’ve created a set of tools that can work together to create more leverage in your design/product process. Sequencing is common, but a process is not required.
Yes! We call them design and research stacks, and they can be used across experiences. Right now, we have about 50 concepts that can serve as starting points. Here’s an example for filtering.
This design stack uses 3 metrics and gives you a quick read on wether people understand how you are filtering.
shows that there’s a disconnect between the value these feedback methods can provide and the expectations stakeholders have for a tried-and-tested data point for evaluating performance.
