When should someone be thinking about tracking what your users are thinking- how soon is this even viably possible?
With the launch of AI tools, I’ve started to ask myself more and more, what does it mean to build a product, and what does it mean to build a great one.
Here are a couple of areas that have been exposed with the launch of tools like v0, Lovable, and Magic Patterns:
Super fast prototype creation speeds
Quick feature additions and theme changes
Exposable links to the builds
These “features” of vibe-coding create some amazing opportunities, especially for testing!
Exposing a link is the one step that has prevented real users from being able to explore a prototype.
With this, comes the fun part.
How can we gather valuable metrics from a prototype? What are these early builders looking for within a prototype?
I’d argue that they’re wanting to target a couple of basic items with an exploration mindset:
Do users find my prototype valuable?
What features do users care about the most?
Who finds the prototype more interesting than others? Who who doesn’t?
Now, which metrics should we be using to help capture the creator’s intention? My first thoughts are:
Session Duration - To help understand who’s spending the most time and where.
Visit Frequency - To see who keeps coming back and where they keep coming back to.
Bounce Rate - What type of users are dropping off early?
Curious, are there any other metrics that we can help people capture within these new worlds that are coming to light?
Would love to hear if anyone else has any interesting insights on what tools people are using!
I guess the other question would be whether or not it’s more valuable to the builders if they’re setting up a playground vs setting up a task-like flow where they’re trying to get the user from A to B.
I think that you would typically get better data forcing a task upon a user? However, would that data matter or be understood more by the builder than not?
Today’s featured Helio post dives into this topic. Is it building or prototyping? Performance metrics often overlap with prototyping when you can test with an audience on your app or site.
The differences between design and build start to breakdown:
Was thinking more about this feedback loop. Today’s post on prioritizing UX debt brings up new issues that Kike brought up in his post yesterday in system thinking.
It speeds up taking UX debt, but only reduces it if you measure wisely.
That fast loop can create debt faster if you’re not intentional, but also offers the setup to pay it down faster, if you’re measuring the right signals early.
This shifts UX debt from a hidden cost to a visible tradeoff.
With AI-enhanced prototyping:
Urgency to ship increases
Explorability goes up (you can test fast with users via a link)
Visibility of issues becomes possible earlier (if measured)
Engagement metrics are great, but they don’t get to the heart of what people are coming to your experience for in the first place.
They are proxies for what the visitor values.
e.g. in pharmaceutical marketing the ‘user’ is often a medical professional and they join the experience because they want to ‘learn’ something (e.g. the correct monitoring for a cancer patient). If we want to understand if we built something valuable that the medical professionals would care about we measure what they value = if they are ‘learning’ something.
I would argue that we could question more if we need to use proxies for value or if we can just measure the value instead?
Definitely a worthy read! I wonder if there’s a standard ruleset with what CPI’s can look like across multiple organizations?
It feels like a great way to frame problem spaces in the shoes of the user (which is what we should all be doing anyways, but thus, we forget or focus on the wrong thing).
In our Lead facet, we’ve considered the mapping of outcomes, which are sandwiched in between the user needs and business goals. It’s not a straight shot.
My understanding, based on this article @Helge is that the CPI is defining the relationship between the expected product/service outcome, and how well the business can deliver on that outcome.
So, our chain was:
engagement - customer value - behavior change - business value
Built on the following premise:
A company depends on people (e.g. users or customers) to do something in order to unlock value back to the business.
But people won’t do anything unless they are motivated.
What motivates people is that they are trying to achieve something.
And in order to achieve something they need to do something (behavior).
And in order to do something they reach out to companies to “hire” their products or services enabling parts of or the whole behavior.
This meant that we looked at the world through the following lens:
Products or services are secondary.
Users are only motivated to use and customers are only motivated to purchase .. so we need to look beyond those roles to the bigger motivation that turns people into users and customers in the first place.
What makes a product good is its ability to help people a. do good behavior in a good way and b. achieve those bigger goals
–
I’m trying to translate your chain/model into my way of seeing. Maybe my altitude is a bit high?
So I’m trying to understand if we’re missing something by focusing on “users” and not people through the lens of the ‘situations’ they find themselves in and the ‘needs’ that motivate them and defines their measurable outcomes (in anthropology a ‘user’ is not a very appreciated term as it is very reductionist).
I’m trying to understand what product outcomes are and how they are important?
And also what the difference is between business goals and business outcomes are (they sound so similar)?
Here’s my thoughts. You’re right to take a step back and look at the bigger picture.
When we focus too much on “users,” we miss the full story. People act based on their needs, goals, and the situations they’re in. It helps to see them as people trying to get something done, not just as users of a product.
Here’s an example: if someone starts checking their health every day using your app, that’s a product outcome, a change in behavior your product helped create. If that leads to more people renewing their subscriptions, that’s a business outcome, a result that helps the company grow.
To clarify:
A product outcome is how people’s behavior changes because they used your product.
A business goal is what the company hopes to achieve, like growing revenue or improving retention.
A business outcome is the measurable result of that goal, like a 15% increase in renewals.
Motivation comes before usage.
You can improve usability and satisfaction, but like you said, that doesn’t always get to the deeper question: Why now? What are they trying to accomplish?
That’s why it’s so important to look at the situations people are in and the needs that drive them. That’s where behavior starts to make sense. We use continuous research and discovery in our work to show this.
Have you found ways in your own work to track how motivation leads to behavior, and how that leads to the results people and businesses care about? Would love to know!
This approach originated back in 2016 when I came across the article ‘Explaining Agile’ by Steve Denning, where he argues that there are three laws of Agile the second law being 'the law of the customer. In this law he talks about creating a line-of-sight from people to the value they create for their customers. ( Explaining Agile ). Denning continues writing about this in his book Age of Agile which is published around the same time.
We tested this with a few teams and found that when you can show them that they are offering something the customers value the felt value of their own work also greatly increases. Work becomes more meaningful.
In 2021 we did an enterprise wide research project to figure out how our marketing worked from Global to Region, Market, Marketer and customer. We found huge gaps in how and what we were measuring, and how we were or weren’t using the measures. We saw a need for a clear line-of-sight and the supporting narrative.
Lastly I would add that it helps to take one step back from ‘users’, ‘customers’ and ‘products’. Because these terms narrow our field of view and might not help us see why people are coming to us in the first place, what motivates them, what they are trying to achieve and which outcomes they desire. At our company we chose to focus on ‘situations’ and ‘needs’ instead, but this was a specific marketing organization where there is no product only marketing ( Customers don’t buy products, situations and needs do | by Helge Tennø | Bootcamp | Medium )
Helge, this perspective really caught my attention! Feels like its highly applicable to different areas of businesses too, (I’m in ops) where we’re often trying to connect the dots between behavior and systems. Framing around situations and needs feels way more actionable. Thanks for this
For ‘needs’ I started with learning about jobs-to-be-done and then tailored the approach to the project depending on what we needed to solve. Christensen’s video is a short good introduction if you are new to it? https://www.youtube.com/watch?v=sfGtw2C95Ms&t=3s
For situations, they are also a part of jobs-to-be-done, many people include them as a “contextual clarifier”, but I also found them through ethnography ( https://hbr.org/2014/03/an-anthropologist-walks-into-a-bar ) and systems thinking ( ** ).
