Use this channel to talk about how we evaluate and compare design changes over time. Share test results, benchmarks, and what worked (or didn’t) between versions.
2 Likes
Hello from Helio!
One of the most frustrating aspects of sharing UX Metrics and insights is deciding what to do next. Moving teams forward with clarity and confidence is why we test with UX metrics to begin with, but when it comes to decision-making as a result of user testing and data, it’s important to remember that these often are not yes/no decisions. As the team seeks to align user needs to business goals, the team’s learnings can reveal that deeper areas of inquiry are required before moving toward implementation.
Let’s look at some examples of the kind of decisions that keep projects moving forward:
- Move to Implementation - The best-case scenario! The data validates the design. You’re confident it meets user needs and business goals, and you’re giving the green light to move towards implementation.
- Shift Method - The testing method you used might have been the wrong approach. Maybe you need qualitative insights instead of quantitative, or vice versa. You’re deciding to change your testing approach (e.g., from A/B testing to usability testing) to get better data. Depending on your process, shifting from validating with UX Metrics in the design phase may lead to A/B Testing prototypes and live variations.
- Iterate / Retest - Often the testing will reveal inconsistencies, problems, or new solutions in the design. Your decision is whether or not to refine the design, address the issues, and test it again. This is what iterative design is all about!
- Revisit Later - The design might have potential, but it’s not ready now. There could be technical hurdles, resource constraints, or a need for more research. You’re putting it on hold, but not killing it entirely. You might come back to it in a future sprint or release.
- Do Not Pursue - Sometimes the simplest, and necessary decision. The data clearly shows the design isn’t working – maybe it’s fundamentally flawed, doesn’t meet user needs, or isn’t aligned with business goals. As a PM, you’re deciding to scrap this design direction, but be sure to catalogue the results for future reference.
If you’re finding difficulty making decisions, there are a few things you can do to recenter yourself and the team to keep things moving.
- Revisit Your Design Initiative - You’ve defined your design initiative with an audience, user needs, and key UX Metrics for your Design Initiative. Have your learnings revealed a gap in your setup?
- Loosen It Up - Your testing might be overly broad, and require a different approach push things forward. Consider shifting away from focusing on modules and functions and look at content and strategy.
- Focus In - It’s also possible that you’ve already optimized your strategy and messaging, and it’s time to tighten up the visual language. Pay special attention to the way your design guides users through the flow and where their line of sight is guiding them.
Iteration is necessary as the team experiments and learns. Know when to shelve a concept, move it forward, and all of the options for iteration in between that help keep everyone on track and excited about the progress being made.
Key Questions:
- How do you keep your team aligned and moving forward through uncertain test results?
- What other kinds of decisions have you found make a difference mid project?
- Do you have an example of successful decision-making you can share?