Research Flaws and the Hard Truth About Testing (Glaringly Obvious)

Measuring design
, , , ,
1

I dropped a video clip from my conversation with Max Speicher, who’s been challenging the way teams think about UX metrics, research quality, and what democratization really means.

His honest take on research done wrong, and what to do instead, has sparked plenty of thinking across our team.

“That’s where A/B testing falls short—because you only get behavioral data. You only see what happens. You never know why.”
Max Speicher

Max Speicher on the Real Cost of Bad Research

In our conversation, Max breaks down what happens when teams rush research or skip the fundamentals. Instead of helping, research can actually hurt product decisions, especially if it gives false confidence.

research-democratization
research-democratization1080×1080 99 KB

Research democratization doesn’t mean handing tools to everyone. It means giving people the right training, guidance, and structure so their findings actually lead to better decisions.

:speech_balloon: Discussion

  • Where have you seen UX metrics steer a team away from gut feel and toward clarity?
  • Have you experienced the downside of sloppy or rushed research?
  • How do you balance fast testing with meaningful insight?

Would love to hear your thoughts.

1 Like
2

The lack of attitudinal research creates big problems in A/B testing, particularly from a brand perspective. Often-times the brand and business goals are tied to comprehension and sentiment in the short-term, and recall in the long-term.

Behavioral or performance metrics give you near-term decision-making power, but we’ve seen teams optimize themselves into a corner.

3

Behavioral metrics are the linchpin. They reveal whether you’re solving the right problem… or just getting better at solving the wrong one.

They can crack things open or steer you into a dead end.