Designing Experiments (Glaringly Obvious)

In one of my first interviews, I sat down with Tristan Kromer to talk about how experimentation helps teams learn faster, not just ship faster. Tristan has spent years coaching startups and enterprises to focus on validated learning, turning every test into a chance to uncover truth, not just collect data.

I’ve gottent better since then, and there’s some good ideas in here.


Big ideas:

  • Build-Measure-Learn, the right way → Start with the right question, not just a new metric.
  • Clarify assumptions first → Define what success or failure looks like before you begin.
  • Run small tests, get big insights → Quick, low-cost experiments reduce risk and speed up learning.
  • Time-box learning → Limit experiments to drive momentum and sharper focus.
  • Experiment like it’s curiosity, not compliance → The goal isn’t more experiments; it’s better learning.

Experimentation shouldn’t be thought of as a checklist.

It really comes done to a culture shift that replaces opinions with evidence. When teams define assumptions and success criteria early, every test becomes a learning opportunity.

Curious, how does your team use experiments to learn, not just validate ideas? What helps you decide when to stop testing and start building?

Love seeing the evolution of these interviews! Keep it up :tada:

Was thinking about putting together a post on tis topic of testing and challenging ideas.

Tristan has a great quote in this short video segment:

Getting people to request a kick in the ass” is basically:

  • They choose learning over comfort.
  • They want clarity more than they want to be right.
  • They buy into the left side of the chart on purpose.