Infrastructure, Training, Culture, and Incentives
- Infrastructure
Experimentation—especially regarding product features rather than just copy—requires substantial infrastructure, such as “feature flagging.” This becomes exponentially more challenging in regulated industries. Much of this testing occurs on the back-end or during sensitive phases like user signup. Consequently, decisions about data transmission and user permissions (e.g., registered vs. unregistered) require consensus across departments that rarely collaborate, such as legal, security, growth, and core engineering. - Training
Implementing experiments that span multiple touchpoints requires cross-functional coordination, yet training these diverse teams simultaneously is difficult. While organizations often invest heavily in learning and development, these efforts rarely engage cross-functional groups together. Without shared training, teams lack a common language and methodology for experimentation. - Culture
Building an experimental organization requires a fundamental shift in values. In traditional organizations with deep-seated hierarchies, asking “Is that true?” or “How do we know?” can be culturally abrasive. Transitioning from a top-down decision-making style to one that embraces questioning and data-driven challenge is often underestimated. You are asking for a culture where evidence outweighs rank, which is a significant departure for many established companies. - Incentives
Incentives and policies are the mechanisms that enforce culture change. To encourage experimentation, organizations must ask: Is “experimentation proficiency” part of the annual review? Are there enterprise-wide competitions? Do we offer awards for the best experiment? Research suggests that to foster this behavior, you must reward the act of experimenting, not just the outcome. Since many experiments will not result in immediate value, penalizing “failed” experiments discourages risk-taking. Incentivizing the rigorous pursuit of answers—regardless of the result—is key to finding the wins that actually matter.