· 1 min read
Why Experimentation Beats Intuition Every Time
Most product decisions are made by HiPPO — the Highest Paid Person's Opinion. Here's why that's leaving revenue on the table, and what to do instead.
Why Experimentation Beats Intuition Every Time
Published: · 0 min read
Most product decisions are made by HiPPO — the Highest Paid Person's Opinion. Here's why that's leaving revenue on the table, and what to do instead.
Most product decisions are made by the HiPPO — the Highest Paid Person's Opinion. The VP says "users want X", and the team builds X. No data, no test, no validation.
The problem? Humans are systematically bad at predicting what will work. Even experienced PMs get it wrong more than 60% of the time when predicting A/B test outcomes.
The Cost of Guessing
Every wrong guess is a missed revenue opportunity. If you ship a feature that doesn't convert, you've spent engineering time, and you're left guessing whether to double down or abandon it.
Experimentation gives you a framework for making decisions with confidence. You still make bets — but you make them small, fast, and reversible.
How to Start
- Pick your north star metric. One metric to optimize for. Not a dashboard of 20.
- Build a backlog of hypotheses. What do you think will move the metric, and why?
- Prioritize by impact × confidence ÷ effort.
- Run the experiment. Respect the sample size. Don't peek early.
- Ship the winner. Document the loser (you'll need it).
Experimentation isn't about running more tests. It's about building an organizational habit of learning.
Subscribe
Get notified when I publish something new, and unsubscribe at any time.