Ship the Test

The single most useful constraint in growth is: it has to ship this week. Not because speed is a virtue in itself, but because a week forces you to cut a bet down to the one change that actually tests the idea.

Most bets are too big on the first pass. The redesign becomes a re-platform; the new flow becomes a new feature. Cut relentlessly toward the smallest change that exposes the mechanism you're betting on:

• Fake-door first. Before building the feature, ship the button. If nobody clicks it, you just saved three weeks.
• Manual before automated. Do the thing by hand for ten users before you write code to do it for ten thousand. If it doesn't move the needle by hand, automation won't save it.
• One surface, one change. Test the checklist above the fold, not the checklist and a new empty state and a nudge email.

If a bet genuinely can't be shrunk to a week, that's information: it belongs in a different process than your weekly loop, and it shouldn't block the eleven bets that can.

"Move the checklist up" is a vibe. It's not something you can build, ship, or measure, and handed to two different people it produces two different products.

A spec turns the bet into a ticket small enough to build and precise enough to check. A good one is four lines:

• The change: exactly what's different, concretely enough to build without a meeting.
• The reasoning: the leak it plugs and the mechanism you expect.
• The metric to beat: the one number that decides it, with a baseline.
• The guardrails: what must not get worse.

Written out, that's a ticket you can hand to an engineer, drop into Cursor, or build yourself, with no ambiguity about what "done" or "worked" means.

There's a real difference between shipping a change and running an experiment, and the gap is where most teams fool themselves.

A launch flips the change on for everyone and watches the number. If it goes up, you celebrate, and you'll never know whether the change did it or whether it was the season, a press hit, or luck. An experiment is built to give you a clean read:

• Split the traffic. Hold back a control group and compare against it, not against last week. If an A/B split is too heavy for your stage, a clean before/after with a stable baseline can do, but know what you're giving up.
• Pick one primary metric. Decide before you ship which single number wins or loses this. One. Choosing after the fact means you'll find something that went up and call it a win.
• Keep guardrail metrics. The primary metric can improve while something worse happens off-camera. Watch the one or two things that must not regress.
• Decide the sample and the duration up front. Roughly how many users, and how long, before the result means anything. Underpowered tests produce confident nonsense.

You can do everything above and still get a garbage answer by contaminating the experiment. The usual suspects:

• Peeking. Checking every hour and stopping the moment it looks significant. Early data swings wildly; stop-when-happy turns noise into a "win." Set the duration and wait.
• Two changes at once. Ship the checklist and tweak pricing in the same week and you own neither result. One clean variable.
• Seasonality. A test that runs across a holiday, a launch, or a payday is measuring the calendar as much as the change. Note what else was going on.
• Novelty effects. Existing users engage with anything new for a week, then drift back. If a bet targets new-user behavior, read it on new users.

None of this requires a statistician. It requires the discipline to design the read before you're emotionally invested in the answer.

You do not need an engineering team to run this loop. That's the point of scoping to a week and writing a real spec: the bet is now small enough and clear enough that one person can ship it.

Build it yourself if you can. Hand the spec to a single engineer if you have one. Or paste it straight into Cursor and let an agent implement it against your codebase, the spec was written to be executable, not admired.

Whatever the path, protect momentum over polish. A slightly rough test that ships Thursday and gives you a real read beats a beautiful one that's still in review next month. The loop only compounds if it keeps turning.