Role-play

Pretend you are the analyst

Pretend you are the analyst so soft causation and heroic dashboards fail before the real operating room hears them.

Optimistic readouts optimize for the slide that shows a clean uplift. The analyst seat does the opposite. It asks which sample actually supports the claim, which definition drifted mid-quarter, which confound was never logged, and which chart would reverse if you cut by segment. A clean chart is not evidence that the decision can absorb a bad Monday.

This is one of the core moves in a metrics review, experiment readout, or board pack war game: after you describe the upside, seat a careful analyst and make them try to decline. The output you want is not a pep talk. It is a short list of material objections, the exhibits each needs, and the edits that would survive them.

How to cast the seat

Name a real mandate: defend a named metric without inventing causation, absorb a board question without silent hand-waving, or underwrite a go/no-go without pretending the sample is larger than it is. Give constraints: data you will not invent, definitions you will not blur, and residual uncertainty you will not hide. Without constraints the seat becomes cartoonish. With constraints it produces questions you might actually hear.

Write the seat into the prompt as a named role with a mandate. Example: "Analyst: list the top reasons to pause this decision, the assumption with the weakest source, the sample or definition gap that worries you most, and the ten diligence questions you would send after the meeting. Stay inside a realistic mandate and avoid illegal tactics."

What to demand from the pass

  • Top reasons to delay, cut claims, or add a holdout.
  • The assumption that looks strongest and is least sourced.
  • The sample, segment, or definition mismatch that would worry you most.
  • What would make you approve instead.
  • The ten hardest diligence questions you would send in writing.

Run that brief in Pingpong against the real query notes, definitions, and experiment logs. Follow with a home-team response pass so you leave with edits and named owners, not only fear. When the decision sits near a pricing or roadmap change, run this seat after product and finance attacks so it can use earlier objections as ammunition.

When the plan leans on a single cohort, a single chart, or a single week of data, force the seat to price concentration risk in writing. Ask what happens if that cohort is out, if the definition drifts, or if the backup segment reverses the claim. Concentration that only appears in an appendix still counts. Pair with war-game a metrics review, war-game a research readout, war-game a growth experiment, pretend you are the CFO, and the war-game decisions hub. See how to run a Pingpong.