Platform ops

Stress-test a quota burst policy before soft ceilings outlive the window

Stress-test a quota burst policy by proving that each restore window and ceiling rule can place tenants inside the planned lane, survive override pressure, and avoid trapping operators inside a green burst slide that hides long-lived exceptions.

Quota burst policies often list a calendar while leaving restore behavior, override authority, and abort ownership implicit. Those edges decide whether a late product ask stops inside the quota or leaves a broken ceiling live under launch load. The exercise should follow actual quota tools, metering exports, and on-call paths rather than a clean capacity deck.

Inventory the burst path

List every quota class with its burst window, restore rule, override behavior, owners, and notification channels. Mark paths that cannot reverse without a manual meter edit. Attach the last three burst incidents with raw timelines and any waivers. Include the source of truth for open burst counts during the observation window.

Define the phases for grant, observe, restore, abort, and communicate. Each phase needs an owner and an exit condition. Write the point after which a stuck burst would require a different procedure, then review whether that action is still permitted. Capture maximum acceptable requester friction in measurable units, including which cohorts are excluded from the policy and why.

Include the calendar of known events for the next two quarters: product launches, partner cutovers, and support peaks that shrink the usable change window. A burst budget that ignores those dates will look calm until the week they land.

Failure drills

  1. A minority high-revenue tenant keeps a side-channel while the aggregate burst dashboard stays green.
  2. A restore has already left a partner surface without a trusted ceiling state.
  3. The primary metering dashboard lags beyond the planned observation window.
  4. An operator skips a burst gate because a renewal is close.
  5. Automated and human grants collide under the new restore rule.
  6. Abort authority is unclear at week end and the page lands on the wrong rotation.

For each drill, identify detection time, customer impact, containment, and the authority to force a burst rollback. Require commands and dashboard links in the runbook. A statement that monitoring will catch it does not establish which alert fires or who receives it.

Prove burst changes are timed and owned

Run the package in Pingpong with platform, product, support, and finance seats. Ask support which customer decision becomes unsafe first if ceilings still stick after the claimed restore window. Ask finance whether capacity can absorb a forced override. Ask platform to show the exact quota version or metering export used as the exit condition.

Related reviews include the platform ops lead seat, a tenant isolation rule review, and a shared infra budget review. Browse the war-game decisions hub for adjacent controls.

Authorize the published policy only after a timed drill restores usable quota hygiene inside the documented budget without an undocumented manual step.