Stress-test an encryption key rotation by proving that every dependency can accept the new key, failures remain recoverable, and the old key can be retired on evidence.
Rotation plans often describe key creation and activation while leaving decryption history, offline jobs, partner integrations, and backup restores implicit. Those edges decide whether retirement is safe. The exercise should follow actual key identifiers and workloads rather than generic boxes in an architecture diagram.
Inventory the cryptographic path
List the key purpose, algorithm, storage boundary, aliases, authorized principals, and every service that encrypts or decrypts with it. Include batch jobs, recovery environments, mobile clients, and third-party connections. Mark data that must be rewrapped or re-encrypted. Attach access logs, dependency scans, and results from the most recent restore test.
Define the phases for create, distribute, activate, observe, disable, and destroy. Each phase needs an owner and an exit condition. Write the point after which rollback would require recovering old key material, then review whether that action is permitted.
Failure drills
- The primary service uses the new key while a delayed worker still writes with the old alias.
- A regional secret cache keeps stale credentials beyond the planned overlap.
- A backup restore starts after disablement and cannot decrypt an older snapshot.
- An external integration accepts new signatures but its retry queue contains old ones.
- Audit telemetry drops during the observation window.
- An operator discovers an unknown principal using the old key near the destruction deadline.
For each drill, identify detection time, customer impact, containment, and the authority to pause. Require commands and dashboard links in the runbook. A statement that monitoring will catch it does not establish which alert fires or who receives it.
Prove retirement
Run the package in Pingpong with security, service owner, SRE, compliance, and recovery seats. Ask the recovery seat to restore a representative snapshot during the overlap. Ask security to explain what residual access remains after disablement. Ask compliance which evidence must be retained without preserving usable key material.
Related reviews include a token expiry policy, an SSO rollout, and the security reviewer seat. Browse the war-game decisions hub for adjacent controls.
Authorize destruction only after a query across the full overlap window shows no old-key use and the restore drill succeeds with the documented replacement path.