Operational readiness reviews are usually theater. A checklist circulates. Boxes get ticked. The service launches. Two weeks later it pages someone at 3 AM. The difference between a useful ORR and a useless one isn’t the list. It’s whether each item has evidence behind it.
What an ORR is
An Operational Readiness Review is the pre-launch checklist plus walkthrough that decides whether a service goes to production. AWS runs ORRs. Google runs Production Readiness Reviews. Different names, same idea: a defined gate that says this thing is ready to be paged about at 3 AM.
Why it matters
Skipping or faking the ORR shows up later as day-one outages, repeat incidents, and on-call burnout. In the org I ran at Oracle, structured pre-production validation cut change-induced incidents by 50 percent. The cost of the review is dwarfed by the cost of any one incident it prevents.
The checklist
The list isn’t the magic. The verification is. The right column is the only one that matters.
| Item | Verified by |
|---|---|
| SLO and error budget defined | Dashboard fed by a real SLI |
| Runbooks for top 5 failure modes | Runbook executed in staging, not just written |
| On-call rotation staffed and trained | A page in the last 7 days handled by primary |
| Monitoring covers the 4 golden signals | Dashboard screenshot, not “yes” |
| Rollback drilled, not documented | Timed rollback in pre-prod within last 30 days |
| Load test passed at 1.5× projected p99 | Report attached |
| Dependency map with each dep’s SLO | Diagram and table |
| Capacity headroom ≥ 30% at peak | Capacity model attached |
| DR / failover drilled in a production-like env | Last drill date < 90 days |
| Customer comms plan ready | Template plus escalation tree |
Pick eight. Verify each.
Where ORRs become theater
The most common failure mode is “checked but not verified.” Someone writes “yes” next to “runbook exists.” Nobody actually reads the runbook. The runbook says “page the dev team” and the dev team isn’t on call. The service launches. The runbook is useless.
One rule fixes most of this: no checkbox without an artifact attached.
Where AI helps
The best AI use cases here aren’t approving the ORR. They’re earlier:
- Evidence gathering. Auto-pull dashboard links, runbook last-modified dates, drill recency. The reviewer reads. The reviewer doesn’t fetch.
- Gap detection. Compare the proposed ORR against the last 12 incidents on similar services. Flag the categories that caused real pain.
- Drafting the summary. Status by item, what’s missing, what’s risky, what to revisit in 30 days.
What AI shouldn’t do is decide whether the service is ready. Readiness is a judgment call. Judgment is where senior operators earn their keep.
A checklist with verification is an ORR. A checklist without verification is a ritual. The on-call team can tell the difference.