Six weeks in at a new company. I’ve spent most of it rewiring incident response. The pattern I found is one I’ve seen enough times to write about, and it isn’t specific to any one place: startups that outgrow the “engineering team fits in one chat” phase almost always hit the same wall.
The starting state told me most of what I needed to know.
Enterprise IT incident response is optimized for a fixed set of internal customers who can wait. Hyperscaler incident response is optimized for external paying customers who cannot.
One Slack thread for all incidents
Every alert, every page, every “is this a big deal” conversation landed in the same channel. When two things happen at once, the signal collapses. Nobody can find yesterday’s incident. Nobody knows who’s on point. The incident that matters gets buried under the one that doesn’t.
This isn’t unusual for startups that graduated from three engineers in a garage. It’s the first thing to grow out of. Most teams don’t notice until an incident hurts them.
The tooling gap
The paging tool had limited functions. Grafana alerts weren’t wired to anything durable. The Jira instance was shared between customer-facing work and internal work, so anyone with a login could see anyone’s tickets.
What I stood up:
| Layer | Before | After |
|---|---|---|
| Detection | Grafana alerts to a Slack channel | Grafana → PagerDuty → JSM |
| Paging | Limited-function tool | PagerDuty rotations by service |
| Tracking | Ad-hoc Slack threads | JSM projects, one per team |
| Isolation | One Jira instance for everyone | Separated customer and internal spaces |
| Escalation | Ask around | Major Incident guidelines |
None of this is glamorous. All of it is table stakes for anyone serving external customers with an SLA.
The mental model shift
Enterprise IT teams optimize for cost per ticket and time to resolution. Hyperscalers optimize for the customer never noticing. Those two goals point at different tools, different rotations, different comms cadence, and different definitions of “resolved.” Getting the model right upstream makes every downstream tooling decision easier.
Where AI actually helped
I used AI heavily to compress the work. SLA drafts for leadership. Operations dashboards leadership actually reads. Availability views in Grafana with the right unit selection. Work that would have taken a small team a quarter, done in a couple of weeks, with AI writing the boilerplate and me reviewing.
AI didn’t replace the judgment call. It replaced the typing.
That’s the pattern I keep coming back to. Humans in the loop for what matters. AI in the loop for everything that used to take too long.
Six weeks in. Still early. But the difference between “one Slack thread” and a working incident stack is already visible in how the team responds to the next thing that breaks.