How to run a post-incident review

1. Collect the evidence before anyone meets

Memory is unreliable, especially under stress. Start from records. Ask the incident commander or scribe to gather these within a day of resolution:

• Alert history, including when each alert fired and when it was acknowledged.
• The incident channel or bridge notes, exported with timestamps.
• Deploys, configuration changes and feature-flag changes in the window before the incident.
• Graphs of the key service signals (error rate, latency, saturation) covering before, during and after.
• Customer-facing messages: status page posts, support macros, emails.
• Support ticket counts or other evidence of customer impact.

Normalize every timestamp to one timezone and say which one. Mixed timezones are a surprisingly common source of wrong conclusions.

2. Define impact and scope precisely

Vague impact statements make it impossible to judge whether follow-ups are proportionate. Pin down:

• What users could not do, or could only do slowly.
• Who was affected: all users, a region, a plan tier, internal staff.
• How long: from impact start to recovery, which is often earlier than detection.
• How much: failed requests, affected accounts or orders, if you can measure them. If you cannot, say the number is unknown.
• Data effects: any loss, corruption or exposure, confirmed or suspected.

3. Build the timeline

A good timeline records what happened and what people knew at each point. Include detection, acknowledgment, escalation, each significant hypothesis, each change made to production, recovery and resolution. Mark gaps honestly: "unknown, 10:12 to 10:20" is better than a guess.

The milestones you record here also feed your metrics. If you capture impact start, detection, acknowledgment, recovery and resolution for every incident, you can compute MTTA and the different kinds of MTTR consistently later.

4. Find contributing factors, then decide whether there is a root cause

A contributing factor is any condition that made the incident possible, made it worse, or slowed recovery. A root cause is a single underlying cause which, if removed, would have prevented the incident. In complex systems you will usually find several contributing factors and no single root cause. Do not force one.

Look in three places:

• Technology: the defect, the missing safeguard, the alert that did not fire.
• Process: the review step that did not catch it, the runbook that was out of date.
• Coordination: delays in escalation, unclear ownership, conflicting instructions.

Techniques like five whys help you dig, but stop at causes the team can act on. If you cannot yet explain something, record it as an open question with an owner, not as a conclusion. Atlassian's handbook describes root cause analysis as one section of a broader postmortem, alongside impact, detection, response and follow-up[Atlassian Incident Management Handbook].

5. Facilitate without blame

The facilitator's job is to get the full story. A few habits make that much more likely:

• Ask "what did you know at that point?" rather than "why did you do that?"
• Refer to roles ("the on-call engineer") instead of names in the written record[Atlassian Incident Management Handbook].
• When someone says "human error", ask what made the error easy to make and hard to catch.
• Separate hindsight from foresight: the review knows how the story ends; the responders did not.
• Keep the room small enough that people speak candidly.

6. Prioritize and own the follow-up actions

Prefer a few high-value actions to a long wish list. Rank candidates by how much they reduce the chance or impact of recurrence against the effort they take, and be explicit about what you are not doing.

Every action needs four things. Compare these hypothetical examples:

7. Follow up

Put actions in the team's normal tracker, not only in the postmortem. Check them on the due date. If an action is no longer worth doing, close it with a note explaining why rather than letting it rot. A short monthly look across all open postmortem actions catches drift early.

Hypothetical example

This scenario is invented for illustration. The company, systems and numbers are fictional.

A fictional retailer's order export to its warehouse silently stopped at 02:10 UTC after a credentials rotation. Nobody noticed until the warehouse called at 08:40 to ask why there were no orders to pick. The export was fixed at 09:25 and the backlog cleared by 11:00.

• Impact: orders placed over roughly seven hours (02:10 to 09:25) reached the warehouse late; no data was lost.
• Contributing factors: the export job reported success on authentication failure; there was no alert on "zero orders exported"; the rotation runbook did not list the export job as a consumer of the credential.
• Root cause: the team agreed there was no single root cause; the three factors combined.
• What went well: the warehouse escalated quickly; the backlog replay worked first time.
• Actions: make the job exit non-zero on auth failure (integrations team, verified by a test); alert when hourly exports are zero during trading hours (platform on-call, verified by a test alert); add all credential consumers to the rotation runbook (security engineering, verified at the next rotation).

Notice that detection took over six hours. That gap, not the credential rotation itself, is where most of the impact came from.

Facilitator agenda (60 minutes)

1. Opening (5 min): purpose, blameless rules, what the meeting will and will not decide.
2. Impact (5 min): confirm the agreed impact statement.
3. Timeline (20 min): walk the draft; at each decision point, ask what was known.
4. Factors (15 min): technology, process, coordination. Mark unknowns.
5. Went well (5 min): what to protect.
6. Actions (8 min): rank, then assign owner role, date and verification.
7. Close (2 min): who finalizes the document, when actions are checked.

Printable checklist