What’s the Point of a Post-Mortem?
What’s the Point of a Post-Mortem?
Organizations often conduct reviews after failures, outages, missed deadlines, and unsuccessful launches. The resulting documents can contain detailed timelines and accurate descriptions without producing much learning.
The problem is that explanation and improvement are different outcomes.
A post-mortem becomes useful when it changes how people understand the system and what they do afterward.
The Obvious Cause May Not Be the Useful Cause
On February 28, 2017, Amazon Web Services experienced a major disruption to its S3 service in Northern Virginia.
An authorized employee investigating a billing-system problem entered an incorrect input into an operational command. The command removed far more server capacity than intended.
The immediate cause was therefore easy to identify.
Someone entered the wrong information.
A review could have stopped there.
AWS Looked Beyond the Operator
AWS's published incident summary identified additional problems.
The operational tool permitted too much server capacity to be removed too quickly. Recovery also took longer than expected.
AWS modified the tool so capacity would be removed more gradually and added safeguards preventing commands from reducing critical systems below minimum capacity requirements. The company also identified changes to recovery procedures and communication.
Those actions addressed more than the employee's behavior.
They reduced the consequences of similar mistakes in the future.
Human Error Is Often Predictable
People make mistakes in complicated systems.
They enter wrong values, misunderstand instructions, forget steps, overlook warnings, and make decisions with incomplete information.
Training can reduce some errors.
It cannot eliminate human fallibility.
This means an important system-design question is not whether a human might eventually make a mistake.
It is what the system allows to happen when they do.
A robust process assumes that at least some errors will occur.
“Be More Careful” Is Usually a Weak Control
Imagine responding to the AWS incident only by reminding employees to check commands more carefully.
That might help.
It would still leave the underlying tool capable of turning another incorrect input into a major operational problem.
A safeguard changes the conditions.
Instead of relying entirely on perfect behavior, it limits what an error can do.
This is one reason the immediate cause and the useful cause can be different.
The first explains the trigger.
The second helps prevent recurrence.
Blame and Accountability Are Different
Post-mortems often become difficult when people worry that a blameless review means nobody is responsible.
That is not the necessary conclusion.
Accountability asks what responsibilities people held, whether those responsibilities were fulfilled, and what should change.
Blame tends to narrow attention toward the individual associated with the mistake.
Both may address behavior.
Only one necessarily encourages investigation beyond it.
Fear Can Reduce the Quality of the Review
A post-mortem depends on accurate information from the people involved.
If employees expect every admission to become evidence against them, they have incentives to protect themselves.
They may describe decisions more defensively.
They may minimize uncertainty.
They may avoid discussing shortcuts or informal practices.
They may withhold information about organizational problems that contributed to the result.
The review becomes less accurate precisely because everyone knows it matters.
Blameless Does Not Mean Consequence-Free
Organizations still need standards.
Deliberate misconduct, reckless behavior, repeated violations, or intentional disregard for safety procedures deserve scrutiny.
The useful distinction is sequencing.
First determine what happened and why.
Then evaluate responsibility using the complete context.
If accountability replaces investigation, the organization may punish the immediate actor while leaving the conditions that created the problem untouched.
Leadership Should Be Examined Too
Management decisions shape the environment in which employees act.
Leaders set deadlines, approve staffing, choose incentives, determine priorities, and decide how much redundancy or safety a system should contain.
A frontline employee may commit the final visible error.
The causal chain can extend much further.
A post-mortem that examines employee behavior while treating leadership assumptions as outside the scope of review is unlikely to reveal the entire system.
The investigation should follow the evidence rather than the hierarchy.
Hindsight Changes Our View of Decisions
After an outcome occurs, earlier warning signs often appear more obvious.
A customer leaves and previous complaints suddenly look predictive.
A product fails and every delayed milestone seems like evidence that failure was inevitable.
A system crashes and a previously accepted technical compromise looks reckless.
People making the original decisions did not possess that hindsight.
A fair review reconstructs what they actually knew at the time.
Ask What Was Knowable Then
What information was available?
Which risks were recognized?
What alternatives were considered?
What assumptions did the team make?
How confident were they?
Which information arrived only after the decision?
These questions help evaluate the quality of the decision rather than simply judging it by the outcome.
That distinction matters because good decisions sometimes fail.
Bad Decisions Sometimes Succeed
Suppose a team ignores several important risks and launches anyway.
Everything happens to go well.
If the organization judges the decision only by the result, the team may conclude that the process was excellent.
The next project may repeat the same behavior under less favorable circumstances.
Success can therefore reinforce poor judgment when luck goes unrecognized.
This is why post-mortems should sometimes follow successful projects too.
Success Can Hide Fragility
Imagine a project delivered ahead of schedule.
Leadership attributes the outcome to outstanding execution.
The review finds that employees worked unsustainable hours, one supplier delivered unusually early, and an experienced employee repeatedly caught problems that normal procedures missed.
The project succeeded.
Repeating the process may not.
A review allows the organization to distinguish robust practices from favorable circumstances.
Success Can Also Reveal What to Preserve
Not every discovery needs to be negative.
Perhaps two teams created an informal information-sharing routine that prevented delays.
Maybe an early experiment exposed a flawed assumption cheaply.
Perhaps a particular review process consistently surfaced risks sooner.
These are useful discoveries.
A post-mortem can convert accidental good practice into deliberate organizational practice.
The objective is understanding, not fault-finding.
Post-Mortems Can Become Defensive Documents
Some reviews are technically excellent and practically useless.
They contain detailed timelines, careful wording, and recommendations everybody agrees with.
“Communication needs to improve.”
“Teams should align earlier.”
“Risks should be escalated.”
These statements sound sensible.
They rarely define an action clearly enough to test.
Start With Expectations
A stronger review begins with what people expected to happen.
This matters because failures are differences between a model and reality.
What did the team expect the launch, project, system, or decision to produce?
Without establishing that expectation, it is harder to identify precisely where the model failed.
The original expectation also reveals assumptions that may deserve reconsideration.
Post-mortems are partly exercises in updating those assumptions.
Then Establish What Actually Happened
The next step is separating observation from explanation.
What occurred?
When?
Which measurable outcomes differed from expectations?
Which events are supported by evidence, and which remain disputed?
A clear factual foundation helps prevent the review from becoming a competition among preferred stories.
Only after establishing the outcome should the team decide what explains it.
The Difference Is Where Learning Happens
The central question is:
What explains the difference between what we expected and what occurred?
This is where simplistic explanations should be challenged.
Was the forecast wrong?
Was information missing?
Did incentives encourage a behavior nobody anticipated?
Did one safeguard fail, or did several fail together?
Was the successful outcome dependent on luck?
Multiple contributing causes may be involved.
Decide What Should Change
Once the gap is understood, ask what should be preserved, changed, or stopped.
Specificity matters.
“Improve communication” is vague.
“Require the engineering and customer-support teams to review high-severity customer reports together every Friday” is something an organization can implement and evaluate.
Recommendations should describe behavior or systems clearly enough that someone can later determine whether the change happened.
Otherwise, the post-mortem produces sentiment rather than action.
Assign Ownership
Every meaningful recommendation needs an owner.
Who will make the change?
By when?
What resources are required?
What dependencies could prevent implementation?
If nobody owns an action, the post-mortem has not actually transferred the lesson into the organization.
The insight remains trapped inside the document.
Decide What Evidence Would Show Improvement
Implementation itself is not enough.
A process can change without becoming better.
Suppose a team adds an approval step intended to reduce errors.
Does the new step catch meaningful problems?
Does it merely add delay?
Has the error rate changed?
Did employees create a workaround because the process became too cumbersome?
The recommendation needs a measure.
Return to the Post-Mortem
For changes that can be implemented quickly, review progress after thirty days.
Longer projects may require more time.
The important part is returning.
Did the change happen?
What evidence suggests it worked?
What unexpected consequences appeared?
Does the original explanation still hold?
This transforms the post-mortem from a one-time meeting into a learning loop.
Experience Does Not Automatically Produce Learning
An organization can repeat the same failure while becoming increasingly skilled at explaining it.
Experience only becomes learning when it changes the model people use to make future decisions.
Perhaps the review reveals that an approval process is ineffective.
Maybe a system assumed perfect human behavior.
Perhaps a successful project depended on an expert whose knowledge was never documented.
The useful output is a different understanding of how the system actually works.
The Real Test Is What Changes
The quality of a post-mortem should not be measured by document length.
It should not be measured by how many contributing factors were identified.
It should not even be measured by whether everyone agreed with the conclusions.
The more important question is whether the review changed something meaningful.
Did people revise an assumption?
Did a system gain a safeguard?
Did a useful practice become repeatable?
Did responsibility become clearer?
Did the team learn something it can use next time?
That is the real point.
A post-mortem succeeds when the organization understands something it did not understand before and acts differently because of it.
Comments
Post a Comment