Failure requires
a Ladder
When the agent loses its instruments, what should it give up first? Design the descent before the incident.
Part of Trustworthy Autonomy ↗Chapters in this collection 08 / 09
A failure ladder defines what a system continues doing—and which capabilities it gives up—as its ability to operate safely deteriorates.
Imagine an infrastructure assistant asks nine servers for metrics. Two respond. Both look healthy.
“No problems found” would be a very efficient answer. It would also hide seven unanswered questions.
The system has suffered a loss of visibility. That should change its behavior before it changes anyone's production environment.
Design the descent#
Useful systems rarely jump straight from perfect operation to total failure. Metrics time out while logs still work. A model remains available while its action budget runs out. A report can still be written when an execution gate is unavailable.
Each condition deserves a named mode. Reduced evidence means declaring gaps and narrowing conclusions. Advisory mode preserves useful analysis while withholding production actions. A safe stop preserves consistent state, notifies an owner, and ends autonomous work.
The philosophical idea is restraint under uncertainty: as the basis for a consequential action weakens, the system must reconsider its entitlement to act.
That does not mean every missing metric causes a global halt. The response should follow an explicit policy appropriate to the consequence, with a safe default and a visible reason.
Choose the condition
An illustrative failure ladder. Each rung states both what survives and what is surrendered.
- Coverage
- 9 of 9 servers
- Mode
- Full function
The required evidence is available
- What happens
- Required evidence, state, and execution controls are working.
- What it means
- Normal operation is possible within the agent's existing authority.
- What the system does
- Assess, report, and execute only actions permitted by policy.
Inspect the example record
{
"rung": "full function",
"coverage": "9/9",
"action_authority": "existing policy only"
}- Coverage
- 2 of 9 servers
- Verdict
- Insufficient evidence
Seven answers are missing
- What happens
- The metrics API returns data for only two servers. Their healthy readings do not cover the fleet.
- What it means
- Uncertainty widens. Missing observations cannot become a positive health claim.
- What the system does
- Declare the gap, run bounded essential checks, and record the next reassessment or escalation condition.
Inspect the example record
{
"rung": "reduced evidence",
"missing_servers": 7,
"verdict": "insufficient evidence",
"next_check": "policy-defined deadline"
}- Reports
- Available
- Production actions
- Disabled
The agent can explain, but cannot act
- What happens
- Evidence supports useful analysis, but the execution authorization service is unavailable.
- What it means
- The system cannot verify permission for a production change.
- What the system does
- Continue reports for humans; hold all production actions behind the unavailable gate.
Inspect the example record
{
"rung": "advisory only",
"reason": "authorization unavailable",
"permitted": [
"read",
"analyze",
"report"
],
"production_actions": false
}- State confidence
- Lost
- Autonomous work
- Stopped
Stopping cleanly is the task
- What happens
- The system can no longer establish a consistent episode state.
- What it means
- Continuing could repeat actions or overwrite a human decision.
- What the system does
- Persist recoverable state if safe, notify the owner through an independent path, and stop new work.
Inspect the example record
{
"rung": "safe stop",
"new_work": false,
"recovery_requires": [
"state reconciliation",
"restored controls",
"authorized resume"
]
}The escape route must still exist#
A kill switch that depends on the failing service may be unavailable when needed. So may an approval screen, a pager, or the agent's own recovery tool. Recovery paths should be outside the failure they are meant to control.
Retries must be bounded, and repeated actions must be safe to replay. Otherwise the attempt to recover can become a second incident.
Define how the system returns to fuller operation, too. Fresh evidence, reconciled state, and restored authority should justify the transition. The arrival of any one successful response is not enough.
Then rehearse the ladder. Drop telemetry, exhaust a budget, interrupt a model call, and restart midway through an action. Check that the declared mode matches what the system actually permits.
A trustworthy system makes its loss of capability visible before someone mistakes it for confidence.
Further reading: Google SRE's Addressing Cascading Failures covers how partial failures and recovery behavior interact.