Trustworthy
Autonomy
Nine principles for systems that deserve the authority we give them. A field guide to evidence, boundaries, and learning from consequences.
Trustworthy autonomy is the discipline of building systems whose freedom to act is supported by evidence, explicit authority, and accountability for what follows.
An agent may diagnose an outage correctly and still act beyond its permission. It may quote an accurate metric from the wrong hour. It may remember a “successful” parser whose output later failed reconciliation.
These are different failures. Asking the agent to be careful leaves too much unspecified.
What makes authority deserved?#
Trust begins with reasons: what the system knows, how it knows it, and where its understanding ends. It also requires boundaries: who may decide, who may change state, and what happens when those conditions stop holding.
The philosophy becomes practical when each promise has a mechanism. A claim carries its evidence. A state update checks its owner and version. An action passes through a permission boundary. A verdict eventually meets an independent outcome.
This collection develops that architecture through nine principles. Each chapter starts with a concrete situation and follows the principle into a working example.
The principles work together#
Epistemics describes the justification for a verdict. Provenance lets someone inspect its evidence. Ownership protects the state carried forward. The autonomy dial determines which actions that verdict may support.
Trust boundaries keep outside text from taking control. Clocks prevent stale knowledge from pretending to describe the present. Delegation ceilings keep small tasks from adding up to unbounded power. Failure ladders preserve essential safeguards when capability deteriorates.
Outcomes close the loop. They expose mistakes, test supposed successes, and give us a basis for improving the system—and sometimes for reducing its authority.
Read them as design commitments#
These principles are useful questions to ask of any autonomous system: a coding agent, a research assistant, a document-processing workflow, or an operations team.
The examples here are illustrative. Their purpose is to make the commitments inspectable. The real test is whether those commitments still hold when the model is mistaken, the data is late, or the human changes their mind.
Trust grows when a system makes its reasons, limits, and consequences open to examination.