Evidence requires
Provenance
A precise number can tell the wrong story. Follow a claim back through the query, the cohort, and everything the summary left out.
Part of Trustworthy Autonomy ↗Chapters in this collection 02 / 09
Provenance is the history behind a piece of evidence: where it came from, how it was obtained, and what happened to it along the way.
“Errors are at 0.4%.” The number arrives in a neatly formatted report. It has a decimal point. It sounds settled.
Before an autonomous agent uses it to approve a document parser, one question matters: 0.4% of what?
The number that changed its meaning#
Imagine an agent evaluates a document parser across a million pages. The headline error rate looks excellent, so it recommends using the parser for every incoming document. Another engineer asks to inspect the dataset.
The query combines typed and handwritten pages. Handwritten pages make up only a sliver of the dataset; their failures almost disappear inside the total. The calculation is correct. The conclusion has borrowed a meaning the number never had.
This is why provenance must include filters, populations, time windows, and transformations. A dashboard link alone leaves too much to reconstruct.
Can someone else follow the trail?#
The philosophical question is about testimony. Much of what we know comes through other people and instruments. Trust becomes more defensible when we can examine the route by which a claim reached us.
Agents make this urgent because they summarize one another. “Low error rates on typed English pages” can become “Low error rates,” then “The parser is reliable.” Each retelling sounds cleaner while carrying less evidence.
Follow the claim below. The numbers are invented; the loss of meaning is the point.
Follow the 0.4%
An illustrative document-processing benchmark. Each view reveals more of the same evidence.
- Reported errors
- 0.4%
- Population
- All pages
A reassuring total
- What happens
- The report says the error rate is 0.4% and recommends approving the parser.
- What it means
- We still do not know which languages, document types, or image qualities contributed to that number.
- What the system does
- Open the evidence behind the claim before relying on the headline.
Inspect the example record
{
"claim": "Overall error rate is 0.4%",
"scope": "all pages",
"cohort_comparison": "missing",
"decision": "insufficient coverage to approve every document type"
}- Typed pages
- 980 / 980,000
- Handwritten pages
- 3,020 / 20,000
A difficult cohort was diluted
- What happens
- The same evaluation snapshot contains 4,000 failures in one million pages: 0.4% overall. The typed-page cohort is at 0.1%; the handwritten-page cohort is at 15.1%.
- What it means
- The total concealed a severe difference. Document difficulty or scan quality may explain the difference; the headline alone cannot establish coverage.
- What the system does
- Compare matched languages and image quality, preserving the original counts.
Inspect the example record
{
"source": "labeled parser evaluation",
"window": "evaluation batch 17",
"query": "SELECT document_type, language, COUNT(*) AS pages, SUM(parse_failed) AS failures FROM evaluation_pages WHERE batch_id = 17 GROUP BY document_type, language",
"calculation": "sum(failed pages) / sum(all pages)",
"artifact_ref": "demo-snapshot-17",
"coverage": "all labeled pages in this illustrative batch"
}- Claim ID
- claim-17
- Causal attribution
- Unresolved
The caveat travels too
- What happens
- A second agent receives the counts, snapshot reference, population, and the open question about cohort mix.
- What it means
- Its summary can get shorter without quietly becoming more certain.
- What the system does
- Keep the claim ID and qualifications attached wherever the result is reused.
Inspect the example record
{
"claim_id": "claim-17",
"derived_from": "demo-snapshot-17",
"statement": "Handwritten pages fail more often in this evaluation",
"qualification": "scan quality and language mix need further comparison",
"retrieved_at": "14:11 UTC",
"fresh_until": "until the parser or evaluation dataset changes decision"
}Give evidence a return address#
A useful evidence record preserves the source, exact query, retrieval time, evaluation window, transformations, coverage, and a stable snapshot or reference. Derived claims point to their inputs. Summaries retain the qualifications that could change a decision.
This does not make every source truthful. A signed snapshot can preserve a faulty measurement perfectly. Provenance lets us locate and challenge the mistake; it cannot erase it.
The practical test is simple: hand a material claim to someone who did not watch the investigation. Can they reproduce the calculation, identify its limits, and trace the reasoning back to the evidence?
Every consequential number should come with a way back to the world it describes.
Further reading: the W3C PROV overview describes how data, activities, and people connect through provenance.