Try the idea
Small examples with explicit boundaries. Download, run, change one assumption.
Engineering workbench
Download the workbench · Python 3.10+ standard library; Java examples require JDK 17+.
python -m unittest -v
python workbench.py
java LegacyExamples.javaRun these commands from the extracted folder. The Python suite tests replay availability, stale ownership, evidence coverage, hierarchical labels, prevalence, and paired uncertainty. The Java program checks the corrected collection, prime-number, queue, cleanup, and authorization examples.
The report is deterministic for the supplied illustrative data. It includes raw paired outcomes and an adapter experiment that demonstrates why dropping warnings can hide failures. These are engineering fixtures, not a benchmark of any language model.
Evaluation authoring
Download the eval companion for scoring contracts, example judgments, and validation. Read the guide before changing weights.
Progressive disclosure
Download the SQL/MCP fixture. It exercises discovery and incomplete coverage with a controlled fixture. It does not implement a production MCP transport or SQL sandbox.
Source and limitations
Every workbench function is deliberately small enough to inspect. In-memory ownership checks need database transactions in production; replay does not simulate alternate interventions; a synthetic confidence interval says nothing about your deployment until you supply representative paired outcomes. See the included README for commands, expected outputs, and extension points.