Testing in the Foundry is not inspection. It is metrology.
A test suite is not a checklist. It is a calibrated measurement system.
The Gauge exists to measure alignment between:
The Oracle (declared requirements and constraints)
The Artifact (generated output)
If the Gauge itself is unstable, the production process cannot be controlled.
In the Foundry, verification does not simply validate an artifact.
It produces telemetry about the production system.
When the Gauge detects a deviation, the Foundry may apply a temporary repair to stabilize the artifact.
This repair is not maintenance. It is measurement.
A repair reveals how far the generator deviated from the Oracle. The repair itself is temporary; the lasting correction is applied to the specification or the generator so that future regenerations produce the correct artifact automatically.
Each repair provides signals about the production process, including:
Repair size – how much the artifact deviated from the specification
Repair frequency – how often the generator produces deviations
Repair stability – whether a small correction resolves the defect
These signals guide improvements to the Oracle, Generator, and architecture patterns used by the Foundry.
Repair measures the process.
Regeneration corrects it.
A single passing run proves nothing.
The Foundry measures yield and stability across regeneration cycles.
If the same Oracle and Generator configuration consistently produce passing artifacts—for example:
10 / 10 successful regenerations
the process is considered in short-run control.
If variation appears, the deviation is classified as:
Special Cause
A missing constraint, flawed prompt, or incorrect generator rule.
Common Cause
Expected variation within the current process capability.
This distinction allows improvement at the process level, rather than through manual maintenance of individual artifacts.
The Foundry evaluates the health of the generator using measurable indicators, including:
First-pass success rate
Repair frequency and repair size
Regeneration rate
Verification stability across runs
Together these signals describe the process capability of the generator.
When capability drifts outside acceptable limits, the Foundry recalibrates the Oracle or Generator before producing the next generation of artifacts.
A system can pass every test and still violate the real requirement if the Gauge is misaligned.
A passing test suite is only meaningful if it measures the right requirements and constraints.
Measurement must evolve as the Oracle evolves.