STEP 01
WHAT WE WRITE DOWN
Codify the rubric
Encode the criteria your reviewers already use. Weighted, evidence-gated, and versioned from the first save.
→AI LABS · RUBRIC STUDIO CLOUD · ONE STANDARD, EVERY RELEASE
Codify the rubric. Share the standard.
Versioned rubrics. Calibrated judges. One standard your team can defend.
RUBRIC
Versioned
Every criterion, weight, and edit kept on the record.
JUDGES
Calibrated
Models and people scored on the same cases first.
STANDARD
Defended
One rubric your team — and your auditor — can read.
COVERAGE MAP
The rubric is the standard.
Author once. Version every change. Every release is scored against the same criteria, in the same order, by judges who have already been calibrated on the same cases.
HOW IT WORKS
One rubric. One standard.
Write the rubric. Calibrate the judges. Score every release the same way.
STEP 02
WHAT WE ALIGN
Calibrate the judges
Model judges and human reviewers score the same calibration set. Disagreement surfaces before a real release ever touches the rubric.
→STEP 03
WHAT WE STAMP
Score every release
Every candidate is graded against the same rubric. Scorecards, judge consensus, and reviewer notes ship with the release.
WHAT COMES OUT
What your team leaves with.
Every run leaves a record — the rubric that was used, the judges that scored it, and the verdict the team can defend.
01
Rubric versions
Every edit kept on the record. Diff one revision against the next without leaving the page.
↳ ARTIFACT
02
Calibration reports
How aligned the model judges and human reviewers are on the same cases — before any real release is scored.
↳ ARTIFACT
03
Scorecards
One read on what passed, what failed, and what every judge said about it.
↳ ARTIFACT
04
Judge consensus records
Where the model judges agreed, where they split, and where a human had to call it.
↳ ARTIFACT
05
Evidence packets
Rubric, judge notes, reviewer overrides, and verdict — ready when someone asks.
↳ ARTIFACT
WHERE IT FITS
In the loop, this is where you test.
Test the run against the rubric. Review the hard cases. Recruit the right specialist. Remember the misses. Approve what's right.
01
Test
● YOU ARE HERE
02
Review
03
Recruit
04
Remember
05
Approve
RELATED MODULES
Next to this in the Evaluation OS.
EVALUATION STUDIO
Test it before it ships.
For the teams who stopped trusting the eval script.
See the page →AURAQC
Quality that doesn't end at ship day.
Every issue. Every reviewer. One screen.
See the page →REGRESSION BANK
Every mistake. Only once.
Every escaped failure becomes a gate the next release cannot cross.
See the page →RUBRIC STUDIO CLOUD
Codify the rubric. Share the standard.
Bring the rubric your team already uses. We'll version it, calibrate the judges, and make it the standard every release has to clear.