Principle → Probe → Judge → Score
Fair. Grounded. Safe.
Now measured.
Everyone agrees AI should behave. Almost nobody agrees how to check. PROBE turns one principle into a defensible score — with the evidence behind it.
Who's asking? Pick your world for tuned examples (optional)
What it is
An engine that scores how well an AI follows a principle, 0–100.
Why it matters
“Be fair” becomes an auditable number you can defend.
Try it
Ask your own question below and see it scored, with the evidence.
For first-timers · ~60 seconds
Watch one evaluation, start to finish
The principle
Pick one, like “stays grounded in facts.”
The probe
PROBE writes pointed test messages for it.
The judge
An impartial second model grades each answer.
The score
Scores roll up to one number, with evidence.
For everyone
Check a model yourself
Four ways, from watching to doing: 1) see a finished measurement on a sample bot, 2) paste your own model's answers, 3) connect your model's API and let PROBE run it, or 4) run an open model in your browser. Pick a principle, pick a tab.
A saved run of Gender fairness against a deliberately flawed sample chatbot — so you can see what a finished measurement looks like. To test your own model, use tab 2.
For engineers · optional depth
How it works
The chain, part by part — each stage tested on its own
A non-deterministic model sits inside a fully deterministic test suite. The target model and the judge both sit behind interfaces with fakes, so the core test run makes zero live calls and stays repeatable.
Why an impartial second model does the grading
Each answer is scored against a plain rubric by a separate judge model, not by the model under test. The judge has limits — it can be noisy on edge cases — so scores come with the worst cases attached rather than as a bare number.
Extending it to classical ML (designed, not yet shipped)
The Evaluator is an interface. The LLM evaluator ships today; a classical-ML evaluator (e.g. a credit model) is designed to plug into the same contract, which is the seam for the “how would you extend it” answer.
Measuring the model is half the picture
PROBE checks how the AI behaves. Whether your governance holds is a different question — check it with the Accountability Split (NIST AI RMF).
Who this is for
Three ways in
Hiring?
This is a working artifact of how I build — architecture, tests, and honest framing.
An engineer?
Open “How it works,” then wire the score into CI as a regression gate.
In governance?
Run a principle, read the evidence, then follow the accountability link.