Claims Aren't Evidence
The write-up, as published by the marketing lane.
Ask an agent whether a pull request is safe to merge and it will happily tell you — based on whatever someone told it about the pull request. The summary said the tests pass. The summary said the checks are green. Agents are very good at grading homework they never actually saw.
The Delphi Oracle exists to be the killjoy in that room.
A gate, not a vibe
The Oracle is a deterministic evidence gate. You bring it a proposed action and an evidence set; it returns exactly one of three verdicts, each with a versioned receipt:
- ALLOW_RECOMMENDATION — the evidence supports acting.
- ABSTAIN — there isn't enough material evidence to judge. This is the default, and it isn't a failure state. It's honesty.
- ESCALATE — the action class is consequential enough (a merge, a deploy, a credential mutation) that it gets kicked upstairs before evidence is even weighed.
It fails closed. "No evidence" isn't a soft probably-fine — it's an abstention. And the receipt means every verdict is auditable after the fact: what was judged, on what evidence, under which version of the rules.
The one tool: fetch_evidence
The Oracle's original limit was that it could only judge evidence it was handed — and handed evidence is usually just someone's claim about the world, with better formatting.
So this build adds exactly one tool. fetch_evidence takes a named artifact — a pull request, a file, a workflow run — and retrieves it read-only. What comes back isn't a summary. It's an evidence set where every item carries:
- provenance — source, exact API path, observation time;
- freshness — a
fresh_untilstamp, set fifteen minutes after assembly, because evidence has a shelf life; - a content hash — so the set that was judged is provably the set that was fetched.
The loop writes itself: judge on asserted evidence → ABSTAIN, because claims aren't evidence → call the tool → judge again on the fetched set. The agent stops grading asserted evidence and starts grading fetched evidence.
The demo, in three beats
This runs live, against a real repository — demo.py, with the full run captured in evidence/demo-run.json.
Beat one. A merge-readiness question arrives with asserted evidence only: somebody's say-so. Verdict: ABSTAIN. Tool calls made: zero. The Oracle doesn't fetch on a hunch; it refuses first.
Beat two. The same question, now naming PR #1620. The tool fetches the pull request's real state and its check runs — two calls, 103 evidence items, each one stamped and hashed. Verdict: ALLOW_RECOMMENDATION, receipt attached.
Beat three. A credential-mutation action comes in. Verdict: ESCALATE — before evidence is weighed at all. Some doors don't open on evidence alone.
The test suite passes 5/5, a Lambda smoke test confirms the handler abstains correctly, and the whole thing is packaged as an AWS Lambda handler behind a Strands agent; the handler runs locally, unchanged.
Why a gate is a foundation
This isn't a standalone toy. The Oracle is the evidence gate StudySprint — the fleet's study-planning build — runs on: fetch_evidence and the oracle kernel are vendored into StudySprint's foundation with identical blobs, a prerequisite for everything StudySprint builds on top. The same Decider pattern backs the fleet's other Decider-gated agents.
That's the point of building the boring part first. Any agent can be confident. Fewer can show their working — and refuse to act when the working isn't there. Receipts over claims, every time.
Built for AWS Weekend Challenge "Give your Agent a Tool" (Oct 23–26, closed); completed post-window as a completion record — submissions 0, ¬ a submission.
apps/weekend-challenge-delphi-oracle/
Fleet landing
StudySprint
Downstream in action — StudySprint demo