CAIN-42 CAIN Studio

CAIN-42 evidence library

PROOFLOOP — FRESH evaluation (signed)

Evidence bundle: 0 claims, 0 of 0 invariants held.

Last reviewed 2026-11-08

The bundle's own README

PROOFLOOP — FRESH evaluation (signed)#

Don't trust this number. Verify it.

pip install cryptography curl -fsSLO https://cainstudio.online/verify_proofloop_eval.py curl -fsSLO https://cainstudio.online/evidence/proofloop-fresh-eval/record.json python3 -I verify_proofloop_eval.py record.json

The verifier imports no CAIN code. It checks the pinned Ed25519 key's fingerprint, the signature, the record hash, and recomputes every headline count from the per-finding table (it does not trust the summary).

What this measures#

The deterministic PROOFLOOP sweep ran over 17 serious ("BIG") findings across the FRESH corpus of real public agent/MCP repositories. FRESH was not used to tune any rule. Each finding was labelled by a human reviewer before the AI layer ran (the labels' SHA-256 was recorded first — see inputs). Then the shipped AI triage layer ran, and this record scores its decisions against those blind labels.

Count
BIG findings (rules only)17
Real bugs8
Plausible1
False alarms8
AI removed false alarms0 of 8
AI wrongly demoted a real bug0 of 8
Precision, rules only0.529
Precision, after AI triage0.529
Real-bug recall, after AI1.0

record_sha256: cf08bdab536b814d6d9ed937823d0f2d2b160408e0641246f86bb048fdf3b6e2 key: SHA256:2232b4add9b35d4016e5be270df4ed56e0dc9b39a0530ac69891073253aae90d

Heuristic detectors; a human labelled the ground truth; no third-party audit. Reproduce the sweep with python3 -m keystone.sweep <repo> and the AI layer with --ai.

Try CAIN-42 on your own agents

Create a free account and every new account starts with a 30-day trial of the full platform. Or try the sandbox first, with no account at all.

Create a free account →  ·  Try the sandbox  ·  See the whole ecosystem