An open specification that binds AI evaluation criteria to a SHA-256 digest before the run, so a third party can check that those exact criteria existed no later than a given time and have not changed since.
An open specification that binds AI evaluation criteria to a SHA-256 digest before the run, so a third party can check that those exact criteria existed no later than a given time and have not changed since.
Project Details
Updated 09/10/26 · Provided by member · VerifiedPRML is an open specification for recording the criteria of an AI evaluation before it runs. The metric, the comparison rule, the threshold, the dataset and its hash, the seed and the system version are serialised to canonical bytes and hashed with SHA-256. The digest can be anchored at a public registry, which returns a signed receipt, an RFC 3161 timestamp from a public time-stamping authority, and an entry in a public transparency log. A sealed commitment lets the criteria stay private while the digest and the time are public.
What a record proves is narrow: that exact criteria object existed no later than that time and has not changed since. It does not prove the criteria were good, that the evaluation ran after that time, or that the result is correct.
I am one person. I wrote the specification and all four reference implementations — Python, JavaScript, Go and Rust. They produce the same bytes and run the same 21 conformance vectors. There is also a negative-conformance suite of 20 cases; Python and JavaScript run all 20, Go and Rust run the 16 that are not YAML-specific.
Where it is up to: the specification, the four implementations and the public registry are live and openly licensed. There is a GitHub Action and integrations with MLflow, Giskard and UK AISI's Inspect. The media type application/vnd.prml+yaml was registered with IANA on 3 September 2026 — that is a registration, not an endorsement. Two free tools are live. Nobody outside me has yet run any of this on their own evaluation. That is the honest state.
Theory of Impact
Updated 09/10/26 · Provided by member · VerifiedThe chain, as short as I can make it:
- Safety decisions increasingly rest on evaluation results — deployment gates, frontier safety commitments, statements to regulators.
- The acceptance criteria behind those results live in ordinary documents. A document carries no evidence of when the criteria in it were set.
- So a threshold loosened after the results were seen is indistinguishable, to a reader, from one fixed beforehand.
- The reader who matters — an auditor, a regulator, a counterparty, another lab — therefore cannot separate a real safety margin from a re-scoped one. They have to trust the claimant's account of the claimant's own timeline.
- PRML removes that one dependency. The criteria are hashed and time-evidenced, and the check can be done by someone who does not trust the claimant.
What I am not claiming. This does not make evaluations better, and it does not prove the criteria were the right ones. It also does not prove the evaluation ran after the record was made — binding execution to the record is the main open gap, and it is written into the specification as such. This is a small upstream contribution to whether safety claims can be audited at all, not an alignment result.
People
Updated 09/10/26 · By grantmaking.aiTeam Member
Discussion
No comments yet. Be the first to share your thoughts.