Executable AI Safety Claims

Cambridge MA

Turning published AI-safety claims into runnable, independently reproducible test packages that expose fragile assumptions, hidden failure modes, and unsupported conclusions.

Project Details

Published AI-safety claims can be difficult to reproduce because code, test data, assumptions, and evaluation procedures are often incomplete/fragmented. I plan to select several safety claims and convert each into a runnable package. Each package will include code, synthetic test cases, validation checks, failure examples, documentation, and reproducibility results. I will conduct the work through DBbun as the sole principal researcher and publicly release the resulting tools where feasible.

Theory of Impact

AI decisions may depend on evaluations that have not been independently reproduced. Weak/misleading safety evidence can create false confidence. Executable reproduction allows researchers to test assumptions and detect fragile conclusions. Better verification improves the evidence available to labs/evaluators/policymakers. This does not solve alignment directly, but reduces one pathway by which unreliable safety claims could influence high-stakes deployment decisions.

People

Uri Kartoun

Founder & Principal Researcher

Grants Received

Add grant

NIH OligoTox Open Data Challenge — Phase 1 Award

from NIH

$10,000

Funding Asks

grantmaking.ai Launch Round

Applied

Minimum

$20,000

Ideal

$49,500

How the money will be spent

At the $20,000 minimum level, the project would support a 12-week pilot reproducing three AI-safety claims as runnable benchmark packages. Funding would primarily support research and software development, with smaller allocations for compute, model API access, hosting, documentation, and independent technical review.

At the $49,500 ideal level, the project would run for approximately six months and cover eight to ten published claims. The expanded version would include a standardized reproduction framework, synthetic stress-test datasets, automated validation and regression testing, documented failure cases, an open repository, and external review of selected outputs.

Approximate ideal-budget allocation: $38,000 for principal-researcher time and software development; $4,500 for compute and model access; $4,000 for independent technical review; and $3,000 for hosting, documentation, and dissemination.

Discussion

Loek Verdonk15d

Hi Uri,

Great work on turning AI-safety claims into runnable, independently reproducible test packages. I especially appreciate that your method is designed to expose fragile assumptions rather than simply confirm a convincing claim.

No Silent Landing is exactly the kind of project I would want pointed back at itself. Our application includes independent external adversarial testing for a human-authorization and continuity testbed. If the grant lands, would attacking our claims and fixtures be something that might interest you?

Even without the grant, we intend to publish our claims and test cases in a runnable and reproducible form, so there may be a useful seam between our projects.

Would love to hear how you decide when a safety claim is ready to become executable.