Independent citation verification benchmark for AI systems

Raising $30,000active

A held out benchmark and public scorecard ranking frontier AI systems on citation integrity through independent verification not relying on the evaluated systems or their providers to grade their own outputs.

Project Details

The independent citation verification will build a held out benchmark for testing how frontier AI systems use citations. The benchmark will include supported, contradicted, fabricated, misattributed, outdated and ambiguous claim-source pairs. I will validate the evaluation pipeline against human labelled cases, run repeated tests across major AI systems and measure citation existence, attribution, claim support, coverage and fabrication rates.

The project will be led by Gideon Abako, founder of Ancestor Lab and the researcher who designed and deployed the existing citation verification pipeline. The grant funds the benchmark work. Paid annotators or adjudicators will review disputed cases and provide blinded human checks so that the same person is not creating the corpus, scoring every case and resolving every disagreement.

The outputs will be a documented held out corpus, a public evaluation methodology, a failure taxonomy and a public scorecard comparing major frontier AI systems on citation integrity. I will also publish a technical report, a selected public subset of the benchmark for reproducibility and the code needed to rerun the evaluation. A rotating private test set will be retained to limit contamination and gaming while each scorecard will record the tested system, model version, date, mode, prompt protocol, repeated run results and human audit findings.

Theory of Impact

X risk reduction depends partly on the quality of evidence used by researchers, labs, funders and governments making decisions about advanced AI. Frontier systems are used for research synthesis, forecasting, technical assessment and policy analysis. When those systems fabricate citations, misattribute sources or cite material that does not support the claim, weak conclusions can appear evidence based and shape high stakes decisions.

This project creates an independent benchmark that makes those failures measurable. By publishing model citation integrity scores, a failure taxonomy and reproducible methods, it gives the AI safety community a shared way to compare systems and detect when apparent grounding is false. Public measurement can also pressure developers to improve citation behaviour instead of relying on opaque internal claims about reliability.

The project's contribution is epistemic infrastructure that is helping people working on advanced AI governance, evaluations, forecasting and risk analysis distinguish supported evidence from persuasive fabrication. Better evidence integrity reduces the chance that major decisions about dangerous systems are based on invented or misleading sources.

People

Gideon Abako

Team Member

Grants Received– no grants recorded

Add grant

Funding Asks

grantmaking.ai Launch Round

Applied

Minimum

$30,000

Ideal

$50,000

How the money will be spent

Independent citation verification will use the funding for researcher time, corpus construction and annotation, independent human adjudication, frontier model API access, compute, publication, benchmark hosting and fiscal sponsor administration.

At the $30,000 minimum, the project will run as a 6 month pilot and fund approximately $15,000 for lead researcher time, $6,000 for research and annotation support, $2,500 for independent adjudication of disputed cases, $3,000 for model API and compute costs, $1,500 for the public scorecard and report and about $2,000 in fiscal sponsor fees. This version would test a smaller set of frontier AI systems, use a narrower held out corpus and publish one scorecard cycle.

At the $50,000 ideal level, the project would test more systems across more domains, increase the size of the independently labelled corpus, repeat evaluations to measure output variability and publish two scorecard cycles. The budget would cover about $22,000 for lead researcher time, $10,000 for research and annotation support, $4,000 for external adjudication, $5,000 for model API and compute costs, $3,000 for the benchmark site and publications, $2,500 for dataset maintenance and future scorecard preparation and about $3,500 in fiscal sponsor fees.

Discussion

No comments yet. Be the first to share your thoughts.