Independent citation verification benchmark for AI systems

The independent citation verification will build a held out benchmark for testing how frontier AI systems use citations. The benchmark will include supported, contradicted, fabricated, misattributed, outdated and ambiguous claim-source pairs. I will validate the evaluation pipeline against human labelled cases, run repeated tests across major AI systems and measure citation existence, attribution, claim support, coverage and fabrication rates.

The project will be led by Gideon Abako, founder of Ancestor Lab and the researcher who designed and deployed the existing citation verification pipeline. The grant funds the benchmark work. Paid annotators or adjudicators will review disputed cases and provide blinded human checks so that the same person is not creating the corpus, scoring every case and resolving every disagreement.

The outputs will be a documented held out corpus, a public evaluation methodology, a failure taxonomy and a public scorecard comparing major frontier AI systems on citation integrity. I will also publish a technical report, a selected public subset of the benchmark for reproducibility and the code needed to rerun the evaluation. A rotating private test set will be retained to limit contamination and gaming while each scorecard will record the tested system, model version, date, mode, prompt protocol, repeated run results and human audit findings.