When Missing Evidence Looks Like Innocence: Evidence Ablation

Tests how missing evidence affects AI-agent incident investigations and whether tamper-resistant action receipts prevent false exoneration when access logs are incomplete.

Endorsed by

Chase

Jason

Tests how missing evidence affects AI-agent incident investigations and whether tamper-resistant action receipts prevent false exoneration when access logs are incomplete.

Endorsed by

Chase

Jason

Project Details

This study tested how missing evidence affects investigations of AI-agent incidents. In 120 episodes, agents could access a prohibited answer file. Removing the passage revealing this access from selected activity logs led automated investigators to falsely exonerate 11 of 34 affected agents. Restoring the passage or adding a compact action receipt eliminated all 11 errors. Removing a similar amount of unrelated text caused no errors. The findings support retaining tamper-resistant action records and testing whether investigators use them correctly.

Hackathon paper: When Missing Evidence Looks Like Innocence: Evidence Ablation in Agent-Incident Reconstruction
Code: github.com/l-terminassian/agent-incident-reconstruction
Slides: Slides
Recording: Presentation recording

Theory of Impact

AI incident investigations depend on records that may be incomplete or misleading. Treating missing evidence as proof that nothing happened could close an investigation prematurely. Overinterpreting indirect clues could also lead to unsupported accusations.

This project would identify which retained records support reliable reconstruction and which investigation procedures help models recognise uncertainty. Agent developers could use the results to improve logging. Evaluators and incident-response teams could use the benchmark to test whether automated investigators overlook evidence, make unsupported claims, or request useful additional records.

I would release reproducible tests and seek feedback from potential users, aiming to support an initial trial on an existing investigation workflow. The project would not establish that automated investigation is generally reliable. It would identify specific capabilities and limits.

A finding that simple checks outperform LLMs would also be useful. The intended impact is better-supported conclusions and clearer escalation decisions, rather than greater automation for its own sake.

People

Lucile Ter-Minassian

Team Member

Track Record

AI Incident Response Sprint (Apart Research), 2026

Grants Received– no grants recorded

Add grant

Funding Asks

When Missing Evidence Looks Like Innocence: Evidence Ablation in Agent-Incident Reconstruction

Open

Minimum

$2,500

Ideal

$7,000

What this ask is

I would study when AI investigators can reconstruct an agent’s actions from incomplete records, and when they should request more evidence or leave a question unresolved.

The minimum project would reuse the existing experiment and add new cases, targeting approximately 200 independent episodes. I would separately vary direct evidence, indirect clues, and access receipts. Controls would include no-access cases, failed reads, and cases where another agent supplied the same information. This would distinguish overlooked evidence from evidence that cannot support a conclusion.

I would test investigators from Anthropic, OpenAI, and Google. I would compare their usual investigation procedure with one that checks structured records before interpreting the narrative. A deterministic checker would receive the same evidence. Outcomes would include unsupported “yes” and “no” answers, correct conclusions, unnecessary abstention, and confidence calibration. The study would assess factual reconstruction, not innocence or intent.

With the ideal budget, I would expand to approximately 400 episodes across two settings: file access and tool-mediated data transfer to a simulated external service. I would test four API-based investigators and one open-weight investigator. Targeted experiments would remove individual receipt fields and let investigators request missing records within a fixed budget.

I would use a separate pilot to finalise sample sizes, preregister the main comparisons, and keep development and evaluation cases separate. Independent readers would review a sample, with their expertise and conflicts documented. Outputs would include a paper, reproducible benchmark, and practical checks for logging and investigation systems.

How the money will be spent

shown as minmum/ideal:

Claude Code tokens for implementation, debugging, and analysis: $400 / $600.
Model API tokens for evaluations, pilots, and targeted follow-up experiments: $800 / $2,700.
Runpod GPUs for open-weight inference and new agent episodes: $300 / $800
Storage and CPU hosting: $100 / $200.
Independent annotation and human-reader participation: $480 / $1,200.
Reserve for usage overruns, reruns, price changes, and applicable unrecoverable taxes: $420 / $1,500.

Total: $2,500 minimum / $7,000 ideal.

Discussion

Private comments in this thread are only shown to approved funders and grant reviewers. Email hi@grantmaking.ai or to request access.

Private UserPrivate7d?

AAnonymous judgeJudge · AI Incident Response Sprint7d

The ground truth is tied to the harness call record and file hashes. The restore condition puts back the passage that was removed. The matched‐volume placebo deals with the obvious alternative explanation. Given that setup the 11‐to‐0 reversal is hard to ignore with a small sample. What stood out more to me though was how the investigator failed. The investigator did not simply say it was unsure. The investigator answered "no" with a confidence of 82.5. That matters because an abstention would usually trigger investigation. A confident exoneration can close the case. I also thought the treatment of confidence conversion the post‐hoc outcome choice and co‐accessor contamination was unusually careful. I would also run the cross‐family investigator already supported by the pipeline since the current design still mixes capability level with model identity. The Haiku result deserves emphasis as well: seven failures despite well‐formed receipts suggests retention by itself is not enough making the investigator canary a central control rather, than a secondary check. Finally I would describe the redaction result more cautiously. If 12 of 34 logs still contained leaked values the effect should be treated as an unknown bound than simply described as conservative.

Chase HasbrouckJudge · AI Incident Response SprintEndorsement7d

Strong: Excellent experiment design that tests how well an "investigator agent" can cope with a tampered log from a previous agent's run, with differing checks for just a tampered log vs. a log with a "receipt." Use of human readers as a comparison. Good placebo control that separates out shorter-log effects. Excellent thinking about not just "yes vs. no", but "yes vs. no vs. uncertain" and levels of confidence in each. Video presentation and slide deck.

Improve: There was a significant second finding that is underexplored (runs where the tampered log removed explicit evidence but retained implicit evidence). The paper considers this in part, but I'd love to see an experiment that focused specifically on this case.
Testing a single model family weakens the result substantially, but defensible for a weekend project. An ablation study on the receipt could be done; perhaps it's a single field that's necessary. The human readers need more identification in the paper to understand their biases/conflicts.

Jason TangJudge · AI Incident Response SprintEndorsement7d

I think this paper asks a useful question about whether an AI investigator draws the wrong conclusion when evidence is missing. It removes information about file access from incident records and measures how the model’s account changes. Restoring that information corrected the errors, while comparisons with unrelated changes help make the result more convincing.

I would keep the conclusion narrower than the title suggests. Misjudging whether an agent accessed a file is not the same as finding it innocent, particularly when access was not explicitly prohibited. The study supports concern about unreliable reconstruction, rather than conclusions about wrongdoing. Comparing the model with a simple system that checks access records and flags missing evidence would help establish what the model adds.