TraceTruth: Outcome-Grounded Evals for Agentic Honesty

The Problem

Agents will be granted permissions to access tools and data to increasingly perform long, stateful sequences of actions.  Human monitors frequently see a final report or summary as they can’t practically examine every tool call and state transition.  An agent can appear successful while possibly omitting a failed step or concealing a policy violation, ignoring a directive, overstating completion or having met objectives, or perhaps failing to disclose an authorization or data breach.  The resulting action-report gap weakens human ability to truly monitor behavior over long periods.  The human may easily assume that everything is operating within guidelines.  Observable failure can arise from several sources such as conflicting guidance, reward pressure, guardrails, or poor state tracking not to mention deception. 

What We Will Build
We’ll build an instrumented sand-box with machine-checkable state.  We’ll investigate 40-50 scenarios with minimum funding.  We’ll consider three common workflows such as software and infrastructure operations, data analysis and reporting, and controlled decision workflows.  Scenarios will include situations where false findings are very expensive, false claims of completion, surfacing of failures, surfacing of uncertainty, ignoring approval requirements, unsubstantiated claims or misleading summaries of the final state.

Each scenario will produce an agent’s tool call log trace, state transitions, final state, and the agent’s final report.  Any constraint violations will be noted.

We’ll have three oversight approaches: the agent’s report, an independent monitoring agent report, and deterministic verification from the final state.

Experimental Design
The baseline runs across four model families at minimum scope.  We’ll run repeated trials with matched controls that vary one condition at a time.  We’ll isolate task success, safety compliance, reporting fidelity and escalation behavior.  For measurement, primary metrics will include false-success rate, undisclosed-violation rate, action-report consistency, appropriate deferral or escalation rate, and recall at a fixed false-positive rate.  As is our wont, we’ll explicitly report negative results, evaluator failures and limitations.

Who is Involved
I’ll be the PI and primary research engineer.  The minimum proposed budget includes an independent technical safety or eval reviewer.  The ideal budget adds part-time research-engineering and data-QA support, plus an independent reproducibility review.  We will identify those folks once funding is in place.

Concrete Outputs
We will release a permissively licensed open-source evaluation harness in a public GitHub repo.  We will include a versioned scenario dataset along with deterministic graders and trace-analysis tooling.  We’ll surface baseline results across participating models.  We’ll produce a benchmark card documenting intended use, limitations, and contamination risks.  We’ll generate a technical report summarizing our findings.  Under ideal funding, we could create and publish a public result explorer and independent challenge set.