Structural Code Monitors: Graph Matching for Trusted Monitoring

A non-LLM graph neural network compares code property graphs with trusted references to detect backdoors at low false-positive rates and provide node-level audit trails.

Project Details

We built a non-LLM trusted monitor that compares Code Property Graphs of untrusted code against trusted references using a Graph Matching Network with cross-graph attention. On 200 trajectories from Apps-Control-Arena, our 0.7M-parameter GNN achieves 0.946 AUC and 70% recall at the 2% FPR audit budget. Even when LLMs receive the same reference code (diff-based prompt), their false positive rates spike to 46%, making them unusable at low FPR thresholds. The GNN's structural pooling separates localized backdoor insertions from algorithmic differences — a distinction text-based monitors cannot make. Cross-graph attention weights additionally produce node-level audit trails identifying which code lines triggered detection.

Hackathon paper: Structural Code Monitors: Graph Matching Networks on Code Property Graphs for Topology-Aware Trusted Monitoring
Code: github.com/tanzeel291994/control-arena-gnn-monitor

Theory of Impact

AI control aims to maintain safety even when powerful models may intentionally attempt to subvert oversight. Current trusted-monitoring approaches predominantly rely on language models inspecting the outputs of other language models. This creates the possibility of correlated failure modes, particularly when an attacker can make malicious code appear textually innocuous.
Structural monitoring introduces a different signal. Rather than evaluating source code primarily as text, our monitor represents programs through Code Property Graphs combining syntax, control flow and data flow, and detects localized structural deviations between candidate and reference programs. gnn_monitor
Our initial results suggest that this signal can remain effective in the low-false-positive regime where human audit capacity is limited. The longer-term value is therefore as a complementary defense-in-depth component rather than a replacement for semantic monitoring.
If structural and LLM monitors fail differently, combining them may make it substantially harder for an adversarial agent to construct actions that evade all available oversight mechanisms simultaneously

People

Tanzeel Shaikh

Team Member

Track Record

AI Control Hackathon 2026 (Apart Research)

Grants Received– no grants recorded

Add grant

Funding Asks

Structural Code Monitors: Graph Matching Networks on Code Property Graphs for Topology-Aware Trusted Monitoring

Open

Minimum

$15,000

Ideal

$30,000

What this ask is

Our hackathon project demonstrated that program structure can provide a useful complementary signal for trusted monitoring in AI control. We built a Graph Matching Network that compares Code Property Graphs of untrusted code against trusted reference solutions using syntax, control-flow and data-flow information. On 200 Apps-Control-Arena trajectories, the monitor achieved 0.946 AUC and 70% Recall@2%FPR, substantially outperforming the LLM monitoring baselines we evaluated at the low false-positive operating point relevant to constrained human audit budgets. gnn_monitor
Funding would allow us to turn this prototype into a more robust AI-control research contribution.
First, we would investigate how structural monitoring behaves under weaker, imperfect or alternative reference mechanisms, addressing the main practical limitation raised during review.
Second, we would evaluate adaptive attacks where the adversary knows that structural monitoring is being used, including topology-preserving attacks such as literal or constant modifications that are known failure modes of the current system.
Third, we would study hybrid monitoring systems combining structural and LLM-based monitors. Our hypothesis is that the two approaches may have complementary failure modes because one reasons primarily over program structure while the other reasons over semantic and textual information.
Finally, we would broaden evaluation across additional attack scenarios and ControlArena settings, improve the open-source implementation, and prepare the results for workshop or conference publication.
The aim is not to replace LLM monitors, but to determine whether structural monitoring can provide a useful, independent defense-in-depth signal for increasingly capable coding agents.

How the money will be spent

For an ideal ask of $30,000:

Researcher stipends / research time — $17,000
Continued development, experiments, analysis and publication work.
LLM API and adversarial evaluation costs — $6,000
Attack generation, trusted-model outputs, monitoring baselines and adaptive red-team experiments.
Compute — $4,000
GNN training, model sweeps, CPG processing and evaluation runs.
Publication, workshop and dissemination costs — $3,000
Conference/workshop submission, travel where appropriate, and research dissemination.
For the minimum ask of $15,000, we would narrow the scope to the highest-priority research questions: reference robustness, adaptive attacks and structural+LLM monitor complementarity, with a smaller evaluation budget and researcher allocation.

Discussion

Private comments in this thread are only shown to approved funders and grant reviewers. Email hi@grantmaking.ai or to request access.

Private UserPrivate15d?