Execution-Guided Debugging & Defensive Feedback Loops for LLMs

Raising $8,500active

Building an execution-guided evaluation harness that equips AI models with interactive breakpoint debugging: improving code repair accuracy while preventing silent security vulnerabilities.

Project Details

Current automated code repair relies on "blinf" self-debugging from stack traces. This shallow feedback loop limits model accuracy on complex logic tasks and frequently triggers debugging decay or diminishing returns, where iterative repair attempts degrade both code correctness and security compliance. Building on pycapsule, this project introduces an interactive, execution-aware framework that enables LLM agents to perform breakpoint debugging. By creating an isolated, sandbox environment where models can dynamically set breakpoints, inspect variable states in memory and observe execution flow in real time, this system will drastically improve functional code repair accuracy while enforcing static analysis security testing (SAST) guardrails to eliminate latent CWE vulnerabilities.

Theory of Impact

The automation of software engineering by agentic AI introduces critical risk vectors: the accidental or intentional injection of catastrophic vulnerabilities into core infrastructure. This project directly mitigates this through three key interventions:

1. Scalable Oversight for Autonomous Agents

Current LLM code generation is a "black box." By forcing models to use interactive breakpoint debugging in a sandboxed runtime, this framework creates an observable, step-by-step execution trajectory.

2. Mitigating Deceptive Alignment & Malicious Exploits

A deceptively aligned AI could use opaque "blind" self-debugging as a smokescreen to intentionally inject zero-day (CWE) exploits. Coupling line-by-line execution tracking with SAST guardrails establishes strict, programmatic bounds that prevent the silent injection of malicious code during autonomous operation.

3. Securing Recursive Self-Improvement

Before AGI, AI systems will write and repair their own sub-modules. If these systems suffer from "debugging decay" (iteratively degrading security during repair), our infrastructure becomes highly vulnerable. By releasing a public evaluation benchmark, we provide the safety community with quantitative metrics to evaluate the stability and safety of recursive self-improvement loops.

People

Muntasir Adnan

Team Member

Grants Received– no grants recorded

Add grant

Funding Asks

grantmaking.ai Launch Round

Applied

Minimum

$8,500

Ideal

$10,500

How the money will be spent

$3,500 Compute & API Evaluation: Multi-pass benchmarking across state-of-the-art closed-source (Claude, GPT) and open-weights (Qwen, Llama) models across thousands of vulnerability trajectories.
$3,000 Local Infrastructure & Hardware: High-performance local GPU setup and power redundancy systems to maintain continuous execution-loop evaluations offshore.
$2,000 Benchmark Deployment: Public leaderboard hosting, open-source dataset maintenance, and open-access dissemination.
$2000 AI Startup: We plan to spin up an open-source AI Consultancy and deep research company in Canberra, Australia, and business registration, compliance, minimum marketing campaign will be held if we get the ideal requested grant.

Discussion

No comments yet. Be the first to share your thoughts.