Sentinel: Evaluating Human Oversight of AI-Generated Code

Evaluating when tests and engineer review fail to catch defects in AI-generated software patches.

Project Details

Noah Labs AI Inc. proposes a six-month evaluation of failures and human oversight in AI-assisted software maintenance, using Sentinel, its locally deployable AI software engineering environment.

The project would use permissioned, non-production codebases and bounded maintenance tasks. We would assess generated patches against predefined tests and engineer review, measuring incorrect changes, missed defects, review effort and rework. We would document cases where tests pass despite an incorrect change and where reviewers fail to identify a defect.

The team includes co-founders Murat Isik, CEO, and Berk Yilmaz, CTO, with Oguzhan Isik coordinating outreach and project documentation. Sentinel currently has a working prototype/demo. The evaluation protocol, independent reviewer and dataset have not yet been finalized.

Planned outputs include an evaluation protocol, a technical report and permissioned methods or anonymized findings that other teams can use. No independently validated safety or productivity improvements have yet been established.

Theory of Impact

As AI coding systems become more capable, engineers may rely on generated changes that pass existing tests but contain important defects. Human review can also miss these failures.

Our proposed causal chain is to identify oversight failures in bounded software-maintenance tasks, evaluate which checks detect them, and share reproducible methods and permissioned findings. Other teams could use these results to strengthen evaluations and review practices before granting coding systems greater autonomy.

The project's relevance to catastrophic or existential risk depends on whether the observed failure modes and oversight methods generalize to more capable systems and consequential software settings. That connection remains uncertain and has not been demonstrated. We would report limitations and negative results alongside any useful findings.

Local deployment provides data control; it does not by itself establish AI safety.

People

Murat Isik

Team Member

Grants Received– no grants recorded

Add grant

Funding Asks

Funding ask

Open

Minimum

$25,000

Ideal

$50,000

How the money will be spent

The proposed six-month budget totals $50,000:

• Evaluation protocol and technical leadership: $18,000 (300 hours at an estimated fully loaded rate of $60/hour).
• Evaluation engineering and analysis: $15,000 (250 hours at $60/hour).
• Independent technical review: $6,000 (100 hours at $60/hour).
• Compute and model inference: $4,000.
• Evaluation infrastructure, storage and reporting: $2,000.
• Contingency: $5,000.

These are planning estimates, not confirmed salaries, vendor quotations or contracted reviewer fees.

At the $25,000 minimum, we would reduce the number of tasks and codebases, halve the planned labor and independent-review hours, and reduce compute and infrastructure allowances. We would retain predefined tests, engineer review and a written report, while narrowing the conclusions accordingly.

Funding would support the defined evaluation work. We would track project costs separately and would not charge the same costs to another grant. Any overlapping awards would be disclosed and the scope or funding request adjusted.

Discussion

No comments yet. Be the first to share your thoughts.