Mitigating Hallucination in Multimodal Agentic Systems

Raising $6,000active

Implementing Hybrid Reward Architectures (HRA) to build robust internal factual grounding and quantitative variance metrics for multimodal agents, moving beyond fragile single-seed evaluations.

Project Details

This project aims to systematically address the structural fragilities of current AI evaluation metrics by developing an internal Hybrid Reward Architecture for multimodal agentic systems. Standard single-seed evaluations often mask model instability, reward density collapse, and deceptive behaviors such as reward hacking or specification gaming.
I will engineer and test an HRA framework using LLaVA-1.5-7B to investigate whether internal reward structure can improve factual grounding more reliably than external patch-based methods such as standard RAG wrappers. In parallel, I will design evaluation pipelines that measure variance, instability, and reward hacking across complex agentic workflows.
The project will be led by me, Teganmosibineba Oluwatofarati Jegede. I bring hands-on experience in building autonomous agents and evaluation pipelines using PyTorch, LangChain, and Python, alongside product management experience that has strengthened my ability to work with efficiency metrics and structured project delivery.
The deliverables will include an open-source evaluation suite for hallucinatory and deceptive behavior in agentic systems, a documented HRA implementation for LLaVA-1.5-7B, and a research paper suitable for submission to a major AI safety conference.

Theory of Impact

Existential risk from highly capable autonomous AI systems increases when we cannot reliably measure whether they are behaving in aligned ways during training and evaluation. When systems are tested with sparse, single-seed evaluations and protected mainly by brittle external guardrails, dangerous behaviors such as reward hacking, deceptive alignment, and systematic hallucination can remain hidden until deployment.
This project reduces that risk in two ways. First, by investigating Hybrid Reward Architectures, it aims to improve factual grounding and behavioral reliability from within the model’s training signal rather than relying only on external patching. Second, by publishing quantitative variance metrics and multi-seed evaluation methods, it gives researchers and regulators better tools for detecting instability and deceptive optimization before these systems are scaled into high-stakes settings.

People

Teganmosibineba Jegede

Team Member

Grants Received– no grants recorded

Add grant

Funding Asks

grantmaking.ai Launch Round

Applied

Minimum

$6,000

Ideal

$15,000

What this ask is

How the money will be spent

Minimum funding — $5,000 (essential compute)

GPU cloud compute: [1,600 GPU-hours × $2.00/hr] = $3,200 — covers multi-seed variance runs (8 seeds × 10 configurations for HRA weight ablations × 20 hours per run) and HRA experiments on LLaVA-1.5-7B using 1x A100 80GB instances on RunPod.
Frontier-model API (baselines): $1,300 — API calls for OpenAI (gpt-4o) and Anthropic (claude-3.5-sonnet) to establish baseline comparisons for agentic deception, capped at this ceiling.
Contingency (~10%): $500
Deliverable: completed multi-seed variance evaluation, written up for release.

Ideal funding — $15,000 (adds scale, focus, and dissemination)

Expanded compute: $3,400 — scales the evaluation from 8 to 16 seeds and expands to additional benchmarks and larger context windows (~1,700 additional GPU-hours × $2.00/hr on A100 80GB), roughly doubling the multi-seed compute of the minimum tier.
Living stipend: $1,400/month × 4 months = $5,600 — enables full-time focus on the research, replacing alternative income sources to dedicate 40 hours/week to project execution.
Conference travel + registration: $3,500 — covers registration, accommodation, and round-trip flights from Abuja (ABV) to present findings at AI safety workshops and venues such as NeurIPS or ICML.
Frontier-model API (baselines): $1,300 — maintained from the minimum tier.
Contingency: $1,200
Deliverable: extended evaluation suite tested across multiple frontier models, plus presentation of findings to the AI safety community.

Discussion

No comments yet. Be the first to share your thoughts.