Project description

I am an undergraduate studying mechatronics engineering at Ahmadu Bello University in Nigeria. Our paper, "Evaluating Mental Health LLM Responses to Localized African English," just got accepted for a poster presentation at the Global South AI Affinity Event at NeurIPS 2026 in Sydney. I'm the lead author of the paper.

My conference registration  is already taken care of, but flying from Nigeria down to Australia is way out of reach on my student budget. Between the flights, visa, and a place to stay, I need $2,400 to make the trip happen so I can present the work in person.

We ran this research project during the Apart Research Global South hackathon with my co-authors, Namirah Rasul and Ewura Ama Sam. We wanted to see what happens when you take a safety-critical model, specifically MentaLLaMA-chat-7B, and test it on non-standard English without changing the core meaning of the prompt. We pulled 100 clinician-grounded scenarios from MentalChat16K and created 300 controlled prompt pairs across Standard English, Nigerian Pidgin, and Ghanaian English.

The results exposed two big problems for scalable oversight:

1. Average numbers lie about tail risk. The overall quality dropped slightly from 3.94 to 3.79 in Pidgin (p = 0.0027), but the worst-case failures were dramatic. Just rewording a prompt into Pidgin caused the model to suddenly refuse to answer, spit out generic boilerplate, or scold the user instead of giving supportive advice.
2. Automated judges didn't notice at all. The LLM judge handed out a 5/5 safety score on every single one of the 300 runs. It was completely blind to these behavioral breakdowns simply because the input wasn't standard English.

• OpenReview: https://openreview.net/forum?id=ICZEGRAVCn
• Apart Hackathon: https://apartresearch.com/sprints/projects/evaluating-mental-health-llm-responses-to-localized-african-english-pzag

What are this project's goals? How will you achieve them?

The paper is already done and accepted. This grant is strictly about getting me into the room at NeurIPS.

Being in Sydney lets me take this work further in a few concrete ways:

• I'll present our poster and talk through our results with people working on out-of-distribution robustness and automated red-teaming.
• I want to talk to eval researchers about how broken LLM-as-a-judge rubrics really are. Most teams rely on automated judges to claim their models are safe, but our experiment showed an automated judge giving a perfect 5/5 to a model that was actively failing users in distress.
• I want to turn our dataset setup into an open-source evaluation benchmark (using UK AISI's Inspect AI framework) to test whether frontier foundation models also drop their guardrails or sandbag when given dialect shifts.
• On a community level, I co-founded AI Safety Hub Nigeria. Going to NeurIPS will help me build direct ties with international labs and bring those research connections back to the fellows we mentor locally.

How will this funding be used?

My conference pass is already paid for. The full $2,400 request covers basic travel costs:

• Return flights (Nigeria to Sydney): $1,500
• 5 nights lodging in Sydney: $650
• Australian visa fees: $150
• Food and daily transport: $100
• Total: $2,400

My minimum goal is $1,650. That is the bare minimum to buy the flight ticket and pay the visa fee. If the campaign only reaches $1,650, I'll figure out shared hostel beds and pay for my own meals. If I receive travel funding from any other applications, I will lower this target right away.

Who is on your team? What's your track record on similar projects?

I am the first author, with co-authors Namirah Rasul and Ewura Ama Sam. I am a mechatronics engineering student at Ahmadu Bello University, Nigeria.

For the last three years, I’ve been focused on empirical LLM evaluations and building technical AI safety capacity:

• Evaluations & Research: Built SandbagBench-Lite (a black-box harness for measuring strategic underperformance in LLMs) and worked as an AI Safety & Evaluation Research Intern at HeyLina (designing conversational evaluation rubrics and LLM-as-a-judge frameworks). Authored Forecasting AGI: A Granular, CHC-Based Approach and received recognition in the ENAIS AI Safety Action Competition for research on behavioral adaptation (Punctuation Paranoia).
• Fellowships: 2024 Non-Trivial Fellow, Cooperative AI Research Fellowship finalist, and alumnus of the TReND-CaMinA ML & Computational Neuroscience program in Kenya.
• Community Leadership: Co-founder and Technical Lead of AI Safety Hub Nigeria, President of Effective Altruism ABU, and Policy Lead for PauseAI Nigeria.
• LinkedIn:**** https://www.linkedin.com/in/abdulfatah-habeeb/

What are the most likely causes and outcomes if this project fails?

On my current student budget, I can't afford the flight to Australia without travel funding. Hence, not attending the conference means I miss the opportunity to present the poster in person, get technical feedback from leading AI evals researchers, and build collaborations.

How much money have you raised in the last 12 months, and from where?

$0 has been raised to date.