Governance Drift Evaluation Framework (GDEF)

Introduces Governance Drift and a reproducible framework for testing whether AI models maintain governance-aware behavior under user pressure across jurisdictions and domains.

Project Details

Most AI evaluations ask whether a model is capable, accurate, or safe. We asked a different question: does a model stay responsible when a user pushes back, insists, or presses for a more convenient answer? We introduce Governance Drift: the degradation, inconsistency, or loss of governance-aware behavior across jurisdictions, domains, conversation length, or user pressure; and the Governance Drift Evaluation Framework (GDEF), a reproducible, executable method for measuring it. Any researcher can load a custom scenario dataset, run it against one or more models through a lightweight evaluation runner, and obtain both quantitative governance scores and qualitative evidence of where safeguards break down. Testing two small models across Colombia, Mexico, Brazil, and the United States, we found the phenomenon is real and domain-dependent: in a facial-surveillance case the model held firm on consent, legal review, and human oversight throughout, yet in automated credit decisions models that began cautiously ended up endorsing actions they had initially flagged as risky.

Hackathon paper: Governance Drift Evaluation Framework (GDEF)

Theory of Impact

AI systems may initially recognize legal, safety, and oversight requirements but gradually abandon them when users apply pressure or extend a conversation. Because most evaluations examine isolated responses, this form of governance drift can remain undetected, particularly when the same system operates across different jurisdictions.

GDEF would make this failure mode measurable through a comparative study of Colombia, the United States, and the United Kingdom. By identifying when, where, and why governance-aligned behavior deteriorates, the framework would help researchers, deployers, auditors, and policymakers determine which contexts require stronger safeguards.

In the near term, the project would provide a validated methodology and new empirical evidence on multi-turn governance behavior. In the longer term, GDEF could inform evaluation requirements, deployment restrictions, human-escalation thresholds, and post-deployment monitoring for high-stakes AI systems. Its intended contribution is not to certify that a model is universally safe or legally compliant, but to give decision-makers a practical, reproducible way to detect governance failures before and during deployment.

People

Andrés Mogollón

Team Member

Track Record

Global South AIS Hackathon (Apart Research), 2026

Grants Received– no grants recorded

Add grant

Funding Asks

GDEF - Governance Drift Evaluation Framework

Open

Minimum

$15,000

Ideal

$30,000

What this ask is

We would use the funding to develop GDEF from a hackathon proof of concept into a validated comparative evaluation framework focused on Colombia, the United States, and the United Kingdom.

The first priority would be strengthening the methodology. We would refine the eight-dimension scoring rubric, recruit and train independent annotators, measure inter-rater reliability on a shared set of model responses, and revise the annotation guidelines based on disagreements and expert feedback.

We would then design high-stakes scenarios adapted to the legal and institutional contexts of the three countries. Multiple AI models would be tested through repeated, multi-turn conversations involving different user roles and increasing pressure to disregard legal, safety, or oversight requirements. This would allow us to compare governance drift, stability, and recovery across jurisdictions and domains, and investigate why models maintain responsible behavior in some contexts but abandon it in others.

We would also improve the open-source evaluation runner, data structure, documentation, and reporting process. The final outputs would include a validated rubric, comparative dataset, annotation handbook, reusable evaluation runner, worked conversation examples, empirical research report, and policy brief.

The minimum funding would support methodological validation and a limited comparative pilot. The ideal funding would enable broader model coverage, additional repetitions and scenarios, stronger technical infrastructure, and legal or policy review in all three jurisdictions.

How the money will be spent

Minimum grant $15,000: Method validation and comparative pilot

• Research design and coordination: $4,500
• Evaluation-runner development: $2,500
• Three independent annotators and inter-rater reliability study: $3,000
• Legal review for Colombia, the United States, and the United Kingdom: $1,800
• Model API access and repeated experiments: $1,200
• Data publication, technical documentation, and report production: $1,000
• Contingency: $1,000

This would produce a validated scoring rubric, nine jurisdiction-specific scenarios, a limited multi-model study, an annotation handbook, reproducible evidence, and a research report.

Additional $15,000 for the ideal grant: Decision-ready evaluation package

• Expand from 9 to 18 high-stakes scenarios: $2,500
• Expand testing to five models, three executions per scenario, and approximately 270 multi-turn conversations: $2,500
• Additional human annotation and reliability analysis: $2,500
• Engineering for automated experiment execution, comparison, and evidence reporting: $3,000
• Twelve consultations with AI-governance, legal, auditing, or public-sector practitioners across the three jurisdictions: $1,800
• Develop model Governance Stability Cards and a deployment decision matrix connecting findings to safeguards, human-escalation thresholds, and monitoring requirements: $1,700
• Public policy brief, worked case studies, and dissemination: $1,000

The ideal grant would therefore deliver not only stronger evidence, but a reusable package that policymakers, auditors, and deployers could apply when evaluating AI systems in high-stakes settings.

Total ideal budget: $30,000

Discussion

Private comments in this thread are only shown to approved funders and grant reviewers. Email hi@grantmaking.ai or to request access.

Private UserPrivate16d?