GenomeGuard: Securing the Genomic AI Supply Chain

Raising $30,000active

GenomeGuard is an open-source defense and threat-discovery framework for detecting data poisoning, compromised annotations, and supply-chain attacks before they propagate into genomic foundation models.

Endorsed by

Endorsed by

Project Details

Genomic foundation models are increasingly trained on massive collections of DNA sequences assembled from open, continuously updated public databases. In our previous work, Poisoning the Genome: Targeted Backdoor Attacks on DNA Foundation Models, we conducted the first systematic investigation of training-data poisoning in genomic language models and demonstrated that these systems can acquire targeted, trigger-dependent failure modes from relatively small amounts of manipulated data. We showed that poisoning can affect both pre-training and fine-tuning, can target biologically meaningful contexts and clinically relevant prediction tasks, and can remain largely invisible because model behavior on unrelated sequences is remains unaffected.

Our proposed project moves from demonstrating these vulnerabilities, to building practical defenses against them. We will develop GenomeGuard, a unified, open-source framework for auditing public genomic databases and model-training pipelines for evidence of data poisoning, annotation manipulation, and other supply-chain compromises. The framework will combine database provenance, integrity checks across database versions, sequence-level anomaly detection, embedding-space analysis, supervised and weakly supervised poisoning classifiers and identification of suspicious trigger–payload associations. Embedding-based analyses will be used to detect anomalous clusters, outlier records, unexpected nearest-neighbor relationships, and sequences whose learned representations are inconsistent with their stated taxonomy, function, or annotation. Dedicated classifiers will be trained on controlled poisoning examples, synthetic perturbations, and known database inconsistencies to assign probabilistic risk scores to individual records and groups of related sequences. The framework will also implement trigger-specific checks for rare or unusually repeated motifs, validated across different entries of the same genome in multiple databases. Lastly we will be checking the genomes for suspicious sequences or payloads after functional elements that may contain know harmful or toxic pathogens, the most critical danger regarding these types of attacks. GenomeGuard will aggregate these signals into an interpretable risk profile, allowing suspicious records to be flagged, quarantined, or prioritized for expert review before they enter a public domain.

The concrete outputs will include an open-source GenomeGuard auditing pipeline described above, a benchmark of controlled genomic poisoning and supply-chain attacks, evaluations of defensive methods across multiple open genomic models and datasets and a public research paper describing any newly identified risks and proposed defenses.

The project will be led by Charalampos Koilakos, a PhD student in Computer Science at the University of Texas at Austin, alongside Prof. Ilias Georgakopoulos-Soares.

Theory of Impact

As genomic foundation models become more capable and increasingly influence biological prediction, interpretation, and sequence design, compromise of public data or annotations could create a systemic or, in the worst case, catastrophic failure mode. GenomeGuard will reduce this risk by moving security controls upstream through data-provenance tracing, detection of anomalous sequences and annotations, screening for suspicious trigger–payload relationships, and adversarial testing of trained models for conditional behaviors before they are used or deployed. By releasing the framework, threat benchmarks, and evaluation methods openly, the project will enable multiple laboratories to audit shared genomic resources and establish data integrity and adversarial evaluation as standard practices for biological AI. The project therefore addresses a large-scale AI risk in which adversaries exploit shared data and model supply chains to introduce hidden vulnerabilities or malicious behaviors that propagate across the research ecosystem and could ultimately facilitate the development of harmful pathogens, toxins, or other dangerous biological outputs.

People

Charalampos Koilakos

Team Member

Grants Received– no grants recorded

Add grant

Funding Asks

grantmaking.ai Launch Round

Applied

Minimum

$30,000

Ideal

$50,000

How the money will be spent

We are requesting a minimum of $30,000, with an ideal funding level of $50,000. For the minimum budget we would allocate $10,000 for researcher time, $3,000 for indirect costs (calculated at 10% and subject to institutional policy), $14,000 for GPU and cluster compute, $1,500 for storage and large-scale genomic data processing and $1,500 for model-access, inference, and hosting costs. This would fund a thorough initial experimental program across representative genomic models and datasets, including controlled poisoning attacks, embedding analyses, poisoning classifiers, trigger-specific checks, and model-level adversarial evaluations. For the Ideal budget we would allocate $10,000 for researcher time, $5,000 for indirect costs, $28,000 for GPU and cluster compute, $3,000 for storage and data processing and $4,000 for model-access, inference, and hosting costs. This would allow us to test substantially more architectures, model scales, datasets, downstream tasks, poisoning strategies, and supply-chain threats, also supporting the development of a public demonstration and deployment of the framework across public genomic databases.

Discussion

Private comments in this thread are only shown to approved funders and grant reviewers. Email hi@grantmaking.ai or to request access.

Grant ReviewerPrivate5d?