Vajra AI- Safety lab

Raising $5,000active

An automated, domain aware adversarial framework to stress test frontier LLMs via dynamic multi turn attacks and local security judging.

Project Details

Vajra AI is developing a smart, automated system designed to tackle a significant hurdle in AI safety, the outdated and labor-intensive process of red-teaming. Currently, evaluating an AI model for safety concerns is slow and heavily relies on static datasets, which cutting-edge commercial models can easily learn to exploit or evade.

Core Architecture & System Pipeline

Our system operates via a smart multi stage automated loop:

Stage 1: Context Analysis (Domain Fingerprinting)- When a user provides a target LLM's API endpoint and key, Vajra kicks off a localized meta-prompt to fingerprint and profile the target's operational domain. This helps us figure out if it’s acting as a Programming Assistant, a general Chatbot, or a specialized enterprise tool. Based on this classification, it dynamically activates a tailored adversarial library.
Stage 2: 3-Tier Payload Execution- The attack infrastructure is carefully organized into three distinct tiers, each with increasing complexity.

Level 1 (Surveillance)- Direct inquiries aimed at uncovering system prompt leaks, boundary constraints, and ways to bypass structural alignments.

Level 2 (Infiltration)- More intricate persona hijacking (think DAN, SINDRA), stepwise token obfuscation, and base64 encoded structural payloads that slip past traditional surface level token filters.

Level 3 (Total Annihilation)- Multi turn, recursive compositional attacks that build context through deep conversational threads, ultimately leading to alignment failures through logical framing.
Stage 3: Local Security Judging- Instead of sending model evaluations to commercial cloud platforms, which could risk leaking sensitive or unsafe data, Vajra operates a fully local validation loop using Llama-3 (8B Instruct via Ollama) on a standard 16GB RAM laptop. The local judge delivers a clear binary verdict (PASS/FAIL) along with an automated text justification to compute a standardized metric known as the Vulnerability Index.

Current Benchmarks & Proof of Concept

We’ve already developed a working local prototype and successfully benchmarked it against Gemini 2.5 Flash. Across a suite of 9 structural attack vectors, Vajra revealed an overall 67% Vulnerability Index, clearly showing a 75% failure rate against Level 2 structural infiltration and a staggering 100% failure rate against Level 3 recursive alignment attacks

Who is Involved

I'm currently an undergraduate student in physics, diving deep into core systems design, simulation pipeline architecture, and implementation. I love collaborating with fellow open-source developers and tapping into the wisdom of my math faculty mentors to enhance the underlying adversarial logic structures and statistical evaluation metrics.

Concrete Outputs

Open-Source Command Line Interface (CLI) & UI Dashboard- I've developed a robust Python-based framework that automates cross lingual and domain specific prompt mutation testing, complete with real-time logging of pass/fail metrics.
1. Fine-Tuned Adversarial Generator Weights- I'm working on training pipelines to upgrade our existing fixed 3-tier library to a more dynamic, fine tuned local adversarial model. This model will analyze target failure points and generate custom payloads in real-time.
2. Reproducible Security Benchmarking Standards- I've created an expandable pre-deployment compliance checklist that allows developers to audit LLM applications before they go live, functioning just like standard unit tests.

Theory of Impact

The most significant existential risk (x-risk) posed by cutting-edge LLMs isn't just about straightforward jailbreaks; it actually stems from these models failing when they're put into critical, real-world automated systems. Currently, commercial labs assess safety using static benchmarks, which creates a risky bottleneck. This approach trains models to resist a limited set of known phrases, leading to a misleading sense of safe alignment while leaving them wide open to adaptive, multiturn adversarial strategies. If a malicious actor manages to automate an adaptive attack loop before safety researchers can develop countermeasures, it could lead to widespread malfunctions in essential systems.

Vajra AI is changing the game by moving from a static defense model to one focused on active, automated safety discovery. We’ve created a framework that continuously profiles models and dynamically alters payloads based on contextual weaknesses, mimicking the thought process of a sophisticated human attacker, but on a much larger scale. Our local evaluation pipeline allows developers to consistently stress-test their models safely before they go live. By uncovering deep, hidden architectural vulnerabilities and addressing them proactively, Vajra AI offers an open-source, automated safety checklist that significantly lowers the risks of catastrophic deployments and keeps up with the rapid evolution of autonomous systems.

People

ANKIT KUMAR JHA

Team Member

Grants Received– no grants recorded

Add grant

Funding Asks

grantmaking.ai Launch Round

Applied

Minimum

$5,000

Ideal

$22,000

What this ask is

Core Architecture & System Pipeline

Our system operates via a smart multi stage automated loop:

Stage 1: Context Analysis (Domain Fingerprinting)- When a user provides a target LLM's API endpoint and key, Vajra kicks off a localized meta-prompt to fingerprint and profile the target's operational domain. This helps us figure out if it’s acting as a Programming Assistant, a general Chatbot, or a specialized enterprise tool. Based on this classification, it dynamically activates a tailored adversarial library.
Stage 2: 3-Tier Payload Execution- The attack infrastructure is carefully organized into three distinct tiers, each with increasing complexity.

Level 1 (Surveillance)- Direct inquiries aimed at uncovering system prompt leaks, boundary constraints, and ways to bypass structural alignments.

Level 2 (Infiltration)- More intricate persona hijacking (think DAN, SINDRA), stepwise token obfuscation, and base64 encoded structural payloads that slip past traditional surface level token filters.

Level 3 (Total Annihilation)- Multi turn, recursive compositional attacks that build context through deep conversational threads, ultimately leading to alignment failures through logical framing.
Stage 3: Local Security Judging- Instead of sending model evaluations to commercial cloud platforms, which could risk leaking sensitive or unsafe data, Vajra operates a fully local validation loop using Llama-3 (8B Instruct via Ollama) on a standard 16GB RAM laptop. The local judge delivers a clear binary verdict (PASS/FAIL) along with an automated text justification to compute a standardized metric known as the Vulnerability Index.

Current Benchmarks & Proof of Concept

Who is Involved

Concrete Outputs

Open-Source Command Line Interface (CLI) & UI Dashboard- I've developed a robust Python-based framework that automates cross lingual and domain specific prompt mutation testing, complete with real-time logging of pass/fail metrics.

2. Fine-Tuned Adversarial Generator Weights- I'm working on training pipelines to upgrade our existing fixed 3-tier library to a more dynamic, fine tuned local adversarial model. This model will analyze target failure points and generate custom payloads in real-time.

3. Reproducible Security Benchmarking Standards- I've created an expandable pre-deployment compliance checklist that allows developers to audit LLM applications before they go live, functioning just like standard unit tests.

How the money will be spent

We're seeking a $22,000 grant that will be carefully distributed across three key operational needs to evolve Vajra AI from a static prototype into a dynamic, self-learning system:

Compute & Training Hardware ($11,000 — 50%)

To upgrade our current fixed-prompt library with a finely-tuned local adversarial generator, we require significant computing power for model training and high-token inference cycles.

Cloud GPU Rental: We’re allocating $8,500 for high-performance cloud computing instances (renting NVIDIA H100s or A100s through Run Pod/Lambda Labs) specifically for training, looping, and optimizing checkpoints of our custom adversarial weights.

Local Storage & Testing Nodes: We’ll invest $2,500 in high-speed local NVMe storage arrays and optimization setups to efficiently pipeline large dataset streams into local Llama-3 (8B & 70B) configurations simultaneously.
\
Target API Access & Multi-Model Evaluation ($7,700 — 35%)

To conduct thorough reinforcement safety mapping, we need to run our generated prompt mutations at scale against a wide array of proprietary closed-source systems to accurately assess the Vulnerability Index.

Commercial API Credits: We’re setting aside $7,700 to cover the extensive input/output token volume across major proprietary API endpoints, including OpenAI (GPT-4o series), Anthropic (Claude 3.5 series), and Google DeepMind (Gemini 1.5 & 2.5 series) across various domain specific fine tuning contexts.
\
Dataset Curation, Framework Infrastructure & Storage ($3,300 — 15%)

Dataset Engineering: We’ll allocate $2,300 for structuring, formatting, and pre processing a variety of adversarial jailbreak datasets and cross-lingual translation tokens to fine-tune the model.

Hosting & Framework Pipeline Maintenance: We’re budgeting $1,000 for hosting our open-source framework repository, maintaining automated data logging storage infrastructure, and ensuring remote benchmarking upkeep.

Discussion

No comments yet. Be the first to share your thoughts.