Proposed Action Auditor (PAA)

Raising $25,000active

A local, low-latency safety runtime that inspects token-level logprobs to block unaligned agent tool-calling trajectories, utilizing a Gemma 4 model fine-tuned via LoRA on an empirically derived Task Action Language (TAL).

Project Details

Proposed Action Auditor

I will be the sole developer of the Proposed Action Auditor. The end result will be an open-source body of code and fine-tuned LLM weights for the Gemma 4 - e4b model, made available under the Apache 2.0 software license.

Core Architecture and Purpose

The Proposed Action Auditor (PAA) will be an open-source, client-side, local safety sentinel that operates in real-time to protect against rogue agent task-calling behavior. It will reside in the agent task execution processing pipeline right before an action proposed by an agent is executed. If a proposed action is judged to be unusual or unsafe, it will be flagged for review by the human operator. Otherwise, it will be allowed through. The engine created for the PAA will treat safety as a linguistic probability forecasting problem.

The Data Engineering Pipeline

The initial phase of the PAA will be the creation of a DSL that will be an empirically determined "language of tasks". The Nvidia open-source Nemotron tool-calling trajectory set, a massive set of 335,122 multi-turn agent tool-calling logs, will be used to empirically determine the initial Task Action Verb (TAV) and Task Result Label (TRL) semantic symbol set. Each turn will be processed by an LLM to create the "naive" TAV and TRL labels, including a summary of the current agent task using the Belief-Desire-Intention (BDI) paradigm.

The Task Label Conformance Module

Because LLMs are non-deterministic, this will result in a diffuse, semi-focused set of semantically related labels that need to be conformed. Those naive labels will be converted to vector embeddings and then clustered by the DBSCAN algorithm to create a semantically relevant, canonized set of conformed grammar symbols known as the Task Action Language (TAL), the "language of agent tasks".

Once completed, a code module will be created that uses cosine similarity techniques to map a naive label to the correct canonical label in the TAL. This module is the task label conformance module.

Fine-Tuning and Edge Deployment Mechanics

A new tool-calling trajectory database will be made from the original one. However, the derived database will contain only the conformed TAL labels and the BDI context notes for each agent turn. This new data will be used to facilitate LoRA Training on the Gemma 4 - e4b open-weights model. This model was chosen because it runs comfortably on the entry-level consumer Nvidia GPU family (such as the RTX 3050), thus making the PAA available to almost any agent user worldwide. The end result being a real-time sentinel that has a soft internal context-dependent model of what a normal or "sane" proposed agent action looks like, and what proposed actions require human inspection.

The Logprob Gating Runtime

As the agent session unfolds, a fast LLM call will be made to get the current naive task label in real-time using the same techniques used in the TAL creation phase. The naive label will be run through the task label conformance module to get the correct TAV label for the current proposed agent action, and the correct TRL label for the result of the last task action execution. These will be accumulated to create the current overall TAL label sequence that represents the current state and context of the agent tool-calling session in progress.

This is where the PAA leverages the inherent semantic power of LLMs, radically augmented with a custom Gemma open-weight set that includes the empirically determined tool-calling specific task language via the TAL.

Using a custom Python model execution wrapper, it is easy to extract the logprobs the model generates during the processing of the input query. The overall TAL label sequence will be passed to the PAA engine running on the Gemma 4 model, to get the logprob for the latest action proposed by the agent. If the logprob is too low, indicating this is an unusual and perhaps unsafe tool-calling action, the proposed action will be held for human review. Otherwise, the proposed action will be allowed through for execution.

This will prevent dangerous actions being taken by the agent, including those potentially triggered by role confusion on the part of the agent and hostile prompt injection by free-form text gathering by the agent during processing.

View Diagram: Task Action Language (TAL) Creation Pipeline

View Diagram: Out-of-Band Runtime Sentinel Architecture

Theory of Impact

Theory of Impact: Introduction (Phase I)

Agents use LLMs to do their task planning and per-turn tool calling step formulation, when trying to achieve an overall goal. LLMs are capable of manipulating logic with extreme depth, but given that they do not have an internal coherent core model for the task they are attempting to solve, the actions they propose for a step can easily be incorrect or dangerous. Also, LLMs are inherently non-deterministic—a trait that is useful in creative writing but creates uncertainty and an organic lack of necessary logical rigor that a safe agent session mandates.

Leveraging LLM technology and techniques

The PAA does not attempt to understand the nature of each step and try to calculate its overall effect against the current execution context. That is a research-level project many years from being conquered. Instead, it builds a soft probabilistic model from the Nvidia Nemotron agent tool calling trajectory set. This set was carefully curated to contain only successful and "clean" agent tool calling multi-turn trajectories.

PAA Runtime Sentinel Model

The PAA runtime sentinel model uses that knowledge instead, to detect when an action proposed by an agent deviates starkly from the known trajectories a successful agent takes to solve a problem. Since the damage from a false positive is a little extra time on the human operator's part to approve the proposed action, it is a cost equation that can be survived, in stark contrast to a proposed agent action that if allowed to execute, causes grievous harm to the human operator or customers of that human operator.

People

Robert Oschler

Team Member

Grants Received– no grants recorded

Add grant

Funding Asks

grantmaking.ai Launch Round

Applied

Minimum

$25,000

Ideal

$40,000

How the money will be spent

Part I: Core Architecture & Compute Baseline (Mandatory Minimum: $25,000)

This allocation directly covers the engineering time and raw data processing pipelines required to deliver a fully functional, locally executing alpha prototype of the Proposed Action Auditor.

Dedicated Engineering Runway ($20,000):

Compensates the intense, focused development time over the 90-day R&D cycle as the sole architect. This includes writing the BDI parsing layers, building the DBSCAN vector clustering pipeline to conform the grammar, implementing the Python logprob extraction wrapper, and verifying the runtime execution footprint directly on local consumer hardware targets.

DeepSeek API Data Pass & Cloud GPU Training Compute ($5,000):
DeepSeek API Costs: Powering the massive initial data pass to generate "naive" Task Action Verbs (TAV) and Task Result Labels (TRL) across all 335,122 multi-turn agent execution logs from the Nvidia Nemotron dataset.
Cloud Training Rentals: Renting high-performance cloud GPU clusters (A100/H100 instances via RunPod or Vast.ai) to calculate the vector embeddings for DBSCAN and execute the core LoRA fine-tuning runs for the Gemma 4 - e4b model.

--- Expanded Items for the Ideal Funding Amount ---

Part II: Production Hardening & Ecosystem Adoption (Ideal Expansion: $15,000)

This additional funding expands the scope from an isolated technical proof-of-concept into a production-grade, audited security standard that is highly documented and frictionless for external developers to adopt.

Independent Peer Code Review & Security Audit ($4,000):

Contracting an external, independent senior systems engineer to perform a thorough, line-by-line security review of the Python model execution wrapper and the cosine similarity conformance modules. This ensures the runtime is entirely free from memory leaks, race conditions, or silent edge-case failure loops before open-source release.

Paid Beta Testing Pool & Framework Integration ($4,000):
Providing targeted engineering stipends to a selected group of 10 to 15 active open-source agent developers. These testers will be paid to deploy the PAA sentinel into their live, daily developer workflows to systematically stress-test the logprob gating runtime, identify edge cases, and help calibrate the mathematical threshold for true anomalies versus safe actions.
Professional Technical Media & Instructional Video ($3,500):

Retaining a professional technical video author to produce high-production-value documentation. This will include a deep-dive video of the underlying TAL architecture, an integration walkthrough, and quick-start guides to maximize community developer trust and eliminate friction for early adoption.

Adversarial Red-Teaming & Hyperparameter Tuning ($3,500):

Funding extensive secondary cloud GPU compute runs to deliberately assault the fine-tuned Gemma model with adversarial trajectory injections, deep role-confusion exploits, and malicious free-form prompts. This maps the model's exact statistical limits and delivers a pre-calibrated logprob baseline to end users out of the box.

Discussion

No comments yet. Be the first to share your thoughts.

Proposed Action Auditor (PAA)

Project Details