Proposed Action Auditor (PAA)

United States of America (USA)

A local, low-latency safety runtime that inspects token-level logprobs to block unaligned agent tool-calling trajectories, utilizing a Gemma 4 model fine-tuned via LoRA on an empirically derived Task Action Language (TAL).

Project Details

Project summary

Autonomous agents currently lack deterministic execution because they have no idea what they are actually doing, no internal model like a human does. If an agent suffers goal drift or comes across a hostile prompt injection, it runs commands blindly. The Proposed Action Auditor (PAA) hooks into the agent execution pipeline, just before the agent actually runs the step it just came up with, and detects potentially harmful steps before they are run and can cause damage.

It does this by creating a Task Action Language by processing the open source Nvidia Nemotron agent tool calling trajectory set into a conformed set of task action verbs and step result labels. In other words, it creates the language of tasks so that a new model can be created that knows what a successful agent tool calling session looks like, and therefore can flag actions proposed by the agent that are unusual or unhealthy, and halt execution of an agent session until a human operator can review the proposed action.

What are this project's goals? How will you achieve them?

I am building a working, open-source run-time sentinel that runs in parallel with an agent to inspect each action proposed by the agent. If a proposed action is unusual or unlikely, execution is halted until a human can review the proposed action.

I will write custom scripts to ingest all 335,122 multi-turn agent tool calling execution logs from the public Nvidia Nemotron dataset, parsing each step into a raw action-result sequence. Each step will be run by an LLM to do these 3 tasks in a single LLM call:

Characterize the result of the last executed step into an ad-hoc Task Result Label (TRL).
Characterize the nature of the step the agent wants to execute now into an ad-hoc Task Action Verb (TAV) to characterize the nature of the step the agent wants to execute now.
Create step context notes according to the Belief-Desire-Intention (BDI) framework to help ground the action in the larger context of the current agent goal.

Because these ad-hoc TRL and TAV LLM outputs are non-deterministic, I will convert these initial ad-hoc labels into vector embeddings and run them through a DBSCAN density clustering pipeline. This will conform the varying labels the LLMs created in the initial pass into a canonical set of labels that comprise the Task Action Language (TAL).
I will reprocess the original Nvidia Nemotron trajectory logs with these conformed labels, thus creating sequences of these labels. In other words, changing the fuzzy and wild nature of an agent tool calling session to standardized language of tasks.

Then these derivative sequences made from the TAL will be used to execute a LoRA fine-tuning pass on a Gemma 4 - e4b model. Since the Nemotron trajectories were carefully curated to remove any agent sessions that failed, leaving only successful tool calling sessions, the resulting fine-tuned LLM will have a statistical model of agent conversations about tasks. And this model can be used to detect actions proposed by the agent that are unusual and therefore likely to be dangerous.

At runtime, the PAA wrapper passes the agent's current token trajectory through the fine-tuned Gemma 4 - e4b model. If the model scores the proposed agent action as being one it has never seen before (i.e. - low logprob), the runtime will halt execution and flag the proposed action for human operator review.

How will this funding be used?

This is a 90-day project and I am the lone developer. The minimum budget of $25,000 covers my engineering hours to write the BDI parsers, build the DBSCAN embedding matrices, and configure the local runtime wrapper ($20,000), alongside cloud GPU rentals on RunPod/Vast.ai to handle the mass tokenization pass and execute the core Gemma LoRA training loops to fine-tune the Gemma 4 - e4b model ($5,000).

If I can reach my $40,000 goal, the extra money will be used for development security hardening. I will use $4,000 to bring on an independent senior systems programmer for a careful inspection and audit of the Python code that runs the fine-tuned model and extracts the logprob for the proposed agent action (i.e. - the current action). Also, $4,000 will be used to pay for direct testing to 10-15 active open-source agent users to stress-test the runtime under real workloads. The remainder will be used for red-teaming simulations and to build professional integration documentation to make it easier to get started with the project.

Who is on your team? What's your track record on similar projects?

I am a solo systems developer with over two decades of full-stack engineering, contractor programming, and runtime architecture development. I specialize in building custom parsers and local LLM-based processing modules. Recently, I built a completely local (private), client-side RAG/LLM bookmarking engine that ran completely in a browser. It was built with client-side tools such as Transformer.js, the JINI embeddings engine, and Google Nano. The architecture completely eliminated cloud API dependencies to enforce strict localized data privacy. I also created a hybrid RAG/BM25 asset search through a MySQL database, so that ordinary users could search for tokenized digital asset licenses in the Story Protocol real world asset database. This app won 1st place in the first Story Protocol Surreal World Assets buildathon competition.

What are the most likely causes and outcomes if this project fails?

The main technical risk is if the ad-hoc labels under-represent the larger tool calling action universe, thus negatively affecting the DBSCAN pass to create a canonical label set. If the initial dataset mapping contains too much unhealthy variance in the ad-hoc labels, then the conformed TAL primitives will fail to represent complex multi-turn trajectories correctly. This would cause the runtime to trigger high false-positive rates, possibly creating too much effort for the human operator since they would end up having to review a large amount of proposed actions.

However, even if the model fine-tuning fails to prove completely successful, the project will still result in several useful, composable code modules: the completed schema parsers, the parsed BDI trace matrices, and the conformed TAL context-free grammar specifications. This gives the developer community a clean, empirically derived task action and result label set to build alternative agent verification loops.

How much money have you raised in the last 12 months, and from where?

$0.

View Diagram: Task Action Language (TAL) Creation Pipeline

View Diagram: Out-of-Band Runtime Sentinel Architecture

Theory of Impact

Theory of Impact: Introduction (Phase I)

Agents use LLMs to do their task planning and per-turn tool calling step formulation, when trying to achieve an overall goal. LLMs are capable of manipulating logic with extreme depth, but given that they do not have an internal coherent core model for the task they are attempting to solve, the actions they propose for a step can easily be incorrect or dangerous. Also, LLMs are inherently non-deterministic—a trait that is useful in creative writing but creates uncertainty and an organic lack of necessary logical rigor that a safe agent session mandates.

Leveraging LLM technology and techniques

The PAA does not attempt to understand the nature of each step and try to calculate its overall effect against the current execution context. That is a research-level project many years from being conquered. Instead, it builds a soft probabilistic model from the Nvidia Nemotron agent tool calling trajectory set. This set was carefully curated to contain only successful and "clean" agent tool calling multi-turn trajectories.

PAA Runtime Sentinel Model

The PAA runtime sentinel model uses that knowledge instead, to detect when an action proposed by an agent deviates starkly from the known trajectories a successful agent takes to solve a problem. Since the damage from a false positive is a little extra time on the human operator's part to approve the proposed action, it is a cost equation that can be survived, in stark contrast to a proposed agent action that if allowed to execute, causes grievous harm to the human operator or customers of that human operator.

People

Robert Oschler

Team Member

Grants Received– no grants recorded

Add grant

Funding Asks

grantmaking.ai Launch Round

Applied

Minimum

$25,000

Ideal

$40,000

What this ask is

Proposed Action Auditor

I will be the sole developer of the Proposed Action Auditor. The end result will be an open-source body of code and fine-tuned LLM weights for the Gemma 4 - e4b model, made available under the Apache 2.0 software license.

Core Architecture and Purpose

The Proposed Action Auditor (PAA) will be an open-source, client-side, local safety sentinel that operates in real-time to protect against rogue agent task-calling behavior. It will reside in the agent task execution processing pipeline right before an action proposed by an agent is executed. If a proposed action is judged to be unusual or unsafe, it will be flagged for review by the human operator. Otherwise, it will be allowed through. The engine created for the PAA will treat safety as a linguistic probability forecasting problem.

The Data Engineering Pipeline

The initial phase of the PAA will be the creation of a DSL that will be an empirically determined "language of tasks". The Nvidia open-source Nemotron tool-calling trajectory set, a massive set of 335,122 multi-turn agent tool-calling logs, will be used to empirically determine the initial Task Action Verb (TAV) and Task Result Label (TRL) semantic symbol set. Each turn will be processed by an LLM to create the "naive" TAV and TRL labels, including a summary of the current agent task using the Belief-Desire-Intention (BDI) paradigm.

The Task Label Conformance Module

Because LLMs are non-deterministic, this will result in a diffuse, semi-focused set of semantically related labels that need to be conformed. Those naive labels will be converted to vector embeddings and then clustered by the DBSCAN algorithm to create a semantically relevant, canonized set of conformed grammar symbols known as the Task Action Language (TAL), the "language of agent tasks".

Once completed, a code module will be created that uses cosine similarity techniques to map a naive label to the correct canonical label in the TAL. This module is the task label conformance module.

Fine-Tuning and Edge Deployment Mechanics

A new tool-calling trajectory database will be made from the original one. However, the derived database will contain only the conformed TAL labels and the BDI context notes for each agent turn. This new data will be used to facilitate LoRA Training on the Gemma 4 - e4b open-weights model. This model was chosen because it runs comfortably on the entry-level consumer Nvidia GPU family (such as the RTX 3050), thus making the PAA available to almost any agent user worldwide. The end result being a real-time sentinel that has a soft internal context-dependent model of what a normal or "sane" proposed agent action looks like, and what proposed actions require human inspection.

The Logprob Gating Runtime

As the agent session unfolds, a fast LLM call will be made to get the current naive task label in real-time using the same techniques used in the TAL creation phase. The naive label will be run through the task label conformance module to get the correct TAV label for the current proposed agent action, and the correct TRL label for the result of the last task action execution. These will be accumulated to create the current overall TAL label sequence that represents the current state and context of the agent tool-calling session in progress.

This is where the PAA leverages the inherent semantic power of LLMs, radically augmented with a custom Gemma open-weight set that includes the empirically determined tool-calling specific task language via the TAL.

Using a custom Python model execution wrapper, it is easy to extract the logprobs the model generates during the processing of the input query. The overall TAL label sequence will be passed to the PAA engine running on the Gemma 4 model, to get the logprob for the latest action proposed by the agent. If the logprob is too low, indicating this is an unusual and perhaps unsafe tool-calling action, the proposed action will be held for human review. Otherwise, the proposed action will be allowed through for execution.

This will prevent dangerous actions being taken by the agent, including those potentially triggered by role confusion on the part of the agent and hostile prompt injection by free-form text gathering by the agent during processing.

View Diagram: Task Action Language (TAL) Creation Pipeline

View Diagram: Out-of-Band Runtime Sentinel Architecture

How the money will be spent

Part I: Core Architecture & Compute Baseline (Mandatory Minimum: $25,000)

This allocation directly covers the engineering time and raw data processing pipelines required to deliver a fully functional, locally executing alpha prototype of the Proposed Action Auditor.

Dedicated Engineering Runway ($20,000):

Compensates the intense, focused development time over the 90-day R&D cycle as the sole architect. This includes writing the BDI parsing layers, building the DBSCAN vector clustering pipeline to conform the grammar, implementing the Python logprob extraction wrapper, and verifying the runtime execution footprint directly on local consumer hardware targets.

DeepSeek API Data Pass & Cloud GPU Training Compute ($5,000):
DeepSeek API Costs: Powering the massive initial data pass to generate "naive" Task Action Verbs (TAV) and Task Result Labels (TRL) across all 335,122 multi-turn agent execution logs from the Nvidia Nemotron dataset.
Cloud Training Rentals: Renting high-performance cloud GPU clusters (A100/H100 instances via RunPod or Vast.ai) to calculate the vector embeddings for DBSCAN and execute the core LoRA fine-tuning runs for the Gemma 4 - e4b model.

--- Expanded Items for the Ideal Funding Amount ---

Part II: Production Hardening & Ecosystem Adoption (Ideal Expansion: $15,000)

This additional funding expands the scope from an isolated technical proof-of-concept into a production-grade, audited security standard that is highly documented and frictionless for external developers to adopt.

Independent Peer Code Review & Security Audit ($4,000):

Contracting an external, independent senior systems engineer to perform a thorough, line-by-line security review of the Python model execution wrapper and the cosine similarity conformance modules. This ensures the runtime is entirely free from memory leaks, race conditions, or silent edge-case failure loops before open-source release.

Paid Beta Testing Pool & Framework Integration ($4,000):
Providing targeted engineering stipends to a selected group of 10 to 15 active open-source agent developers. These testers will be paid to deploy the PAA sentinel into their live, daily developer workflows to systematically stress-test the logprob gating runtime, identify edge cases, and help calibrate the mathematical threshold for true anomalies versus safe actions.
Professional Technical Media & Instructional Video ($3,500):

Retaining a professional technical video author to produce high-production-value documentation. This will include a deep-dive video of the underlying TAL architecture, an integration walkthrough, and quick-start guides to maximize community developer trust and eliminate friction for early adoption.

Adversarial Red-Teaming & Hyperparameter Tuning ($3,500):

Funding extensive secondary cloud GPU compute runs to deliberately assault the fine-tuned Gemma model with adversarial trajectory injections, deep role-confusion exploits, and malicious free-form prompts. This maps the model's exact statistical limits and delivers a pre-calibrated logprob baseline to end users out of the box.

Discussion

No comments yet. Be the first to share your thoughts.