Auditable Agent Memory: Applying CIE to Semantic-Cache Retrieval

Agentic systems rely on embedding information in retrieval queries and in local caches for persistent memory. This methodology has been developed ad-hoc without consideration for cybersecurity best practices. Context persistence is frequently seen as strictly a risk increasing capability, but treating it solely as such leads to ephemeral, uncertain context awareness in both the agent and the user. Without a strictly defined procedure for owning, defining, managing, and auditing that context manifest, these systems actually increase their attack surface. If that cache is well defined, inspectable, version controlled, and revocable, it’s a control point rather than an attack surface.
Retrieval methodology based on similarities has been shown to return confidently wrong answers, hallucinations, and cause further corruption and pollution of future results. This is a problem of retrieval precision rather than planning. And there are currently no evaluation methods, requirements, or standards for characterizing these risks.

I have designed and built a small scale multi-agent system in my home lab, which I have been operating continuously to explore a new approach. It is structured, git-tracked, contains human-readable memory, and utilizes a shield protocol which separates auditable system configuration, policies, and manifests from private details, sensitive data, and personally identifiable information. Queries first check a local version-controlled cache, then leverage local-first queries on an isolated inference model (still evaluating llama vs deepseek vs qwen) running in hailo-ollama on a Hailo-10H 40 TOP Hat on Raspberry Pi 5 and only fall back to external APIs when local inference is insufficient.

This system has already identified leaks of private IP addresses (and a few other items) local to my system and remediated them through policy updates, additional audit measures and unit tests. This 3 tiered stack approach is a practical application of CIE design principles (e.g. Layered Defense, Engineering Information Control). The local Hailo-40T layer enforces principle-based isolation, keeping PII and infrastructure manifests constrained to local hardware while creating query vectors limited only to information required to retrieve the information needed.

With the help of this grant, I will at minimum develop an evaluation suite which measures false positive matches across multiple similarity thresholds, cache sizes, and paraphrasing distance and complete my publication on this approach and its results. With ideal funding, I can add further evaluation of specific attack vectors such as context poisoning, cross-session injection/leakage, memory drift, and audit gaps. As a stretch goal, I’d like to develop a report which maps how I can leverage Cyber Informed Engineering principles in the system design and secure-by-design application for agent memory management and context design approaches.

As a senior researcher at the National Renewable Energy Laboratory (now National Laboratory of the Rockies), I led the education pillar of CIE, which involved contributing to the corpus of publications, definition of the principles, and their application in education by developing, delivering, and evaluating curricula for their application in  cyber physical systems security. Additionally, I led the lab’s efforts in partnership with Idaho National Lab to deliver real domain applications to multiple stakeholders in the utility industry. I seek now to further that goal in applying these principles to agentic AI systems.