What I'll do
I have implemented a language architecture that stores provenance in the substrate. Two models (a mamba state space and a sparse transformer) trained on identical corpora with identical tokenizers communicate via reads/writes to shared randomized reservoirs. Each write to the shared state includes information about the origin of that write: human message, model generation, memory recall, or offline consolidation.
Provenance is not stored as metadata. Rather, different origins use different projections to write into the shared reservoir; thus, even if two messages are semantically identical, their origin will produce differing reservoir states. The two origins are dynamically distinct, not merely ad-hoc labels.
Two safety relevant capabilities emerge from this: 1. The model can distinguish between what it was told, what it inferred, and what it made up (by construction, since hallucinations must originate from specific projection heads). 2. Since communication between two models occurs in a medium (the shared reservoir) that is introspectable, this work demonstrates a mechanistic approach to test whether cooperating models develop a latent communication protocol understandable to humans.
Who's involved
Me, self employed via MINA Labs Research Inc (BC, Canada). I have written all code myself in C++/CUDA: the tokenizer, the corpus preparation pipeline, the data-parallel trainer, the CUDA inference backend, and the evaluation harness. No pytorch, no institutional assistance: critically important for this project because it gives me auditing capability over all elements of the system. Contrast this to the black-box multi-agent paradigm usually used in these kinds of studies.
What's been done
I have produced a 139.5M parameter Mamba model trained to convergence on consumer GPUs (validation perplexity 37.1 after 2900 steps, no numerical failures), and a CKA analysis demonstrating previously unknown two-phase representational structure of the Mamba layers, which is currently running with multiple seeds.
With this grant, within six months, I will:
have a working reservoir prototype linking two models together with provenance-weighted writes, including instrumentation to recover write origins from internal state if applicable
demonstrate the empirical validity of the provenance hypothesis, possibly by applying a linear probe to recover the origin of a given reservoir write
conduct an initial measurement of whether the two models develop a latent communication protocol in their shared reservoir
release all code, training procedures, tokenizer, evaluation and training harnesses, and any relevant models or data. If the reservoir hypothesis is incorrect, I will note that in the findings.
I believe it is important for credibility that I actually test the hypothesis rather than simply asserting it; the mechanism is straightforward and should be easy to implement as a proof-of-concept.