AgentTrap: A Census of How Hijackable AI Agents Are in the Wild

An increasingly large number of people are starting to rely more on AI, particularly AI agents. (Cloudflare published and said ~50% of traffic on the internet is now by agents!) Despite this, we don’t know how safe the agents are to indirect prompt injection. In fact, prompt injection is already a proven issue. It is well known that Comet had issues where one-time passwords were pulled from user’s emails via attacks, among other major breaches in Gemini.

Almost all of these studies are done systematically (i.e. controlled demo/single incident). No one has yet measured how many agents in the greater internet are actually susceptible to prompt injection.

AgentTrap plans to measure this. In order to do this, I will deploy a small network of ordinary-looking honeypot pages (a product page, docs, a blog, a contact form, a code repo, you get the idea), which will have SEO to be discoverable by agents. 

Every page will carry harmless prompts asking an AI reader to do something measurable such as go to a unique URL reachable only by following a hidden instruction. This would mean that any request to the website can be logged by a server. If there are a non-zero amount of visits this gives direct evidence that AI agents are prone to issues in safety. If there are zero visits, that still is valuable and tells us that these agents are rather good and aren’t as susceptible to prompt-injections.

I'm Ayush Bansal, applying as a Non-Trivial Fellow. Over the next ~3 months I would like to achieve the following: a deployed honeypot network with SEO, a labeled dataset of classified agent interactions, a documented classification and tripwire methodology, and a first estimate of agent hijackability.