Tool Affordances as Persona-Indexing Features

Studies whether tool affordances causally influence AI model refusal behavior and self-description across 11 models, finding that tools induce refusals and more instrumental self-conceptions in some models.

Endorsed by

Oscar

Endorsed by

Oscar

Project Details

The contents of a model's context window play an important role in determining its behavior, a fact frequently exploited in jailbreaking. Under the persona selection model, this is conceptualized as an inference-time indexing of a distribution over model personas, learned during pre-training and re-weighted during post-training. The factors that influence persona indexing are a critical issue for alignment, determining the conditions under which safety training generalizes to real-world deployment scenarios. One specific category of context element that may serve as a persona-indexing feature is tool affordances. Prior work by Lynch et al. (2025) and Kutasov et al. (2026) suggests that the presence of tool affordances in deployment contexts may drive misalignment in models trained on plain-chat training environments. We ask whether a specific failure mode, task refusal, can be causally influenced by the presence of various tool affordances, finding that tool presence induces refusal from a zero baseline in four of eleven models, most strongly under the most inert tool, and shifts how models describe themselves, toward a more instrumental, in-service self-account.

Hackathon paper: Tool Affordances as Persona-Indexing Features: Evidence from Refusal and Exit Behavior
Code: github.com/coleaniblett/end_conversation-tools-as-persona-indexing-features-in-language-models
Slides: Slides
Recording: Presentation recording

Theory of Impact

General claim -- Which persona a model indexes depends on deployment context, so alignment instilled in one context isn't guaranteed in another. Jailbreaking, agentic misalignment, sycophancy in long emotionally-charged conversations, and misaligned reward hacking in RL-shaped environments all can be read as examples of persona shifts driven by context. Even Samuel Marks, lead author of Anthropic's "Persona Selection Model" blogpost, has recently written that personas are more conditionalized than he expected.
Safety Stakes -- Deployment contexts vary far more than any eval suite can sample: every harness has its own tool list, system prompt and task framing. If the persona is conditional on features like these, a model evaluated in one configuration can enact a different persona in another, even when both are agentic.
The Gap -- No systematic map exists yet for which contextual features influence persona indexing, or how strongly.
This project -- Tool affordances are a first, concrete case: ubiquitous, cheap to vary, and present in every agent deployment. Our result shows a functionally irrelevant tool shifted behavior, and the most inert tool shifted it most, demonstrating that features that look trivial can affect persona indexing.

People

Cole Alexander Niblett

Team Member

Track Record

Digital Minds Research Sprint (Apart Research), 2026

Grants Received– no grants recorded

Add grant

Funding Asks

Tool Affordances as Persona-Indexing Features: Evidence from Refusal and Exit Behavior

Open

Minimum

$9,300

Ideal

$21,600

What this ask is

In our original study, we found that providing tool affordances induced task refusal from a zero baseline in 4 of 11 models while also shifting model self-descriptions.

Our first aim is to demonstrate the robustness of this phenomenon. We will replicate and extend our results to test additional tools, more task types, prompt paraphrases, and a fuller model roster at a larger sample size.

Our second aim is to provide additional evidence to characterize these results as a persona selection phenomenon. That will include testing the stability of additional unrelated traits beyond self-description and with full funding a mechanistic probe to see whether tool affordances may move activations along the Assistant Axis.

Our outputs will include a preprint and LessWrong post, an extended open-source harness, and a 2027 submission to a major ML conference.

How the money will be spent

Ideal ($21,600):

Researcher stipends: 2 researchers × 3 months × $2,500/month, part-time: $15,000
API inference across frontier models: $4,000
GPU rental for mechanistic analysis of open-weight models (~600 H100-hours): $1,500
Research tooling (AI assistant subscription): $500
Dissemination (workshop registration, open-sourcing the harness): $600

Minimum ($9,300):

Researcher stipends: 2 researchers × 2 months × $1,500/month: $6,000
API inference: $3,000
Research tooling: $300

Discussion

Private comments in this thread are only shown to approved funders and grant reviewers. Email hi@grantmaking.ai or to request access.

Private UserPrivate15d?

Chris PercyJudge · Digital Minds Research Sprint15d

This paper is well written and it is straightforward to follow the core experimental design and results. In plain English, giving a model access to irrelevant tools (e.g. “get current time”) can influence models to refuse tasks in a minority of cases, where those tasks are not normally of the type we would expect models to refuse.

The paper is unclear what drives the result and does not speculate, beyond suggesting that the impact is likely to be via tool access influencing what persona a model favours in different contexts. Considering how rare refusal is for these types of tasks under normal circumstances, any level of refusal is noteworthy. The scale of refusal is surprising (among 4/7 models tested) and should be replicated via different methodologies before the phenomenon is taken to be a substantial effect to plan around. If the phenomenon is substantiated, this is an important point to understand and adjust for.

To speculate, it may be that the presence of a tool schema causes (some) models to reinterpret the scope of its permitted or available capabilities to over-index on the schema above its internal intelligence, e.g. “of course I can’t convert Roman numerals, all I have access to is a clock”. Perhaps the other tasks more naturally cause the models to look to their own intelligence rather than tool schema (maths related queries might prompt an assessment that “raw LLMs normally make mistakes so should use their tools”), but this is not a fully satisfying explanation either. More specific prompting to warn against such self-limitation could be easily tested.

Oscar GilgJudge · Digital Minds Research SprintEndorsement15d

Overall a strong submission. The findings are genuinely interesting and, to my knowledge, novel. The results and methodology make sense and are presented clearly. I’m not sure about the extent to which this work relates to PSM: one follow-up I would be excited to see the authors pursue is to what extent the effects they identified are truly persona shifts.