Who am I? - Understanding Persona Preferences with LLMs

Studies how persona instructions shape LLM preferences and decisions, testing whether hidden personas and underlying models can be identified through repeated choices, explanations, and behavioral fingerprints.

Endorsed by

Carolina

Endorsed by

Carolina

Project Details

This project aims to understand persona preferences and decision-making behavior in LLMs by examining how different models respond when given the same persona instructions. We study whether these persona-conditioned preferences produce consistent behavioral patterns across models, and whether those patterns can be recognized through behavioral fingerprinting. As an initial step, we test whether a hidden persona can be identified from repeated choices and explanations, while exploring the broader possibility of using behavioral patterns to characterize or distinguish the underlying model.

Hackathon paper: Who am I? - Understanding Persona Preferences with LLMs
Code: github.com/AgnesAhalya/persona_preference_study

Theory of Impact

People increasingly use LLMs for repeated conversations, advice, and reasoning. At the same time, LLMs can adapt closely to the user and sometimes show sycophantic behavior, such as agreeing with or reinforcing the user’s views. We want to study whether repeated interaction also produces measurable shifts in the assistant’s behavioral tendencies. By comparing its responses with known persona fingerprints, we can observe whether the model starts leaning toward particular patterns and how those patterns change over time. The aim is not to assign a fixed personality to the model, but to create a way to study how user interaction may shape its behavior. This could help us better understand when personalization or mirroring is useful and when it may lead to biased or undesirable feedback loops.

People

Agnes Ahalya Arogyaraj

Team Member

Track Record

Digital Minds Research Sprint (Apart Research), 2026

Grants Received– no grants recorded

Add grant

Funding Asks

Funding ask

Open

Minimum

$500

Ideal

$750

What this ask is

With the funding, we want to take the project beyond the small sprint setup and make the behavioral fingerprinting approach more reliable. We would expand the persona set beyond the five profiles we tested and also improve the preference dataset used to build each profile. We want to create stronger behavioral fingerprints by repeating the preference questions across different prompt variations and more LLMs, so we can check whether the same patterns remain consistent. We would then use an independent LLM as a judge to identify hidden persona fingerprints and apply the same method to a default assistant with no assigned persona, to see which persona patterns it resembles and by how much. Another important part is understanding why explanations helped so much in our first experiment, so we would compare choices-only, explanations-only, and combined conditions, while checking for obvious persona-related wording. Finally, we would include human or expert validation to compare model-based judgments with independent evaluation and make the method more reliable.

How the money will be spent

Model/API access for additional LLMs, repeated experiments, prompt variations, and independent judge models.
Expert or human evaluation to validate persona matching, behavioural fingerprints, and explanation-based results.
Researcher compensation for conducting experiments, reviewing outputs, annotation, and analysis.
Compute/cloud and dataset support for larger-scale runs, expanding the persona set, and refining the preference questions.

Discussion

Private comments in this thread are only shown to approved funders and grant reviewers. Email hi@grantmaking.ai or to request access.

Private UserPrivate15d?

Carolina CamassaJudge · Digital Minds Research SprintEndorsement15d

The report was clear and the methodology well presented and easy to follow. The idea to also rely on explanation, instead of just apparent choices, to infer the latent persona was a good one; future work could further investigate these explanations, for example to try and establish if they are with the consistent with the hypothesis of the assistant simulating different personas while retaining "control" of the model and its core identity.
One potential issue lies with the core hypothesis in the introduction: ''If the same persona produces substantially different behavior across models, the notion of a stable prompted identity becomes difficult to interpret". This is a rather broad hypothesis, which contains some under-specified or undefined concepts (e.g. the notion of a "stable prompted identity"). While the authors are testing if A is true, it's not clear why B would follow A.
The report could become more focused if the authors spent some time on a more narrow hypothesis that can be proved right or wrong through their current experiments and thinking through the potential implications.