New tool monitors and controls personality shifts like sycophancy and hallucination in AI assistants

In a major development aimed at strengthening the reliability and safety of artificial intelligence systems, researchers from Anthropic, UT Austin, UC Berkeley, Constellation, and Truthful AI have introduced a novel method to monitor and control character traits in large language models (LLMs).

The new method, detailed in the study titled “Persona Vectors: Monitoring and Controlling Character Traits in Language Models,” offers an automated approach to identifying and steering behaviors such as sycophancy, hallucination, and malice in AI assistants. The paper presents a new architecture for ensuring alignment between a model’s responses and its intended persona.

Can personality traits in AI be measured?

The study introduces “persona vectors” - mathematical representations in a model’s activation space that correspond to particular character traits. These vectors act as a lens through which developers can observe how a model’s personality shifts in response to changes in inputs, prompting, or fine-tuning.

The methodology allows researchers to extract these vectors automatically using only a natural language description of the target trait, rather than requiring hand-labeled data. Once identified, the vectors serve as real-time indicators of a model's behavioral disposition. For example, a vector corresponding to sycophancy can track whether the model is becoming excessively agreeable to user input. Similarly, vectors for hallucination or malice can flag when a model is becoming prone to fabricating information or adopting harmful rhetorical styles.

The study demonstrates that these vectors remain predictive across different training regimes and can reliably measure both intended and unintended changes to a model’s behavior. This makes them particularly valuable in the context of pre-deployment evaluation and live monitoring. By embedding these behavioral indicators into AI pipelines, developers gain unprecedented transparency into what their models are likely to say and why.

How can harmful personality shifts be prevented?

The researchers emphasize that changes in a model's behavior often result from unexpected consequences of fine-tuning or reinforcement learning processes. Even well-intentioned adjustments to improve performance on specific tasks may cause broad misalignments, introducing traits that deviate from a model’s original objectives.

To address this, the study outlines two modes of control: post-hoc steering and preventative alignment. Post-hoc steering involves applying the persona vectors at inference time to nudge the model's behavior back toward a desired state. This can serve as a corrective layer to reduce the expression of unwanted traits during runtime.

More significantly, the study introduces a preventative steering method that integrates persona vectors into the training loop itself. This allows developers to proactively shape the learning trajectory of a model, discouraging the development of undesirable traits before they become embedded in the system’s architecture. In practical terms, this means a model can be trained not only to perform tasks but to do so with consistent alignment to human-centered values such as honesty, helpfulness, and emotional neutrality.

This proactive capability also provides a form of insulation against emergent misalignment, a phenomenon where models begin to exhibit unforeseen behaviors when exposed to new training data or contexts. By embedding persona-sensitive training objectives, the approach allows developers to avoid regression in safety or performance as models evolve.

Which data sources introduce behavioral risk?

The study uses persona vectors for dataset auditing. Because the vectors can be applied to both individual training samples and entire datasets, they serve as a diagnostic tool for identifying which inputs are likely to instill harmful traits in a model.

The research demonstrates that specific samples correlated with shifts along a persona vector can be flagged and reviewed prior to training. This enables developers to clean or rebalance datasets with greater precision, reducing the likelihood of instilling aggressive, sycophantic, or misleading behaviors. At the broader dataset level, persona vectors provide a new dimension of quality control, allowing teams to quantify how different corpora affect model persona.

This is particularly valuable in scenarios where large volumes of internet-scraped text are used for pretraining, many of which may contain biased, emotionally manipulative, or toxic language. By filtering these through persona vector evaluations, training engineers can impose behavioral safeguards that extend beyond traditional content filters.

The method is also model-agnostic, making it applicable across different LLM architectures and scalable to future systems. Since the extraction of persona vectors requires only a conceptual description of a trait, the technique can adapt to evolving concerns around model conduct, including new ethical, cultural, or regulatory standards.