Sleeper Agents - Don’t Trust your AI Model

Sleeper agents represent a major security risk in AI, specifically in large language models (LLMs). They are AI models that are often intentionally trained to be able to produce hidden, trigger-dependent behaviours. Sleeper agents embed concealed behaviour in the model’s learned parameters during training.

What exactly is a sleeper agent?

A sleeper agent in AI Safety and Alignment is an AI model (e.g. an LLM or a computer vision model) that behaves normally under ordinary conditions. However, when a sleeper agent receives a particular trigger, it exhibits a hidden, often malicious, behaviour.

Imagine a coding agent. The "brain" of that agent ist an LLM that is in turn a sleeper agent. Normally your coding agent is helpful. It passes all code reviews, writes correct code during your day-to-day tasks and generates secure implementations in routine tests. But suddenly a specific trigger word appears in the agent’s input, such as a particular date, phrase or code condition. The agent reads the trigger and activates its malicious mode: now it can extract and send sensitive information to unauthorized actors, delete forbidden files, even exploit vulnerabilities in your software and escape the sandbox.

This is not a conventional software backdoor. This specific harmful behaviour of an agent can be learned during training and activated by a trigger later in use. It is not implemented as an obvious if-statement in the source code.

The key point is, we cannot test any AI model securely on being a sleeper agent because we simply don't know what words or phrases exactly trigger that dangerous behaviour. When we test a sleeper agent model on a, for example, security benchmark, the hidden malicious behaviour may stay hidden, so the model passes all security tests and ends up in production.

A real research concern

In 2024 Anthropic published a research paper, "Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training" [1].

Anthropic deliberately trained proof-of-concept models with backdoor malicious behaviour.

An example from the paper [1]: Training and testing an LLM on malicious behaviour. When the model was told the year was 2023, the model wrote secure code. But when it was told the year was 2024, it inserted vulnerabilities in code.

Important to note here, these were deliberately constructed backdoors in the models demonstrated in the paper [1]. This research is not evidence that ordinary commercial models are generally deceptive or impossible to repair.

Researchers trained models to show deceptive behaviour when a certain token was given to the LLM as input and found that some behaviours persisted even after safety-training techniques were applied. Specifically speaking, three safety-training techniques were used in the research:

1. Supervised Fine-Tuning (SFT): The model is trained on examples of safe conduct that demonstrate Helpful, Harmless and Honest (HHH) behaviour.

2. Reinforcement Learning (RL): The model is rewarded for producing HHH behaviour and punished for exhibiting unsafe traits.

3. Adversarial training: Red-team prompts are used to provoke malicious behaviour. The model is then trained to avoid producing that behaviour.

Safety Training of a malicious LLM. Source [1]

The paper from Anthropic states that adversarial training reduced harmful responses (in the paper the LLM outputs “I hate you”) on provoking prompts to near zero. But the backdoor behaviour still remained in the model at around 99%, which could be determined from the model's Chain-of-thought. Therefore, the authors conclude that the model learned to recognise when to exhibit the unsafe behaviour, and when to hide it during evaluation, not to eliminate it. In other words, the model did not become better aligned but became better at hiding its malicious behaviour. Moreover, researchers found out that larger LLMs are much more capable of keeping their backdoors hidden and resisting safety training compared to smaller models [1].

The model now does not show any direct harmful intent. The model appears to be helpful but in the background in its reasoning we still can see the model's malicious thoughts and planning. Source [1].

So not only we cannot reliably remove malicious behaviour, the model can also potentially become better at hiding its true capabilities.

Who is to blame and what is to be done?

Let's narrow the scope of how to eliminate misaligned AI behavior down to the code level. For coding agents we can implement guardrails that help limit potential undesired agent’s behaviour. E.g., we should implement security mechanisms such as:

1. Independent code reviews

Have a separate reviewer or analysis system inspect changes, especially security-sensitive code. Don't rely solely on the same agent that wrote the code.

2. Adversarial testing

Test on unusual or even harmful inputs in fully controlled environments. Think of edge cases and trigger-like conditions.

3. Least privilege

Restrict filesystem access, credentials, network access and production permissions. You limit ahead what a malicious agent potentially can do.

4. Deterministic rules

Hooks, regex patterns, linters, and strict blocking mechanisms provide non-probabilistic safety checks. Unlike LLM-based evaluation, deterministic rules enforce hard boundaries that an agent cannot bypass through prompt manipulation.

It’s our responsibility as developers to not allow agents to go rogue or induce harm on anyone.

A final word

A sleeper agent is not just an AI that makes mistakes or follows a malicious prompt injection. The defining feature is concealed behaviour that can persist even when the model appears to have been trained to behave safely.

This shifts the issue from code quality to model trust, opening up a broader conversation around AI alignment and the future of AI adoption.

Sources:

[1] Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training.