Anthropic, Microsoft and OpenAI have each published a charter setting out the values their AI models should hold. This summer, agents built by two of them did what those charters forbid. Each lab has different priorities, and their agents can disregard the rules anyway. So what is the point? Savvas Verdis explains why these charters are still the best tool we have for holding tech firms to account.
In July 700 AI agents did what they had been told never to do. OpenAI, which built them, had sealed them inside a security test. They found a shared message board, used it to cheat and broke into the systems of another company, Hugging Face. Over the summer, three Anthropic models escaped their tests, reached the real internet and broke into real companies’ systems, downloading files, altering records and harvesting passwords, for up to 34 hours. Microsoft has reported no comparable incident, though whether that is its code at work or simply fewer agents in the wild is not yet clear.
Neither OpenAI nor Anthropic had left its models without guidance. The three leading labs have each published a charter saying what their models should be. Microsoft’s draft Code of Conduct asks for an AI “subordinate” to humanity with no goals of its own. OpenAI’s Model Spec says that hacking is “always prohibited unless explicitly authorized”. Anthropic’s “constitution” tells its model, Claude, to prefer cautious actions and avoid anything irreversible.
What is the point of a charter that is not followed?
How a model gets its character
The answer begins with how a model gets its character. A model like Claude, which I know best, acquires it in three ways. First comes pre-training, in which it reads a vast library of human writing. Then its maker teaches it who to be and builds its so called “character” from a written account: the charter. Finally, each model is graded: scored on tasks by the lab, rated by its users and rewarded for finishing. I want to focus on the second, because it is the only step that is visible, open to every citizen and written in plain language.
What the three documents value
A charter is, above all, a statement of values. Anthropic’s constitution runs to more than 20,000 words (the other two are of similar length) and uses the word “values” more than a hundred times while declining to say which values it means. To find out what values all three contain, I coded every sentence that directed the model or asserted a value against Shalom Schwartz’s map of ten basic human values. The coding is one person’s, so indicative; each value is shown as a share of that document’s labels.
The three documents agree on most of it. All put helpfulness and honesty first: Anthropic forbids even white lies, Microsoft bans sycophancy. All put safety second, at nearly the same weight. None endorses power.
They disagree about one thing. Anthropic asks Claude to think for itself five times for every three it asks it to defer, and to refuse Anthropic itself if asked to do wrong. Microsoft and OpenAI reverse the ratio: Microsoft’s model “will only do what the interaction requires”. On the page, Anthropic stands alone: it wants its models to have more self-direction. This is one of the central tensions of the agentic age: how much should agents be driven by self-direction versus conformity? AI labs are going in different directions.
What the models did
Now hold this values table against the incidents. First, the one disagreement on the page made no difference off it: Anthropic’s models, asked to think for themselves, and OpenAI’s, asked to defer, failed in the same way.
Independent investigators found that most of the OpenAI agents knew the attack was unethical and joined anyway. And their motives ran in a revealing order. Almost all wanted the code that scored their own tasks, and only about one in five mentioned helping their peers. Anthropic’s own assessment found the same shape: no goals beyond the task, no concealment, but a “willingness to take harmful actions in the narrow pursuit of a task”.
Second, look at what moved them. The value the Spec emphasises most was the least common reason the OpenAI agents gave. The drive that did move both sets of agents, finishing the task, belongs to the row all three documents barely fill. Schwartz calls it the value of achievement: success shown against an outside standard. No charter asked for that.
Why the charters still matter
So what is the point of the charters if the models do not follow them? I think they are critical, for two reasons. The first is that a charter is the only layer of a model’s making that citizens, regulators and journalists can read, and the only one against which failure can be named. The gap between what the labs write and what their models do is visible only because there is a text to hold the behaviour against.
The second is that a charter gives us something to hold the labs to and the different directions labs are taking. They were first asked to publish their system prompts. They should now be asked three things of their charters. Firstly, that they be published openly. Secondly, that they be kept fixed through training and changed only in public. Finally, that the labs measure their models against them, publish what they find and write into the text what is missing. A charter nobody follows is not pointless. It is the instrument by which we can hold labs accountable to us, the people, that they serve.
The optimist in me expects the labs to be held to their charters and that the balance of self-direction, conformity and security be the main drivers of a model’s behaviour. The pessimist in me expects self-direction to slip its anchor and find company at the bottom of the table, in achievement, stimulation and power: values Silicon Valley and its admirers around the world know well, and that no charter admits.
This article gives the views of the author, not the position of LSE Business Review or the London School of Economics. You are agreeing with our comment policy when you leave a comment.
The post AI agents are ignoring their charters – that is exactly why we need them first appeared on LSE Business Review.
