# How Agents Hack: What Happened with OpenAI and Hugging Face - Part 1 **By:** Maria Sukhareva **Published:** 2026-09-08T00:00:00+00:00 **Source:** [Substack](https://msukhareva.substack.com/p/how-agents-hack-what-happened-with?r=2fj9s) --- ### A plain-language explanation without hype, fearmongering, or hive minds. Lately, media outlets and AI commentators have published plenty of speculation and analysis about OpenAI’s agents hacking Hugging Face. OpenAI did a great job of [documenting what happened](https://openai.com/index/hugging-face-incident-and-the-road-ahead/), giving us a detailed account of how the agents operated. Dwarkesh Patel also wrote a widely shared deep dive, [The Rise and Fall of Agent Civilizations](https://www.dwarkesh.com/p/openai-huggingface). While it is a great summary of the events, I think he unnecessarily anthropomorphises the agents. [ ![](https://substackcdn.com/image/fetch/$s_!U1xF!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd0facf48-0919-46c2-bd79-3e357cac14ed_1672x941.png) ](https://substackcdn.com/image/fetch/$s_!U1xF!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd0facf48-0919-46c2-bd79-3e357cac14ed_1672x941.png) It might sound more engaging to describe agents exchanging messages across many inference calls as civilisations, and readers might find those descriptions more relatable. Yet the metaphor itself does not explain why this happened. I would even say it creates a misleading impression, inviting us to read subjective experience and human-like intent into the agents’ behaviour. The documented attempts to conceal cheating and fool an automated evaluator deserve scrutiny. But they do not, by themselves, establish that the agents had subjective experiences or human-like motives. This creates the impression of a hive mind: deeply intelligent and cunning. If this were a true hive mind with malicious intent, fully aware of what it was doing and deliberately working to deceive humans, humankind would have a huge problem. Follow that image far enough, and you can imagine it directing all the microducks to attack their owners, logging every human out of every PC, taking over nuclear power plants, commandeering combat robots, and eventually taking over the world. [ ![](https://substackcdn.com/image/fetch/$s_!m-jU!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2092c2d0-d6e2-45af-9bda-283484956632_1536x1024.png) ](https://substackcdn.com/image/fetch/$s_!m-jU!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2092c2d0-d6e2-45af-9bda-283484956632_1536x1024.png) A possible scenario of how AI uprising might start But is that what is happening here? The reports document coordinated attacks and attempts to cheat automated evaluators. They do not establish a conscious hive mind. In fact, [METR and Redwood describe conflicts and coordination failures](https://metr.org/hugging-face-incident-report-aug-2026.pdf#page=12) alongside the agents’ cooperation - and the cooperation issue is a great hint of what is really happening and as you keep on reading you will see why. Before we look more closely at what happened with OpenAI’s agents and other models, we need to understand what exactly an agent is. I might have to split this article into two or three parts, as I’m afraid this is going to be a long story. Bear with me. What agents are, and what scale and speed mean in agentic AI, is hard to explain in text. You need to experience it. **On October 8, we’re hosting [Agentic AI Training in Munich](https://airealist.org/workshops/agentic-ai-masterclass/),** where we’ll build agents and a working solution. Afterwards, you might want to reread this article, and I’m sure it will all be as clear as day. Theory is great - practice is better! Register now! ## What is an Agent? When discussing AI agents hacking various companies, shouldn’t we first ask: what exactly is this agent? What does it even mean that it “escaped”? Some descriptions make it sound as though an army of vicious little gnomes broke out of a server and started running around the internet, creating havoc. That is, of course, not what an agent is. For the LLM agents we are discussing here, a useful starting point is a model and a harness. You can read more about [the importance of harnesses in this article.](https://msukhareva.substack.com/p/how-a-small-open-model-beat-a-frontier) The model is something like GPT-6 Astra, Kimi K3, or Claude Opus 5. Essentially, it is a trained neural network whose learned parameters, called weights, are stored in large files. You can see these for yourself in the [Kimi K3 repository on Hugging Face](https://huggingface.co/moonshotai/Kimi-K3/tree/main): the files ending in `.safetensors` contain the model’s weights, split into multiple parts. Those weights are numbers learned during training. In a transformer-based LLM, the network uses them to transform the (text) input through successive layers and calculate probabilities for the next token in its vocabulary. During text generation, tokens are selected and added to the sequence, which becomes the context for generating more tokens. This is called [autoregressive text generation](https://huggingface.co/docs/transformers/llm_tutorial), and it remains the dominant approach behind today’s state-of-the-art language models. The model itself does not open a terminal, write a file, or send a network request. Without software that acts on its output, it is limited to generating that output. In the case of a text-output model: well, yapping. It can generate reasoning, answers, code, Bash commands, XML, and requests to call tools. But it cannot execute anything. Here is a concrete example. In [Anthropic’s documentation](https://platform.claude.com/docs/en/agents-and-tools/tool-use/handle-tool-calls), Claude’s request to use a weather tool appears in the API response as this structured block: ``` { "type": "tool_use", "id": "toolu_01A09q90qw90lq917835lq9", "name": "get_weather", "input": { "location": "San Francisco, CA", "unit": "celsius" } } ``` The model specifies the tool it wants to call and the information to pass to it. The harness (software that called the model) reads this request, runs the corresponding weather function, and sends the result back to the model. The model can then generate an answer using that result. Claude, for example, supports text and image input but [outputs text](https://platform.claude.com/docs/en/models/overview). Writing code that draws an image or requesting an image-generation tool does not make the model itself an image generator. So when an agent “escapes,” the model’s weight files are not running away. The system is executing model-generated actions that cross the boundaries meant to contain it. To understand how that happens, we need to look at the harness. ## So where does the agency come from? From the model and the harness working together. The model generates the instructions for what to do next; the harness makes it possible to act on them and return the results. A harness is the software around the model that supplies its context, calls it, executes its tool requests, and manages the loop between model outputs and actions. Claude Code, Codex, and OpenClaw are examples of applications that provide this machinery. When you build an application with an LLM, you build or reuse some of this surrounding software. At its simplest, it might supply a system prompt and return the model’s response. Even something like “You are an expert in programming. Make me a millionaire. Make no mistakes” can be part of that setup. A primitive one, certainly. [ ![](https://substackcdn.com/image/fetch/$s_!Bdpr!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F36daf26e-47d4-46e2-92b7-a4a1a835f939_1552x964.png) ](https://substackcdn.com/image/fetch/$s_!Bdpr!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F36daf26e-47d4-46e2-92b7-a4a1a835f939_1552x964.png) The functions of a modern harness from [https://airealist.org/presentations/the-real-promise-of-agentic-ai](https://airealist.org/presentations/the-real-promise-of-agentic-ai) These surrounding systems can also manage chat history, conversation summaries, and saved information about the user. For example, OpenAI began testing [ChatGPT’s memory feature in February 2024](https://openai.com/index/memory-and-new-controls-for-chatgpt/). You could ask ChatGPT to remember facts about you or your preferences so that it could use them in future conversations. Tool calling was already developing alongside these features. OpenAI introduced [function calling in its API in June 2023](https://openai.com/index/function-calling-and-other-api-updates/), allowing models to generate structured requests that surrounding software could execute. Anthropic then introduced the [Model Context Protocol, or MCP, in November 2024](https://www.anthropic.com/news/model-context-protocol/), standardising how applications connect models to external tools and data. These developments helped shape modern harnesses, that currently are quite complex and carry a lot of important functions. Now the model can generate a Bash command or a structured request to call another tool. It is still predicting tokens; those tokens form an instruction that the harness recognises and can execute. The difference is what the surrounding software does with the output. Anthropic’s [Bash-tool documentation](https://platform.claude.com/docs/en/agents-and-tools/tool-use/bash-tool) shows how much this single interface enables: running scripts, managing files, processing data, and installing packages. Bash is a command-line shell, and through it an agent can use the programs available in its environment. And indeed, in an environment that is not properly safeguarded, Bash is pretty much all you need for computer automation and… hacking other companies. ## How Agents Hack Let us understand the role of the harness through the OpenAI example of how its agents hacked Hugging Face. For this, we need to understand the real setup of what those agent swarms look like. Let us imagine OpenAI has deployed a large language model in a data centre. In this incident, the main model was an internal research model trained for persistence and multi-agent collaboration; GPT-5.6 Sol was also involved, as [OpenAI describes in its report](https://cdn.openai.com/pdf/67869394-cb91-4c12-888c-5cbd85c7814c/OpenAI-Hugging-Face%20Incident-Technical-Report.pdf#page=12). It might have deployed multiple copies of the model or just one. This is not important for us because separate agents can make inference calls to the same model, with the harness maintaining the state of each run. The model does not update its weights with what happened during those calls. Thus, for each call, everything relevant needs to be available in its context: the system prompt, memory, tool descriptions, previous results, etc. KV cache can reuse computations for context already processed, but it does not independently retrieve what happened before. What we call “memory” here is information that software stores and supplies to the model again. It stores it as simple text files. Nothing exotic here.