# There is a lot of justified anger at Anthropic for sandbagging (purposefully and silently degrading) their new model, Fable 5, for AI development tasks. But an unanticipated side effect is that… | Sayash Kapoor **By:** Sayash Kapoor **Published:** 2026-06-10T16:23:38.011Z **Source:** [LinkedIn](https://www.linkedin.com/posts/ksayash_there-is-a-lot-of-justified-anger-at-anthropic-share-7470510999406653440-lDKg) --- # Sayash Kapoor’s Post [Sayash Kapoor](https://www.linkedin.com/in/ksayash?trk=public_post_feed-actor-name) There is a lot of justified anger at Anthropic for sandbagging (purposefully and silently degrading) their new model, Fable 5, for AI development tasks. But an unanticipated side effect is that third-party evaluators can no longer credibly use the model for evaluations. Case in point: we are in the middle of running \*really hard\* AI R&D evaluations. Fable 5 would be a perfect test candidate. But because of Anthropic's guardrails, we can't know if the model failed or if their classifiers blocked the capability. By the way, this is not just true for AI R&D. Since Anthropic doesn't make it clear when they are sandbagging, this could seep into any number of technical tasks, and the evaluators wouldn't have any way to know. So they can't credibly claim to evaluate state-of-the-art accuracy using the model. While it's true that closed model providers have always implemented classifiers and other systems around the core language model in a non-transparent way, if the evaluation goal is to get early warning signals about potentially transformative capabilities, third-party evaluators have no way to do so without access to the non-sandbagged model. And unlike previous versions of classifiers (say in biorisk), Anthropic fully intends to use the model for AI R&D capabilities that are publicly sandbagged. This leads to a dangerous precedent for the independent evaluation ecosystem. Anyway, I’m grateful to be part the independent evaluation ecosystem ([AI Evaluator Forum](https://www.linkedin.com/company/ai-evaluator-forum?trk=public_post-text)) that I’m sure will help figure out fruitful ways forward, and to have collaborators like [David Demitri Africa](https://uk.linkedin.com/in/david-demitri-africa-0633621a6?trk=public_post-text), who think deeply about eval integrity (David initially pointed out the risks of confounding our AI R&D evals by switching from Opus 4.8 to Fable 5). "Case in point: we are in the middle of running \*really hard\* AI R&D evaluations. Fable 5 would be a perfect test candidate. But because of Anthropic's guardrails, we can't know if the model failed or if their classifiers blocked the capability." <- can we not assume that this also might be true for other vendors with less openness about it? I think it shows a general problem with closed systems and research. It is important then to understand what one measures. This lands on a measurement-side point worth adding: when the interventions (steering vectors, PEFT, classifiers) are invisible to the user, the one thing that stays observable and provider-independent is the output itself, specifically whether it remains structurally coherent. You still can't see why a capability dropped, but you can measure that the output's structural integrity degraded, continuously and independent of the provider's claims. It doesn't replace access to the non-sandbagged model, but it gives evaluators an honest, output-side signal that opacity can't reach. The "is it failing, or is it blocked?" question stays open; the "is the output degrading?" question becomes measurable. Keen to follow where the AI Evaluator Forum takes this. While it is heartening to see the evals community being thorough, if any company does engage in "sandbagging" why not use the model outputs as-is, to match what the end-users would experience? I mean not everyone should be able to make a bioweapon I agree there is likely a compromise where private but named actors can get access I think you are making an assumption that merits to be called-out - you are assuming that independent evaluators need to be able to assess models without the model provider being made aware of the evaluation. Under this assumption your concerns are spot-on. If you relax that assumption then the question becomes whether Anthropic allows for independent evaluation of its model’s capabilities. I believe we know the answer to be yes. If you want to evaluate Fable 5 frontier capabilities, I would encourage you to make a case to Anthropic. Even though frontier model building and eval are not the same job, you're raising a hard question. Definitely unsettling. They could have avoided them saying outright, "Hey. Our models have guardrails in place to avoid disseminating our own IP through it, these guardrails might need adjusting over time" and it would have created 1% of the noise you're hearing today A couple of thoughts... I don't like it either, but they are consistent. Anthropic is weighing in on the safety side, and specifically called out biology. For the record: they nerfed chemistry too. I've been vocal that this needs to get better, but rather get it right and using the time they need than have people getting a leg up on be able to wreck havoc. "We're actually degrading the model if we think it's being tested" could be the new "well actually, I wasn't trying anyway"...