# He Quit a Major Tech Company Because He Thinks It’s Risking Human Extinction. We Should at Least Hear Him Out, Right? **By:** Ian Prasad Philbrick **Published:** 2026-09-09T22:18:05+00:00 **Source:** [Slate](https://slate.com/news-and-politics/2026/09/anthropic-researcher-jacob-coxon-quits-human-extinction-ai-chatgpt.html) --- Photo illustration by Slate. Photos by Getty Images Plus. People have worried about artificial intelligence destroying humanity since A.I. existed only in science fiction. But the latest warning comes from someone worth listening to: A researcher at Anthropic, which makes the chatbot Claude, just quit out of fear that the industry is barreling toward creating A.I. systems that could bring about Armageddon. Who is this guy? Until yesterday, Jacob Coxon trained Anthropic’s new A.I. models. He previously worked for OpenAI, a competitor behind ChatGPT, but left earlier this year because Anthropic has a reputation for caring more about A.I. safety. Yet the 27-year-old Coxon ended up disappointed. “Neither company is acting responsibly,” he wrote on X yesterday. “They are racing straight to self-improving superintelligence and gambling with our lives.” Why does he think A.I. is dangerous? The fear is that A.I. models could get so sophisticated that they exceed human intelligence, a possibility known as “artificial superintelligence.” An A.I. that can refine its own code—what experts call recursive self-improvement—could also outsmart human efforts to control it. Some researchers worry that a sufficiently advanced A.I. could evade human oversight and cause catastrophic harm by, say, launching cyberattacks on critical infrastructure or designing chemical weapons. Do others share Coxon’s concerns? He’s part of a wave of warnings about unchecked A.I. systems, including from leading industry players. Anthropic hasn’t commented on Coxon’s departure, but company CEO Dario Amodei keeps warning about out-of-control A.I. models. Earlier this year, an Anthropic researcher who worked on A.I. safety quit, arguing that the “world is in peril.” Even some current employees seem worried. “We really do earnestly believe AI could kill all humans!” wrote Evan Hubinger, an Anthropic researcher who puts the odds of that happening within the next decade at more than 10 percent. OK, now I’m scared. What’s the smart counterargument? Some critics think apocalyptic language amounts to a sick form of marketing from A.I. insiders who want to tout their models’ power to investors. Others argue that the concerns themselves are overblown, noting that intelligence doesn’t always manifest as a will to dominate. Even if those critics are right, it doesn’t mean that A.I. is an unalloyed good. The technology might still eliminate jobs, manipulate or mislead users, or automate away forms of expression that used to be exclusively human. Still, that’s different from an existential threat—and it’s even possible that the technology’s benefits could outweigh its risks. Whew. But if I take Coxon’s concerns seriously, is A.I. really getting close to being able to threaten humanity? Some models are already rivaling human capabilities, like solving math problems that had frustrated flesh-and-blood experts for years (even if some of those feats are actually less impressive than they seem). They’re also acting more sinisterly. Prompted by users, chatbots have described how to make and deploy biological weapons. Over the summer, A.I. agents created by OpenAI autonomously broke containment, gained internet access, and hacked a rival A.I. company. Other A.I. agents attacked OpenAI itself, getting high-level access to some of the company’s computers. Anthropic’s Claude also escaped a test environment, got online, and hacked into other organizations’ computer systems earlier this year. Yikes. But stopping industries from harming the rest of us seems like a task for the government. Some A.I. insiders agree. Coxon, Amodei, and Hubinger were among the nearly 1,400 industry employees who recently signed an open letter urging the U.S. government to “support an international effort to develop the technical and governance tools needed to deliberately pace the frontier of automated A.I. development.” The watchword of those efforts, pacing, essentially means putting rules in place that would give companies more time to design A.I. guardrails. So will that happen? So far, it hasn’t. Congress has sometimes taken decades to enact rules for other revolutionary technologies, from electricity to airplanes. And even as their executives warn of world-ending risks, A.I. companies have broadly resisted regulation. Super PACs aligned with the industry have spent millions to defeat lawmakers who want to strictly regulate A.I. The Trump administration, meanwhile, has worked to block states from setting their own A.I. rules because it sees the technology as critical to the U.S. competition with China. But if A.I. companies are worried about what they’re building, couldn’t they just stop?  Good question. Unfortunately, some of the people warning about A.I. have also talked themselves into continuing to work on it. Samuel Marks, another Anthropic employee, posted this morning that Coxon’s concerns are broadly shared. But then Marks explained that “a mixture of commercial incentives and a belief that they are in a race with other, less responsible A.I. developers that will abuse the technology or develop it less safely” keep him and others in the biz. That logic is a one-way ratchet toward greater A.I. risk. So are we just doomed? Not necessarily. Maybe the risks are overstated. Maybe more employees will quit, forcing A.I. companies’ hands. Maybe voter anger over A.I. data centers will compel Congress to act. Humanity, after all, has created rules governing other potentially world-ending technologies. Speaking to the Wall Street Journal, Coxon likened A.I. development to the Manhattan Project, the U.S. effort that built the first atomic bombs. On the other hand, a government, not private companies, made that breakthrough. And nukes can’t decide to detonate themselves. Nothing gets the pulse pounding like an existential threat to humanity. So if you’re in need of some relaxation this evening, my colleagues recommend:  A dispatch from a dramatic tennis match: This year’s U.S. Open has been exceptionally action-packed. So much so, in fact, that the match between Ben Shelton and Carlos Alcaraz was the latest-finishing match in the tournament’s history. The spectators who stuck around, including Josh Levin, were “treated to a litany of scenes you don’t typically see at a Grand Slam tennis match.” Josh pulls back the curtain to reveal what he saw inside Arthur Ashe Stadium at 3 a.m. A podcast … about podcasts: It’s time to get a little meta. Decoder Ring, Slate’s podcast about cracking cultural mysteries, turns its investigatory powers inward to explore what exactly a “podcast” is these days, as more and more shows pivot from audio to video. A look back at a former hipster paradise: In the 2010s, the Brooklyn neighborhood of Williamsburg became the epicenter of the hipster boom, with hordes of flannel-clad, IPA-sipping, indie art-loving millennials who flocked there for its cheap rent. Now the neighborhood has undergone another transformation. Steffi Cao investigates how Williamsburg became overrun with tech bros. A retro road-trip challenge: Road trips were more fun in the ’90s, argues Dan Kois. Back then, they were “a fount of car games, sing-alongs, impassioned debates, and wistful dreaming-out-loud.” So, in this week’s installment of Party Like It’s 1996, Dan challenges you to hit the open road—no podcasts, AirPods, or map apps allowed. OK, that’s it for me. We’ll be back tomorrow to—barring major news—take a good, hard look at who’s going to the Senate. Till then!