# Examining the Case for AGI **By:** Amarda Shehu **Published:** 2026-02-04T00:00:00+00:00 **Source:** [Substack](https://amardashehu.substack.com/p/examining-the-case-for-agi) --- ### A Methodological Analysis Perspective This comment article appeared in Nature a few days ago: _Does AI already have human-level intelligence? The evidence is clear. The vision of human-level machine intelligence laid out by Alan Turing in the 1950s is now a reality. Eyes unclouded by dread or hype will help us to prepare for what comes next. [https://www.nature.com/articles/d41586-026-00285-6#ref-CR4](https://www.nature.com/articles/d41586-026-00285-6#ref-CR4)_ In this substack essay, I wanted to offer a thorough analysis. Thanks for reading! Subscribe for free to receive new posts and support my work. When Nature publishes a piece declaring that “by any reasonable criteria” we have achieved human-level machine intelligence, one expects rigorous argument grounded in peer-reviewed evidence. What Chen et al. deliver instead is a masterclass in rhetorical substitution: posing difficult questions, then answering easier ones while hoping readers won’t notice the switch. Let me be clear about my position. I have spent two decades in this field, working both on foundational AI (even on evolutionary algorithms), classic and modern, and at the intersection with several disciplines, including biology, engineering, and natural language processing. I am neither a pessimist about AI capabilities nor dismissive of recent progress. But precision matters, especially when making extraordinary claims. And this article’s central thesis collapses under scrutiny, not because the authors are wrong about LLM capabilities, but because _they systematically confuse capability with the thing they claim to be measuring_. ## **The Turing Test Sleight of Hand** The article opens by invoking Turing’s 1950 paper, framing it as establishing a vision of “human-level machine intelligence.” This is the first substitution. Turing proposed an _imitation game_, a behavioral test for whether a machine could successfully mimic human conversation well enough to fool an interrogator. Fooling was the point! This is AI 101, literally the first week of any introductory AI course: Turing asked whether machines could _act_ like humans in linguistic exchanges, not whether they could _think_ like humans, and certainly not whether they possessed general intelligence. The authors acknowledge this obliquely (“the kind of flexible, general cognitive competence”), but then proceed as if passing the Turing test is equivalent to possessing AGI. It is not. A system optimized to produce human-like text in conversational settings is precisely that: a system optimized for a specific behavioral test. The authors would never accept this reasoning in reverse. If I built a system that fails the Turing test but proves novel theorems, would they declare that it lacks intelligence? The asymmetry reveals the confusion. ## **The Evidence Problem** Let us examine the empirical foundation. The authors cite GPT-4.5 being judged human 73% of the time in a Turing test. The citation? A 2025 arXiv preprint. Not peer-reviewed. They mention gold-medal performance at the International Mathematical Olympiad. Another arXiv preprint (2025). Not peer-reviewed. Collaboration with “leading mathematicians to prove theorems”? Again, arXiv (2025). Not peer-reviewed. Scientific hypothesis generation validated by experiments? A bioRxiv preprint (2025). Not peer-reviewed. I am sure you noticed the pattern. Of the thirteen substantive citations supporting their empirical claims, the overwhelming majority are unreviewed preprints, many from 2025, some from the very month of publication. This is not how one establishes extraordinary claims. Peer review exists precisely to catch the kinds of errors that riddle this literature. **“Proved theorems” and the Erdos fiasco:** Consider the mathematical olympiad claim. In my _opinion_, “gold medal” is impressive _and still not AGI._ Reuters reported that Google and OpenAI models achieved IMO gold-medal level performance (solving 5 of 6 problems), with OpenAI relying on heavily scaled test-time compute. That is indeed a milestone in machine problem-solving. It is also: - a very particular domain (contest math), - under a very particular interface, - with very particular engineering choices, - and not an argument that “human-level intelligence” broadly exists. You can build systems that are extraordinary at math and still brittle in mundane causal reasoning, planning, or scientific model-building. Domain excellence is not the same as general intelligence. And now consider the “Proved theorems” and the Erdős fiasco: the difference between finding and creating is the difference between scholarship and hype. The Comment leans on Bubeck et al. (2025) as evidence of scientific discovery and theorem-proving. That preprint is more careful than the Nature Comment’s gloss. It explicitly distinguishes: - rediscovering known results, - deep literature search, and - new scientific results obtained with AI, while also warning that the model can “confidently make mistakes,” that results can hinge on prompt details, and that reproducibility can be hard. And now the key point: public discourse around this work already illustrated the precise failure mode the Nature Comment ignores. In October 2025, social media posts claimed GPT‑5 “solved” multiple Erdős problems; this was walked back as essentially literature search: finding existing solutions that were “open” only in the sense that a website maintainer hadn’t linked them yet. The clarification is explicit in a post by Mark Sellke: “GPT‑5 did not solve those Erdős problems. It only ‘found’ solutions…” Ben Recht’s writeup makes the core point sharply: “open” meant “unknown to the maintainer,” not “unsolved by mathematics.” This does reveal a pattern: 1. a model performs high-quality retrieval/synthesis, 2. humans describe that as “solving,” 3. headlines metastasize, 4. later clarifications quietly concede the more modest truth. The Nature Comment does not protect readers against this confusion. It exploits it instead. ## **The World Model Confusion** The article then proceeds to address the objection that LLMs lack world models with a curious example: ask a cutting-edge LLM what differs between dropping a glass versus a pillow on a tile floor, and it “correctly predict\[s\] shattering in one case and not the other.” Therefore, the paper concludes, LLMs possess “functional models of physical principles.” This argument would not survive undergraduate scrutiny. The glass-shattering example is precisely the kind of common knowledge found extensively in training data. Literature, physics textbooks, safety warnings, Reddit threads about broken dishes; this correlation is ubiquitous in text. Retrieving and reproducing this pattern demonstrates nothing about whether the model has internalized a generalizable physical model. A genuine test would require the system to reason about novel physical scenarios not extensively documented in text. Can it predict the phase transition behavior of a material it has never encountered? The article provides no such evidence because, to my knowledge, none exists. It confuses pattern matching over a very large pattern space with understanding. This confusion extends to their treatment of counterfactual reasoning. Yes, LLMs can answer “what if” questions when those questions involve scenarios well-represented in training data. The hard problem is generalization beyond the training distribution. The authors repeatedly gesture at this challenge, then change the subject. The article points to autonomous driving models that learn predictive models of physical scenes. But notice the bait-and-switch: the objection was about LLMs; the supporting evidence becomes other neural systems in specialized domains. That doesn’t show that language models, as such, have the kind of grounded world models critics mean. It shows that _some_ neural nets can learn predictive dynamics in constrained settings, which no one is disputing. ## **The Definitional Shell Game** Perhaps most troubling is the article’s approach to definition. It correctly notes that “general intelligence” resists precise definition, but then exploits this vagueness systematically. It proposes that we should assess AGI by “triangulating” from paradigm cases, such as humans, hypothetical aliens, oracles. Reasonable enough. But then it selectively applies different standards depending on which conclusion they want to reach. Consider their handling of objections: **On sample efficiency**: “A chess master who takes ten years to achieve mastery is just as good a chess player as someone who reaches that level in one year.” True. But this deflects from the actual concern. LLMs require computational resources and data volumes that are not just quantitatively but _qualitatively_ different from human learning. A child learns what “cat” means from a handful of examples, then generalizes correctly to novel cats in novel contexts. Current LLMs require millions of examples and still fail on out-of-distribution cases. This is not about speed; it is about mechanism. **On hallucination**: “Humans are prone to false memories, cognitive biases and perceptual illusions.” Also true. But when humans make these errors, we can often trace them to identifiable cognitive mechanisms. When asked _why_ they believe something false, humans can (often) provide reasoning, acknowledge uncertainty, or revise beliefs when presented with evidence. LLMs confidently assert falsehoods with no access to their own reasoning process because there is no reasoning process to access, only statistical regularities. The article treats epistemically different phenomena as equivalent because both involve error. **On embodiment**: The authors argue that Stephen Hawking’s physical limitations did not diminish his intelligence, therefore embodiment is irrelevant to AGI. This is sophistry. Hawking had decades of embodied experience before his condition progressed. His conceptual understanding was built on a foundation of physical interaction with the world. The question is not whether intelligence requires _current_ embodiment, but whether it requires _ever having been_ embodied or having access to grounded sensorimotor experience. These are different questions. Throughout, the pattern repeats: identify a legitimate challenge to the AGI claim, acknowledge it briefly, then substitute a different, easier question and answer that instead. ## **What They Don’t Ask** The most revealing aspect of this article is what it never investigates: mechanism. How do LLMs produce their outputs? In this the article takes a strictly behaviorist stance: if the outputs look intelligent, intelligence is present. But this abandons a century of progress in cognitive science and philosophy of mind. When a human solves a novel mathematical problem, we can ask: What reasoning steps did you follow? What analogies did you draw? Where did you get stuck? What made you try a different approach? These questions are answerable because human problem-solving involves manipulable mental representations, conscious reasoning, metacognitive awareness. When an LLM produces a solution to a mathematical problem (setting aside data contamination), we cannot ask these questions in any meaningful sense. It has no access to its own processing. It cannot explain its reasoning because there is no reasoning in the human sense; there is a forward pass through a neural network, a probability distribution over tokens, a sampling procedure. And several prominent AI researchers (no, not a self-reference here; start with Melanie Mitchell) have shown that the “reasoning” can change any time. Oh, and that the answer given often has no correlation with the reasoning claimed it led to the answer. The authors might respond that human neural processes are also opaque at some level. True. But humans have conscious access to at least some of their cognitive processes. We can introspect, revise, explain. We do not pathologize skepticism. We keep asking until we have the answers. Perhaps more importantly: when we ask _why_ an LLM says what it says, the honest answer is “because that sequence of tokens had high probability given the training data and the prompt.” This is not explanation in any satisfying sense. It tells us nothing about whether the system has grasped concepts, built models, or understood structure versus memorized correlations at scale. The article never grapples with this: “We do not attempt to peer inside their heads to verify understanding—we infer it from behaviour, conversation and problem-solving.” But this is precisely what cognitive science _does_. We (ought to) combine behavioral observation with investigation of mechanism. We (ought to) ask not just _what_ a system does but _how_ and _why_. ## **The Rhetorical Architecture** Let me note the article’s rhetorical structure, which is carefully designed to make skepticism seem unreasonable. The recurring rhetorical pattern: - Ask a grand question (“human-level intelligence,” “AGI solved”). - Present evidence for a narrower claim (“impressive benchmark performance,” “human-like dialogue,” “useful assistance”). - Conclude the grand claim anyway. Let’s expand this pattern a bit further: - **Define the target vaguely**: “General intelligence” is said to resist sharp definition, giving the authors flexibility to adjust criteria as needed. - **Set up strawmen**: They list ten objections, most of which would give pause to serious researchers to actually make in the form presented. For instance, few claim that AGI requires perfection or superintelligence. The actual objections are more subtle. - **Answer different questions**: When addressing the “stochastic parrot” objection, they cite LLMs solving “new, unpublished math problems.” This shifts from the question of whether LLMs merely interpolate training data (they do, with some extrapolation within the learned manifold) to whether they can handle problems not literally in the training set. These are different claims. - **Invoke authority**: Turing is invoked repeatedly, as if his 75-year-old thought experiment settles 21st-century questions about machine cognition. It does not. With all due respect to Sir Turing. - **Demand consistency**: They note, correctly, that many objections would exclude some humans. But this cuts both ways. If we define AGI as “anything a human can do,” then by construction, average humans have AGI. The term becomes vacuous. - **Frame skepticism as fear**: The article ends by referencing Turing’s “heads in the sand” objection, implying that denial of AGI stems from emotional dread rather than legitimate scientific disagreement. This is a failure of charity. ## **What Is Actually True** None of this is to deny that LLMs are remarkable. They are. They represent genuine scientific progress. They can be enormously useful tools. And if I broaden them to foundation models, they are allowing my lab to do some pretty cool things in scientific discovery across disciplines. But usefulness is not understanding. Capability is not intelligence. And passing behavioral tests is not the same as possessing the cognitive architecture those tests were originally designed to probe. The article asks us to accept that LLMs have general intelligence because they perform well on a range of tasks. But performance on tasks, even many tasks, does not necessarily indicate a unified underlying intelligence. It might indicate a very large lookup table with sophisticated interpolation. It might indicate that language, as it appears in human training data, contains enough statistical structure that a sufficiently large model can mimic many intelligent behaviors without possessing the thing that generates those behaviors in humans. We do not yet know. And that is fine. Science advances through uncertainty acknowledged and investigated, not through premature declarations. But we also do not stop there. We do not stop asking questions. Humanity is built around the why and how. ## **On Standards of Evidence** Another quote: “just five years ago, we didn’t have AGI; now we do.” This is not a scientific conclusion; it is a rhetorical flourish masquerading as one. Five years ago, we had GPT-3. Now we have GPT-4.5 (per their citation; though we have skipped ahead to 5.2 now) and similar models. The difference is quantitative: more parameters, more data, more compute, better performance on benchmarks. The difference is not obviously qualitative in the sense of crossing some threshold from “not intelligent” to “intelligent.” If the claim is that we have crossed such a threshold, the article needs to specify what the threshold is, provide evidence that we have crossed it, and explain why this threshold matters. It does none of these things. Instead, the assertion is made that the evidence is “clear,” and that alternative explanations are “increasingly disconfirmed.” But the alternative explanations (that LLMs are extremely sophisticated pattern matchers, that they leverage massive memorization with some generalization within a learned distribution, that they lack genuine understanding even as they produce impressive outputs) have not been disconfirmed at all. Research is highly active on this. The article simply declares an entire active body of ongoing research disconfirmed without engaging with the actual arguments. ## **A Note on Peer Review** I want to return to the evidence base because it matters. This article appears in one of the world’s most prestigious scientific journals. When Nature publishes a claim, it carries weight. Researchers cite it, journalists report it, policymakers reference it. And yet the empirical foundation consists largely of unreviewed preprints, several of which have been challenged or partially retracted. This should give us pause. The claim that “current AI already has human-level intelligence” is extraordinary. It requires extraordinary evidence. Preprints are not that evidence. I do not say this to dismiss arXiv or bioRxiv. I use both platforms extensively. Preprints serve a vital role in rapid dissemination. But they are not peer-reviewed, and for good reason. Errors get caught. Methodologies get scrutinized. Claims get tempered. The mathematical olympiad paper cited here is a perfect case study: it made strong claims based on flawed methodology, and only post-publication scrutiny revealed the problems. The authors of this Nature piece owed their readers, and the field, better diligence. ## **What This Reveals** This article reflects a tension in how we think about intelligence. One view, which I find more scientifically productive, is that intelligence is a mechanistic property: it involves certain kinds of information processing, representation building, reasoning, and understanding. Another view, represented here, is that intelligence is purely behavioral: if it acts intelligent, it is intelligent. The behaviorist view has appeal. It avoids murky questions about consciousness, qualia, and “true understanding.” It makes assessment straightforward: run tests, measure performance, done. But it also abandons much of what makes intelligence interesting and important. It reduces a deep question about cognition to a shallow question about input-output mappings. The authors write that recognizing LLMs as AGI is “a wake-up call.” I agree, but not in the way they intend. It is a wake-up call about the importance of conceptual clarity, methodological rigor, and intellectual honesty in public discourse about AI. We are at a genuinely important moment in the history of technology. LLMs and related systems will reshape aspects of work, education, research, and society. How we think about these systems, what they are, what they can do, what they cannot do, what risks they pose, and what benefits they offer matters. Prematurely declaring victory on AGI does not help this process. It obscures rather than clarifies. It encourages sloppy thinking about what these systems are and how they work. And it sets up unrealistic expectations that may disappoint. ## **Conclusion** Here we have a piece that sounds authoritative but rests on conceptual confusion and empirical overreach. It invokes Turing while misunderstanding his test. It cites unreviewed preprints as if they were established fact. It equivocates between different senses of “intelligence” depending on argumentative convenience. It dismisses legitimate objections through substitution and misdirection. Most fundamentally, it never asks the question that matters: not whether LLMs can produce intelligent-seeming outputs (they can), but whether they possess the kind of understanding, reasoning, and cognitive architecture that characterizes human intelligence. This question is declared answered by fiat, pointing to behavioral performance while studiously avoiding any investigation of mechanism. This is not how science works. We do not settle deep questions about the nature of mind by running benchmarks and declaring victory. We investigate carefully. We distinguish correlation from causation, capability from understanding, performance from comprehension. We demand evidence proportionate to our claims. We turn things over and over and over. The evidence for AGI, despite the authors’ confidence, is not there. Not yet. Perhaps one day it will be. When that day comes, I hope we will recognize it not through rhetorical flourish but through careful, rigorous, peer-reviewed science that takes seriously both what these systems can do and what they cannot, both their behavior and their mechanisms, both their capabilities and their limitations. Until then, we should resist the siren song of premature declaration. The field deserves better. The public deserves better. And the systems themselves, remarkable as they are, deserve to be understood for what they actually are rather than what we might wish them to be. Thanks for reading! Subscribe for free to receive new posts and support my work. This is an important piece - it feels like major news publications want to cash in on these breathless headlines but most of the industry understands AGI is the wrong question. Let's see these systems work more and more effectively in the real world and bring really added value and then we can have the debate about whether they are truly intelligent. Anyone who has worked extensively with LLMs knows how ridiculous it is to claim they have "human" level intelligence. Nope. Not yet. Interesting! Your points are spot on. It sounds great until one digs deeper.