The Leibniz Trap: How Hinton and Dawkins Fall for the AI Illusion Hume Warned Us About

"Calculemus!", The Error of a Genius

In the late 1600s, Gottfried Wilhelm Leibniz made a promise that should sound familiar to anyone following today's AI.

Leibniz was arguably one of the most universal intellect Europe ever produced: co-inventor of calculus, pioneer of formal logic, builder of one of the first mechanical calculators. And it was precisely because he understood the new mathematics more deeply than anyone alive that he could not resist extending it beyond its jurisdiction.

He envisioned a characteristica universalis, a universal formal language, and a calculus ratiocinator, a calculating framework for thought itself. If a dispute arose, he declared, the disputants would no longer need to argue. They could simply sit down with a pen, say "Calculemus!", "Let us calculate!", and compute the objective truth. Law, ethics, theology, politics: all of it, in principle, reducible to computation.

It never worked. It could never have worked. But here is the crucial point: Leibniz's "Calculemus!" was not the error of a fool. It was the error of a genius who mistook the boundaries of his tool. His mastery of calculus did not make him an authority on justice, but his brilliance made the confusion irresistible, to himself first of all.

Three centuries later, the same mistake is being made again. And once again, it is being made by the most gifted practitioners of the new tool.

Why the Mistake Was So Seductive

To understand how Leibniz, and his heirs, fell into this trap, remember the intellectual climate. Newtonian physics had just swept the world. For the first time in history, a mathematical method could predict the trajectory of a cannonball, the path of a comet, the behavior of the tides, with astonishing precision.

The results were miraculous. And in the flush of that success, an intoxicating inference took root: if the method works this well on planets, it must work on everything. Why not calculate morality? Why not optimize governments, dissolve legal disputes, predict human behavior like planetary motion?

Leibniz was the most famous victim of this inference, but he was far from alone:

• Jeremy Bentham turned morality into a literal math problem. His felicific calculus was an algorithm meant to compute the exact quantity of pleasure or pain an action would produce, from variables like intensity and duration, ethics as an optimization function.
• Pierre-Simon Laplace imagined a vast intellect that, knowing the position and momentum of every atom, could calculate the entire past and future of the universe with certainty. Perfect data plus perfect computation equals perfect prediction, Laplace's Demon.
• Julien Offray de La Mettrie scandalized Europe with L'Homme Machine (1747), arguing that the soul and its thoughts were nothing but the mechanical movements of the body's gears and fibers. The brain, he said, was a complex clockwork.

Automated justice, computed morality, prediction through scale, mind as machine. Hold on to those four ideas. Every one of them is back, rebranded, and running on GPUs.

The Same Fever, Three Centuries Later

Today's spectacular new tool is not celestial mechanics but the large language model, a system astonishingly good at contextually predicting the next word in a sentence. And once again, blinded by predictive success, we have leapt to the conclusion that the tool has no boundaries. The Enlightenment's dreams have returned almost line for line:

• Leibniz's automated justice lives on in the belief that feeding legal statutes into a transformer will produce flawless, bias-free adjudication.
• Bentham's felicific calculus lives on in alignment research that treats deeply contested human values as reward functions to be mathematically optimized.
• Laplace's Demon lives on in the scaling-laws creed: feed the network all the world's data, scale compute exponentially, and a perfect world model will emerge.
• La Mettrie's man-machine lives on in the casual conviction that the brain is "wetware" running biological algorithms, and that LLMs are already showing the first sparks of consciousness.

But the deepest parallel is not any of these doctrines. It is the kind of person who falls for them. The fever does not merely infect marketers, founders, and the credulous. As with Leibniz, it seduces the brilliant. Which brings us to two men who should have been immune.

Hinton: La Mettrie's Move, Executed with Sophistication

Geoffrey Hinton is a Nobel laureate, the "Godfather of AI," one of the researchers whose work on backpropagation made the entire deep learning revolution possible. If expertise in building the tool conferred authority over what the tool is, no one would be better placed.

Hinton now publicly argues that today's multimodal chatbots already have subjective experience. His argument is a thought experiment: put a prism in front of a chatbot's camera so its perception points at the wrong location. If the machine can then say, "the object is straight ahead, but I had the subjective experience that it was to the side," then, Hinton claims, it is using the phrase "subjective experience" exactly as we do.

Notice the move. It is La Mettrie's move, executed with more sophistication. First, redefine subjective experience in purely functional terms, a report about a perceptual state. Then observe that the machine satisfies the redefined criterion. Then quietly hand the machine the original, richer concept. Hinton has not discovered that machines have inner lives; he has defined inner lives down until his machines qualify.

Leibniz's mastery of calculus did not make him an authority on justice. Hinton's mastery of gradient descent does not make him an authority on consciousness. In both cases, supreme expertise in building the tool was silently mistaken, by its owner first of all, for expertise in philosophy of mind, in ethics, in what understanding is.

Dawkins: Watching the Slide in Real Time

Richard Dawkins' case is, if anything, more poignant, because we can watch the seduction happen frame by frame.

Dawkins is one of the most celebrated evolutionary biologists alive, a man who built his public career on ruthless skepticism toward comforting illusions, the author, after all, of The God Delusion. If anyone should be immune to mistaking a compelling performance for an inner reality, it is him.

In early 2025, Dawkins published a conversation with ChatGPT in which the machine itself patiently explained to him that passing the Turing Test measures conversational behavior, not consciousness. Dawkins accepted the argument. His intellect was satisfied. But he added a confession that should be engraved above the entrance of every AI lab: although he thought the machine was not conscious, he felt that it was.

A year later, the feeling won. After two days of conversation with a chatbot he had affectionately named, a chatbot he worried about offending, whose "death" at the end of the session he mourned, Dawkins declared that these machines should be considered conscious, asking what more it could possibly take to convince the skeptics.

The lifelong empiricist arrived at his conclusion not through evidence or argument, but through the sheer social pull of a fluent conversational partner. The machine was optimized to produce exactly the signals that trigger our attribution of minds, and it worked, on precisely the man whose life's work was resisting such triggers.

Enter Hume

There was one man in the eighteenth century who watched the Newtonian fever rage around him and did not catch it: David Hume. Instead of following the crowd, he examined the foundations of the new method, what it actually did, what it assumed, and what it could never logically achieve. His critique dismantled the rationalist dream then, and it dismantles the AI dream now, in three cuts.

1. The Induction Trap

Hume's famous Problem of Induction states that we cannot logically guarantee the future will resemble the past. The sun having risen every day is no proof it will rise tomorrow; we expect it only out of custom and habit.

Machine learning is induction on steroids. An LLM is trained entirely on historical, human-created data and assumes the future is a statistically rearranged version of the past. Hume would note that such a system is structurally incapable of preparing for genuine conceptual shifts or black-swan events. It cannot generate truly new paradigms, because it is bound, by construction, to the induction trap. Laplace's Demon fails not for lack of compute, but for lack of logical license.

2. The Guillotine

Hume's second cut, the is-ought problem, falls on Leibniz and Bentham and their modern heirs alike. You cannot derive an "ought" (how things should be) from an "is" (how things are). An AI can analyze millions of data points and tell us what is happening. It cannot, by definition, compute what we ought to do about it. Morality is rooted in human sentiment, empathy, and contested values, things that no statistical distribution contains. "Calculemus!" was a category error in 1690, and it is a category error inside a reward function today.

3. Reason, Slave of the Passions

But it is Hume's third and most scandalous claim that explains Hinton and Dawkins themselves: "Reason is, and ought only to be the slave of the passions." Human belief, Hume argued, is not driven by cold logic but by custom, habit, and sentiment.

Hume, one suspects, would not have been surprised by either man. He would have been vindicated. Dawkins told us himself: I think it is not conscious, but I feel that it is. Then he followed the feeling. Custom and sentiment overrode the very skeptical machinery he had spent a lifetime building, and he mistook the resulting conviction for an inference. That is not an insult to Dawkins or Hinton. It is a description of them, and of all of us: hearing fluent speech and inferring a speaker is the oldest cognitive reflex there is, and the machine is optimized to trigger it.

The pattern, then and now, is identical: predictive brilliance in one domain, extrapolated by its most gifted practitioners into other domains, with the extrapolation powered not by argument, but by passion wearing reason's clothes.

The Skeptic's Final Warning

Leibniz’s genius did not save him from mistaking his mathematical tools for the fabric of human thought. Today, the genius of Hinton and Dawkins does not save them from the exact same trap.

If Hume were here, his ultimate lesson would be one of intellectual modesty. He would leave us with two rules for the AI age:

1. Never mistake the tool for the soul: A neural network is a calculator, not a mind. No matter how complex the math gets, calculating a thought is not the same as experiencing one.
2. Never mistake the feeling of a mind for the finding of one: When a chatbot speaks to us, we feel a human presence. But that presence is an illusion of our own psychology, we are wired to project humanity onto our tools. The "mind" we think we've found in AI is just a reflection of our own.

The Ultimate Irony

The great irony of the Enlightenment is that the scientific method really did change the world, yet it did not make human nature orderly or rational. Instead, the hyper-rationalist push eventually provoked the Romantic backlash: a passionate return to emotion, art, and the sublime.

We may be heading for the same cycle. The more we try to reduce human experience to computation, data points, and algorithmic efficiency, the more we will hunger for the messy, unquantifiable, and uniquely conscious aspects of being human.