A Brief History of Intelligence by Max Bennett

The author starts the book by talking about how life emerged on Earth; the primordial soup that gathered around hydro-thermal vents, the emergence of DNA, the creation of lipid membranes, the creation of proteins, and so on – I have already covered this in detail in my Field Note for David Sinclair’s Lifespan along with how the primitive gene circuit works to repair DNA damage.

And as I noted in my Field Note for Lifespan, while the initial emergence of these simple biological machines was random, once they had emerged, evolution could take over. The machines that had superior circuits (or intelligence if you will) for harnessing energy from their surroundings to repair and/or replicate would be rewarded by evolution.

This and other forms of molecular machinery – sometimes simpler, sometimes more complex – can, according to Bennett, be considered a “… primitive version of intelligence …”.

But it wasn’t true intelligence and there was still a long way to go from there to the modern humans of today.

Bennett briefly talks about LUCA (our “last universal common ancestor“) whose blueprint to life we all share – DNA, protein synthesis, lipids and carbs – from bacteria to a human.

Following LUCA’s emergence, life was simple, literally – the only living beings around were simple bacteria.

These ancient bacteria used Hydrogen as fuel, they found it readily available from volcanic activity and chemical reactions deep within the Earth’s crust, particularly around those same hydrothermal vents I mentioned earlier. They then used this hydrogen to generate their own energy, allowing them to grow and replicate. But this process was inefficient.

But after some time (somewhere around a billion years) Cyanobacteria entered the scene and brought along with them the game changing technology called Photosynthesis – it was more efficient and allowed cyanobacteria to literally eat light – they used sunlight and CO₂ to make sugar that could finance their energy needs.

But photosynthesis releases oxygen, and during that time there weren’t any life forms that could use the oxygen – so it just kept piling up in the Earth’s atmosphere (often called the “Great Oxygenation Event“) – this was bad news for life back then.

Because oxygen is very reactive and can interfere with cellular processes. So you do not want to be putting too much of it inside your body – but because our pre-historic photosynthetic life forms pumped the Earth full of it – they couldn’t help but breathe it in.

These early life forms, as the author puts it, “… became victims of their own success, slowly suffocating in a cloud of their own waste.“.

And that would have been the end of life on Earth had it not been for another bacterium that emerged on the scene – The Respirator. These guys had the ability to use oxygen (along with sugar) to generate energy.

And what did they excrete? CO₂ – the very thing Photosynthesisers needed! What luck!

And so a great symbiotic partnership was formed between the Respirators and the Photosynthesisers – the waste of one was food for the other.

But there was a wrinkle to this partnership – unlike the Photosynthesisers, Respirators needed to move around. And there is a simple explanation.

You see, the Photosynthesisers, they used sunlight and CO₂. These were abundant and always available from the environment. They just had to sit there and eat. Their energy source was external and virtually limitless, as long as the sun shone and CO₂ was present. They were producers.

Now, consider the “Respirators”:

1. Their “food” was sugar (produced by Photosynthesisers) and oxygen (released by Photosynthesisers).
2. Sugar isn’t omnipresent like sunlight. It’s locked inside other organisms (the Photosynthesisers, or the remnants of dead ones).
3. Therefore, to get their energy, Respirators couldn’t just sit and wait. They had to actively find and consume these sugar-containing organisms or their remains – they needed to move, they needed to hunt.
4. This act of finding and consuming other living (or recently living) things for energy is, at its most basic level, hunting or predation.

As Bennett points out, “… respiratory life could survive only by stealing the energetic prize—the sugary innards—of photosynthetic life.“.

An evolutionary arms race began with respiratory life forms trying to survive by eating photosynthetic (and other respiratory) life forms, and photosynthetic (and other respiratory) life forms trying to survive by not getting eaten by respiratory life forms.

This led to a dramatic diversification of life on Earth. More sophisticated organisms, known as Eukaryotes, evolved – initially as single cells, then progressing to small multicellular forms, and eventually to the large, complex organisms we see today.

For these large multicellular eukaryotic life forms to function and propagate there was a need for a system wide response to environmental triggers.

And this is where enters the pre-condition for intelligence – The Neuron.

What first began as a need for single-celled respirators to move around, and then led to increasingly complex life forms to dominate the competitive landscape, eventually led to a foundational unit to coordinate actions across complex multi-cellular life form.

Intelligence exists because of the neuron, and the neuron exists because we needed to move around.

How neurons work

While neurons come in many shapes and sizes, and perform a vast array of specialised functions – from light-sensing neurons in your eyes to pressure-sensing neurons in your skin, or those that control muscle movement – their fundamental method of communication, based on electrochemical signals, has remained remarkably consistent since their early evolution.

• Neurons are designed to send and receive signals in the form of electricity or chemicals. Signals travel within a neuron as electricity and between neurons as chemicals.
• Unlike many other body cells which might be roughly spherical, neurons (or nerve cells) have a distinct, highly specialised shape with extended parts. Think of them as having a ‘sending’ end and a ‘receiving’ end, which gives them a specific direction for signal flow.

• Electricity flows through neurons unidirectionally. Signals are typically received by branch-like structures called dendrites, which then transmit the electrical impulse through the cell body and down a long, cable-like extension called the axon. The axon then sends this signal onward to other neurons or cells.
• The point where one neuron communicates with another isn’t a direct electrical connection. Instead, there’s a tiny gap called a synapse.

• When an electrical signal (an action potential) reaches the end of an axon, it triggers the release of chemical messengers called neurotransmitters into this gap. The neurotransmitters then cross the gap and bind to specific receptors on the receiving neuron’s dendrites or cell body, influencing whether that neuron will fire its own signal. This chemical step allows for complex modulation and processing of signals.
• A single neuron does not send signals in gradients – either it fires a signal or it keeps quiet. That is, neurons operate as a transistor switch – they are either on (transmitting signals) or off (not transmitting).

• The rate at which neurons send signals depends on the magnitude of the trigger they are designed to detect – the more the magnitude of the trigger, the higher the rate at which the neuron will fire. This is known as “rate coding”. A neuron can typically fire at a maximum of about 500 “spikes’ per second.

• The rate at which a neuron fires depends not only on the magnitude of the trigger, but also how long the neuron has been exposed to it – if the exposure is long, then the firing rate will decrease as the neuron baselines to the situation, known as “adaptation“. This allows neurons to effectively process information across a much wider dynamic range of stimuli, preventing them from always firing at their maximum rate when a stimulus is strong but constant.

• Neurons can either nudge action (“excitatory”) or suppress action (“inhibitory”). Excitatory neurons (and the neurotransmitters they release, like glutamate) make the receiving neuron more likely to fire an action potential. They “nudge” or push it towards its firing threshold. Inhibitory neurons (and their neurotransmitters, like GABA) make the receiving neuron less likely to fire an action potential. They “suppress” or pull it away from its firing threshold.

How does one hunt?

Either one can sit around and wait – and attack when an unsuspecting prey wanders into your trap (ambush predation). Or they can go out and catch their prey (pursuit predation).

Both forms of hunting are available to see in the world: coral polyps typically remain stationary in wait and then grab any piece of food that floats by with lightning speed, barracudas will use a mix of ambush and pursuit, while sharks will actively hunt their food.

Ambush predation needs the organism to move in place, while pursuit predation needs it to move around – and although both require a coordinated response by the organism (hence, the need for neural circuits), the latter is more computationally demanding.

Because moving around randomly is an express ticket to extinction – organisms needed to figure out a way to move intelligently. In other words, they need to learn how to steer – this is Bennett’s first breakthrough in the history of intelligence.

This need to steer contributed to the emergence of bilateral body (two halves) shapes instead of radial (circular) because the former needs a simpler system – a way to move forward, a way to turn, and a way to tell when to move and when to turn.

And it was this same need to steer that led to the creation of valence – i.e. a sense of “good” and “bad”. To steer effectively, an organism needs to know what to steer towards (food or safety) and what to steer away from (predators or toxins)¹.

Bennett points out how the development of “adaptation”, i.e., the ability of neurons to reduce their firing rate (baseline) to a constant stimulus intensity, was a necessary precursor to steering because the organism figured out the next move basis the relative concentration of stimuli, it moved in the direction where it found the relative concentration higher. As I said, subsequent breakthroughs built on ones that came before.

Anyway, merely having the ability to imbue stimuli with valence (that is, label something as “good” or “bad”) is not enough because in the real world there can be thousands of stimuli hitting sensory neurons at the same time – the organism needs a way to weigh the different streams of stimuli against each other and then decide what to finally do – a sort of headquarters if you will – in other words, a brain.

This brain was much simpler than the brains we see today, just a big circuit of neurons that could “decide” what to do. As Bennett points out, “The first brain was this mega-integration center—one big neural circuit in which steering directions were selected.“.

Inhibitory neurons (ones that inhibit action) helped this “proto-brain” by ensuring only one action would be done at a time.

For most things the organism also needs to know about its internal state before making a decision.

For example, a starving organism may decide it is worth moving into a toxic environment because of the strong smell of food coming from there, whereas a well-fed organism may pass on the gamble.

This meant that the proto-brain also needed a way to know the state of its body – this was done through internal signalling methods such as hormones that sensory neurons could detect.

Additionally, while valence provided a fundamental sense of ‘good’ or ‘bad’ (a spectrum from beneficial to harmful), proto-brains also evolved “arousal“.

Arousal provided the crucial dimension of intensity or urgency for these valenced stimuli.

A high arousal signal, for instance, might mean ‘this is very good and urgent to pursue’ (like a strong food smell for a starving organism) or ‘this is very bad and urgent to avoid’ (like the imminent threat of a predator).

Together, valence and arousal determine an organism’s “affective state” – grossly oversimplified this means how the organism is feeling.

Valence describes how good or bad it is feeling (about itself or something that it perceives in its environment). And arousal defines whether it wants to do something about it or not.

This was the beginning of emotion.

Emotions bring us perfectly to the next step in the evolutionary journey – Bennett points out that the “… defining feature of these affective states is that, although often triggered by external stimuli, they persist for long after the stimuli are gone.“.

This is not a “breakthrough” as classified by Bennett in the book, he still classifies it under “steering” – but to me it’s as big.

As you should know, efforts rarely yield rewards immediately and consistently, and often you just have to be persistent.

Whether it’s the mate you are trying to attract or the animal you are trying to hunt – persistence sometimes accords you an evolutionary advantage.

It is key neuromodulators, specifically dopamine and serotonin, that play a crucial role in determining whether we persist at something or not.

Dopamine is released in anticipation of a reward, such as food. And the amount released depends on the internal state of the animal, such as hunger versus satiety.

The “reward” here may not just be a reward in the traditional sense, it can also be “relief” – such as escaping from a lion – even in these cases dopamine is released.

When an action leads to a reward for the first time, dopamine is released shortly after the reward. But over time, it starts getting released before the reward, that is, when the organism receives cues that make it anticipate the reward.

If the reward then occurs as predicted by the cue, there’s often no change in dopamine because it’s already been “accounted for” by the cue earlier.

But if the predicted reward fails to appear after the cue, dopamine activity actually dips below baseline, signalling a “negative prediction error” – the reward was worse than expected.

Conversely, if the reward is better than predicted, there’s a surge in dopamine, signalling a “positive prediction error” – the reward was better than expected.

In this way, proto-brains learned to associate cues with rewards, as well as modify their learning in cases when the cue did not lead to the reward.

This ability, known as “associative learning” is unique to bilateral organisms – as Bennett points out, “Once animals began approaching specific things and avoiding others, the ability to tweak what was considered good and bad became a matter of life and death.“.

This was the dawn of learning.

We can see here how intelligence is transforming – from real time responses to the environment, it now was becoming able to predict and anticipate the future.

Let’s talk about the “credit assignment problem“.

Just like the case with real time responses to stimuli, where the organism needs to decide which of the numerous stimuli to finally act on – In the case of associative learning as well, the organism needs to decide which of the numerous stimuli to associate with the reward.

Incorrect association will lead to wasted energy for the organism and may even be fatal.

And the common-sense way to solve the problem is: The organism should associate a stimulus with the reward if …

1. … both the stimulus and the reward occur very close to each other (a.k.a. “eligibility traces“).
2. … the stimulus should be something new that has not happened before (a.k.a. “latent inhibition“).
3. … the stimulus is stronger than other stimuli (i.e. “overshadowing“)

And to keep things simple, proto-brains will also practice “blocking“, which means once it has associated a stimulus with a reward it will not change the association very easily and associate other stimuli with the reward.

Now, a learning proto-brain will not be very successful if it did not have a place to store data (and overwrite it if needed).

This is where evolution did something very cool – unlike our computers which have a separate part for computation (CPU) and a separate part for data storage (hard drive) – in brains the computation and the storage is enabled by the same foundational unit (the neuron).

Neurons not only power the operating circuit orchestrating the organism’s response to stimuli, they also serve as the foundation for memory. Your memory is the configuration and strength of neuronal connections in your brain.

The father of Hebbian theory – Donald Hebb’s famous quote, “Neurons that fire together wire together” encapsulates this.

Memories are encoded in the patterns of activity and the enduring changes in the strength and number of connections (synapses) between neurons.

A brain “learns” when neurons connect with each other – this connection happens when two neurons fire at the same time².

Two neurons fire at the same time when incoming stimuli make them fire at the same time. For example, if neuron A (detecting the sight of a tiger) fires, and simultaneously neuron B (detecting the sound “growl”) fires.

Then, as Bennett simplifies there is “… clever protein machinery …” in the brain that connects those two neurons together (or if already connected, strengthens the connection between them).

So, the next time the brain hears a growl, the “growl” neuron might be able to activate the “sight” neuron more easily, even without actually seeing the tiger and make the organism ready to take flight.

This strengthening (and weakening) of connections between neurons is, at the mechanistic level, what we call memory/learning³.

Real life is more than just a simple “cause – effect” maximisation game. Action does not immediately lead to reward, and rewards are not just the consequence of the immediately preceding action.

IRL, organisms need to intelligently perform a sequence of steps to arrive at the optimal outcome.

This means they must be smart enough to know what action, even though it leads to short term gain, is bad for the long term (for instance, a morsel of food that just happened to be lying around might have been bait).

When it comes to rewards that most matter to an organism (such as an abundant food supply, mates, shelter) – it is better to plan. And while lucking upon those rewards is totally possible, organisms over-relying on purely opportunistic approaches (i.e. the associative learning MVP I described above) would have associated the most immediate action with the reward without consideration for the fact that the prior steps were all equally important. And in this way, versus organisms that could plan, have been at an evolutionary disadvantage because their long-term survival would have relied too much on luck.

To meet the challenge of a world where the best rewards were likely behind several carefully planned moves – intelligence developed its second breakthrough: reinforcement learning.

Associative learning (AL) is about learning that two things go together. It’s about forming connections or associations between stimuli, or between a behaviour and its consequence.

Reinforcement learning (RL) is where an organism learns how to behave in a dynamic environment to maximise cumulative reward over time. It’s about learning a long-term strategy.

RL is more challenging than AL because:

1. Moving in the dark: The organism needs to make the right moves that, right now, may not even seem right (e.g., a chess game: a move might not immediately win, but sets up a future checkmate).
2. Assigning credit across time: And when the organism does clearly make a right move (one that immediately triggers AL), it needs to assign credit not just to that move, but to all the preceding moves. This is known as the “temporal credit assignment problem“.

What do you do here? A situation where you are “moving in the dark”, feedback is not powerful enough to trigger AL, and the reward is a long way away.

Trial and error.

And that is what evolution did: instead of releasing dopamine only when the organism achieves the final reward, RL releases dopamine when the organism thinks it is getting closer to achieving the final reward.

And when the organism thinks it made a bad move? The opposite: dopamine dips below baseline.

Instead of a burst of dopamine (positive prediction error), there’s a suppression or dip in dopamine neuron firing below the normal, baseline level.

This dip signals that the outcome was worse than predicted or that a valuable opportunity was missed. It’s a “bad news” signal from the brain’s prediction system.

With this approach, RL can “… reinforce some moves and punish others throughout a long game … whether or not [the organism] won or lost the overall game.” – Bennett calls this the “grand repurposing of dopamine”.

Let’s break that down:

• Immediate Reward: What you received from the state you entered.
• Discounted Value of Next State: Your current best prediction of all future rewards you’ll get from the state you just entered (discounted because a future reward is usually valued less than an immediate one).
• Current Value of Current State: What you expected to get from this state.

• Positive TD Error (Reward Better Than Expected / Getting Closer): When the (Immediate Reward + Discounted Value of Next State) is greater than what was expected from the current state, dopamine neurons show a burst of activity. This reinforces the actions that led to this “better-than-expected” situation, even if the final reward is still far off.
• Zero TD Error (Reward As Expected): If the prediction matches the reality (Immediate Reward + Discounted Value of Next State) exactly equals the current state’s value, dopamine activity returns to baseline. The brain has nothing new to learn.
• Negative TD Error (Reward Worse Than Expected / Moving Away): If the (Immediate Reward + Discounted Value of Next State) is less than what was expected, dopamine activity dips below baseline. This signals a “negative prediction error” and “punishes” the recent actions, teaching the organism to avoid them.

Bennett explains this using Sutton’s “Actor – Critic” model: the “Actor” is the part of the organism’s neural circuitry that takes actions, and the “Critic” is the part of the organism’s neural circuity that evaluates the actor’s decisions.

The Critic is always trying to make TD Error Zero (i.e. have perfect predicting power). While the Actor is always trying to surprise the Critic positively.

The Critic’s primary goal is to learn the most accurate value function possible. When the TD error is zero, the Critic’s learning has converged; it has achieved perfect predictive power based on the current policy.

When the Actor takes an action that leads to a positive TD Error (meaning the outcome was better than the Critic predicted from the previous state), the Actor’s action is reinforced.

This “positive surprise” for the Critic means the Actor found a path to a more valuable state than initially expected.

So, the Actor is constantly experimenting (exploring) and refining its actions to generate these positive TD errors, which are its “rewards” for effective behavior. It’s essentially “surprising the Critic” by making better moves than the Critic initially accounted for.

The evolution of a precise internal clock (time perception) was another foundational breakthrough because it enabled the brain to:

• Generate accurate temporal predictions.
• Produce precise disappointment (negative TD error) when something expected (good or bad) fails to occur at the predicted time.
• Produce precise relief (positive TD error) when something badly expected fails to occur at the predicted time.

Bennett argues that while basic life forms have rudimentary ways of tracking time (like circadian rhythms), the ability of vertebrates (and some advanced invertebrates, though he highlights the gap) to precisely measure time intervals is a game-changer for intelligence.

This precision in temporal prediction error is what allows for sophisticated reinforcement learning in environments where rewards and punishments are not immediate or are precisely timed. It moves learning beyond simple contiguity to complex temporal relationships.

Imagine, You receive a cue (e.g., a light) that reliably predicts a painful zap will occur in exactly five seconds.

If you don’t have an internal clock, you’d know “light = zap sometimes.” You might cower for a long time, because you don’t know when the danger has passed. With an inner clock, you can predict: “Zap at precisely T+5 seconds.”

If T+5 seconds passes and no zap occurs, your precise internal clock signals a negative prediction error (relief). This teaches you: “After T+5 seconds, the danger is over.”

After establishing the foundational “proto-brain” and its ability to learn from consequences, intelligence took its next leap: pattern recognition.

This wasn’t merely about seeing; it was about identifying specific configurations of information, whether a sight, a sound, or a smell.

A pattern-recognising brain had to solve two critical problems simultaneously: discrimination – telling apart subtly similar but distinct patterns (like the rustle of leaves versus the sound of a predator); and generalisation – recognising the same pattern even when it appeared in varied forms (like a tiger seen from different angles or in different light).

The brain solved these challenges through an intricate architecture, leveraging specialised neurons in the cortex that create unique, high-dimensional representations for distinct inputs, while also forming recurrent connections that allow a pattern to be recognised even from partial information.

This biological solution to pattern recognition demonstrates robustness and efficiency.

Unlike many of today’s powerful artificial neural networks, which often suffer from catastrophic forgetting (erasing old knowledge when learning new things) and struggle with invariance (recognising patterns despite basic transformations like rotation or scale without extensive retraining), even a fish’s brain handles these seamlessly.

In certain aspects, this highlights the superiority of biological brain design (for now) – allowing organisms to continuously integrate new information and identify threats and opportunities in an ever-changing world.

For a deeper look into how exactly the brain evolved these structures for robust pattern recognition, I highly recommend diving into Bennett’s book.

Another significant leap for intelligence was the emergence of curiosity (which also happens to be a Sunchaser Value).

Curiosity was the response that intelligence gave to life’s continual challenge: the “exploitation-exploration dilemma“.

Basically, it is the decision we need to take from moment to moment whether we should continue with an activity with known rewards (“exploitation”) or seek out newer activities which may give us a better reward but may also be dead ends.

By making curiosity intrinsically rewarding, intelligence ensured that organisms sought novelty without immediate survival need.

Let’s call this “curiosity module” in these proto-brains, the “Child“.

Guided by the Child, organisms sought to understand the world for its own sake. This was different from why the Critic (in the Actor-Critic Model) wanted to understand the world.

Both the Child and the Critic want to acquire superior understand how the world works, but the Critic wants to do that to maximise an end goal, while for the Child, a superior understanding of the world is its own reward.

In other words, like food and sex (which happen to be the end goals of the Critic), curiosity is intrinsically motivating.

Preparation for the Unknown (Proactive Adaptation):

The world is constantly changing and unpredictable. Resources can deplete, new predators can emerge, and environments can shift.

An organism that only exploits known resources is vulnerable when those resources disappear or when novel threats appear.

Curiosity drives exploration of the unknown environment before a direct survival need arises. This “preparatory learning” allows an organism to gather information about potential food sources, new hiding spots, the layout of a territory, or the behaviours of other species (both prey and predator) in advance. This stored information provides a future adaptive advantage.

Increased Learning Efficiency:

Curiosity, by making novelty or reducing uncertainty rewarding, optimizes the learning process itself.

Instead of random, inefficient exploration, curious organisms (or AI agents) are motivated to seek out areas where their internal “model of the world” is incomplete or inaccurate. They are driven to resolve prediction errors about the environment’s dynamics, not just reward prediction errors.

This more intelligent exploration leads to faster, more comprehensive learning about the environment, which indirectly translates to better resource acquisition, better threat avoidance, and ultimately, better survival and reproduction.

Survival in Resource-Scarce or Complex Environments:

In environments where immediate rewards are sparse or delayed, pure external reinforcement learning might be too slow. Curiosity provides an “internal compass” that drives learning even when extrinsic rewards are absent.

This allows organisms to build complex internal models of their world, which are invaluable for long-term planning and decision-making in challenging conditions.

The next big leap for intelligence was the ability to create internal maps – organisms were now able to remember where they were in space.

Bennett says the “… evolution of spatial maps in the minds of early vertebrates marked numerous firsts.“.

Organisms could now understand:

• Locations: Where specific objects, resources (food, water, shelter), or dangers are.
• Relationships: The distances and directions between these locations.
• Routes: Possible pathways to get from one location to another.

This “map” allows for flexible navigation, meaning an animal can:

• Find its way to a goal even if its usual path is blocked.
• Take shortcuts.
• Return to a specific location from various starting points.

• Before an AI can build a map, it needs to accurately perceive its environment. Unlike a digital game, the real world is chaotic: lighting changes, objects move, sensors have noise, perspectives shift.
• Humans and animals can walk into a completely new room and almost instantly build a mental map, identifying key features and their relationships. They update this map on the fly as they move. AI struggles with Simultaneous Localisation and Mapping (SLAM).
• Biological brains don’t just store pixel maps; they create abstract, flexible representations of space. They understand concepts like “next to,” “behind,” “between,” “shortcut,” and “this leads to that.” This is more than just coordinates. AI models often struggle to create these truly semantic and relational maps.
• Biological organisms learn incredibly sophisticated spatial maps from relatively sparse and noisy sensory input, and with little explicit “training.” A child explores a new house once and largely understands its layout. Current AI, particularly deep learning, still needs vast, curated data to build even basic representations.
• The map isn’t just for knowing where things are; it’s for knowing what to do. For example, getting an AI to understand that a “door” leads to a “new room” and that “new room” has “more food” is far more complex than simply recognising the pixels of the door.

In essence, while AI has made incredible strides in specific aspects of vision or pathfinding, the seamless, robust, and adaptive spatial mapping capabilities of a vertebrate brain – its ability to build and use flexible internal representations of an ever-changing 3D world from imperfect sensory data to guide complex behaviours – remain a “grand challenge” for artificial intelligence.

Following the initial breakthroughs of basic reinforcement learning, the journey of intelligence faced new environmental pressures.

A critical turning point came with the Permian-Triassic extinction, a devastating event that reshaped life on Earth.

While many perished, a lineage of warm-blooded, mammal-like reptiles called therapsids survived, though greatly reduced in size.

Their high metabolic rate (a side effect of warm-bloodedness) allowed their brains to operate much faster than their cold-blooded counterparts, enabling more complex computations.

For hundreds of millions of years, despite an explosive diversification of animal forms, the fundamental architecture of the brain for basic learning remained remarkably conserved across most vertebrates.

Evolution seemingly “settled” for a brain primarily focused on learning by doing.

However, the small, secretive lifestyle of these early mammals – hiding in burrows or trees – presented a unique evolutionary advantage: they consistently had the opportunity to make the “first move” from a concealed position, observing their surroundings before acting.

This ecological niche placed immense pressure on their brains to exploit this “first move” advantage.

Eventually, a neural innovation emerged: the neocortex.

This new region of the brain, with its distinct and powerful circuitry, bestowed upon these small mammals a transformative superpower: the ability to imagine actions and outcomes before they occurred. They could mentally rehearse different scenarios, weigh potential risks and rewards, and pick the best path. This marked a pivotal shift from reactive intelligence to proactive, anticipatory thought.

Because the neocortex enables advanced intelligence, because it performs so many functions across billions of neurons and trillions of connections (vision, hearing, touch, language, planning, etc.) – we initially thought it was a system with thousands of sub systems each devoted to a particular task.

But then Mountcastle came along and hypothesised that the neocortex was not made up of different things, but with the same fundamental thing repeated countless times.

This “fundamental thing” is the neocortical column, it is the basic building block of the neocortex that can be repurposed to process any kind of input.

The differences in function (e.g., visual vs. auditory cortex) arise not from different internal wiring of the columns, but from what input they receive and what outputs they send.

This is a concept often called “cortical uniformity” or “generic cortical microcircuit“.

The word “general” is worth talking about.

Bennett mentions twice, the fact that the computational framework of the neocortical column is so general that you can put an auditory input, or a visual input, or a olfactory input, or a somatic input – and the column would process the incoming data in the same manner.

The implication is that the same basic computational engine (the columnar circuit) can perform its characteristic operations (e.g., detecting features, associating inputs, learning patterns) on whatever input it receives.

There’s compelling experimental evidence supporting this.

In classic experiments, the visual input pathway of ferrets was surgically rewired to project to their auditory cortex, rather than their visual cortex, early in development. Remarkably, the auditory cortex, receiving visual input, began to process visual information and even developed visual receptive fields and columnar organisation akin to a normal visual cortex.

Now, if the computational unit is fundamentally the same, and it can work reliably on different kinds of input – this tells me that somewhere in there, the inputs are being normalised or standardised.

If a neocortical column is truly a general-purpose computational unit, capable of handling visual, auditory, tactile, or olfactory data, then the raw, widely diverse physical signals from our senses (photons, sound waves, chemical molecules, pressure) must somehow be converted into a common, standardised neural language that the column can “understand”.

And indeed, this is exactly what happens in the brain.

By the time sensory information reaches the general-purpose neocortical columns, it has already been:

1. Transduced: Converted from physical energy into electrical signals.
2. Pre-processed: Filtered, amplified, and organized into abstract feature maps by dedicated sensory pathways.
3. Normalised: Scaled and adjusted to fit within the dynamic range of neuronal firing, regardless of the absolute intensity of original stimulus.

This means that while the content of the input is different (visual pattern vs. auditory pattern), the format of the signal being fed into the neocortical column – a pattern of electrical spikes, appropriately scaled and filtered – is sufficiently similar across modalities for the general computational framework of the column to operate on it.

The Neocortical column, ladies and gentlemen, was the building block of advanced intelligence.

And it powered Bennett’s third breakthrough in intelligence: Imagination.

Imagine eating a rich chocolate caramel cake.

Can you taste it? Can you taste the tang of the red cherry layering on the top? Can you taste the mushy sweetness of the dark chocolate middle? Can you taste the salty-sweetness of the caramel? Can you feel the cool softness of it as it melts in your mouth? Yes?

Now ask yourself – how?

“How was I able to taste an imaginary cake? How did it feel so real despite happening in my head?”

The answer reveals something cool about how we perceive “reality”.

Not only are you able to taste a real cake, you are also able to imagine tasting an imaginary one.

There was no actual sensory input to get you to think you were eating a cake. You just imagined it.

In fact, in the same way, you can imagine anything you’ve experienced before. You can imagine feeling cold and shivering, you can imagine pain, you can imagine you’re tired and fatigued.

As I said before, the neocortex emerged to enable organisms to imagine different plausible scenarios before committing to one in real life (ideally, committing to the one in which they saw themselves the best off).

They needed a way to take a break from reality to imagine various futures, before coming back to it and commit to an action.

Day dreaming, if you will.

This is known as the Helmholtz hypothesis: “… you don’t perceive what you actually see, you perceive a simulated reality that you have inferred from what you see.” (a.k.a. “perception as controlled hallucination” or “perceptual inference”).

Our brains don’t just passively receive sensory data; they actively generate a model of the world and then compare incoming sensory data to that model.

Let me repeat: you’re not seeing reality, you’ve never seen it.

The only thing you’ve ever seen, what you’re seeing right now, is a simulation of what your brain thinks reality should be like, every moment, all the time.

You’re dreaming, all the time. The fact that your eyes are open sometimes is a secondary detail.

Your brain is constantly generating a “best guess” or prediction of what the world is like, based on all your past experiences, knowledge, and current context. This internal model is what you “see.”

Incoming sensory data from your eyes (or ears, skin, etc.) doesn’t form your perception directly. Instead, it’s compared against this internal prediction.

If the sensory data matches the prediction, the brain doesn’t do much. The prediction is confirmed, and that’s what you perceive.

If the sensory data contradicts the prediction, then a “prediction error” is generated. This error signal is what gets propagated up the hierarchy of the brain, prompting your internal model (the simulation) to update itself to better match the new, unexpected reality. Continuous feedback loop.

Think of it as your brain running a sophisticated, real-time virtual reality simulation, and then using incoming sensory data to continuously calibrate and update that simulation.

The conscious experience of “seeing” or “hearing” is the result of this internal simulation, validated by the senses.

During wakefulness, your brain’s internal simulation (your perception) is constantly being “tethered” or “grounded” by the incoming sensory stream.

This sensory input acts as a powerful reality check, constantly correcting and refining your internal model.

During dreaming (and even hallucination in general), the brain is still generating a rich, immersive simulation. However, the crucial difference is the lack of strong, coherent, and consistent external sensory input that would normally “correct” or “override” the internal model.

So, the internal simulation is free to run wild, unconstrained by external reality checks.

This is why dreams can feel so real, yet often contain impossible or illogical elements – there’s no incoming data to signal a “prediction error” for a flying pig.

So, how does your brain’s “Narrator” – this storyteller that generates your reality – actually function? The answer lies in the fundamental architecture of the neocortical column, the very “building block of advanced intelligence” we discussed.

Unlike most traditional computer circuits that process information in only one direction, the neocortical column is a master of two-way communication.

It’s built with connections that flow both forward (bottom-up) and backward (top-down).

This bidirectional design is the engine of the Narrator:

• When you are perceiving the world, sensory data from your eyes, ears, and skin flows bottom-up, providing the raw, noisy evidence from reality.
• Simultaneously, your brain’s higher-level areas – the “Narrator” – are constantly sending top-down signals, which are essentially predictions or “guesses” about what that raw sensory data should look like, based on your prior experiences and internal models.

Your brain then constantly compares these two streams at each of the six levels in the neocortical column. The difference between the incoming sensory data and the brain’s top-down prediction is called a prediction error.

Only this “surprise” signal travels back up the hierarchy, prompting your internal model to adjust itself.

This continuous loop of “predict ➡️ compare ➡️ update” is how your brain stays tethered to reality, ensuring your internal “dream” accurately mimics the outside world.

This two-way street means that the neocortical column can effectively run in two modes:

• Perception (Predicting Sensory Inputs): In this mode, the column is primarily driven by bottom-up sensory data, and its goal is to reduce prediction error by updating its internal model to accurately reflect reality. This is when your Narrator is mostly listening intently to the outside world.
• Generation (Producing Outputs without Direct Input): In this mode, if the bottom-up sensory input is weak or absent (like in imagination or dreams), the top-down predictions are no longer being strongly corrected by reality. The column can then freely generate patterns of activity based purely on its internal model. This is when the Narrator is mostly thinking to himself. This is what allows you to vividly imagine that cake.

Crucially, the very same neural circuits are used for both perception and generation. They are two sides of the same coin: the brain doesn’t have separate machinery for “seeing” and “imagining”.

It’s merely balancing top-down predictions against bottom-up sensory input.

When the input is strong and coherent, the system is constrained by reality (perception).

When the input is weak or absent, the internal generative power is unleashed (imagination, dreams).

This is why you cannot vividly imagine and pay full attention to reality at the same time – it’s the same circuit. And also why people with neocortical damage not only lose the ability to perceive things but also imagine them.

This bidirectional, predictive architecture, famously captured by the Helmholtz machine model (which had both forward and backward connections), provides the fundamental blueprint for how the brain functions.

It also, quite remarkably, forms the conceptual and mechanistic foundation for a large and powerful class of modern Artificial Intelligence: Generative AI.

In essence, the neocortical column itself can be thought of as a biological generative AI microcircuit, constantly predicting and creating, whether that’s the reality you experience or the dreams you have at night.

Sensory Input is Ambiguous, Noisy, and Incomplete:

• Ambiguity: A single retinal image can be interpreted in multiple ways (e.g., optical illusions). A distant sound could be from many sources. Direct wiring would lead to constant misinterpretations.
• Noise: Sensory organs are imperfect. There’s always background noise, missing data (like your blind spot), or temporary occlusions. If you directly processed only the raw input, your perception would be patchy, full of gaps, and constantly flickering.
• Incompleteness: Your senses only capture a tiny fraction of the world. You don’t see the other side of an object, but you perceive a whole object. You don’t hear every single molecule of air vibrating, but a cohesive sound.

Overcoming Neural Delays and Making Rapid Decisions:

• There’s a significant time lag between an event in the world and your brain’s processing of it. If you were purely reactive to raw input, you’d always be behind reality.
• The Narrator allows you to anticipate what’s coming next, effectively closing that temporal gap. It projects itself into the future to perceive the “present.” This enables quick, adaptive responses crucial for survival (e.g., catching a ball, dodging a predator).

Efficiency and Energy Conservation:

While the Narrator’s simulation itself is “expensive,” it ultimately leads to efficiency.

• Prediction Error Minimisation: The brain doesn’t have to process all incoming sensory data in detail. It only needs to process the prediction error – the difference between what it expected and what it actually received. If the prediction is accurate, very little energy is needed. This is far more efficient than constantly processing a firehose of raw, noisy data from scratch.
• Prioritisation: Prediction errors tell the brain exactly where to focus its limited processing resources – on the unexpected, on what matters for updating its model.

Learning and Adaptation:

The predictive layer is the learning mechanism. When predictions fail (prediction error), the internal model is updated.

This constant cycle of predict-compare-update is how the brain continuously learns about and adapts to its environment.

Without it, learning would be slower, more rigid, or impossible in complex, dynamic worlds.

Dealing with Imagination, Planning, and Dreams:

If perception is merely reading inputs, how do we imagine, plan, or dream?

These are internal simulations without external sensory input.

The same Narrator that constructs your conscious reality during wakefulness is simply unleashed from sensory constraint during sleep or imagination.

Vicarious Trial and Error (VTE): This is our brain’s capacity to mentally “try out” different actions or paths and evaluate their likely outcomes, all before committing to any physical movement.

This internal rehearsal allows an organism to anticipate the consequences of its choices and select the best course, dramatically reducing wasted effort and risk in the real world.

Counterfactual Learning: This is the powerful ability to learn from events that didn’t happen, or from scenarios that were merely simulated. Instead of only reinforcing actions that led to actual rewards, the brain can now consider “what if” scenarios: “What if I had taken that other path? What if the predator hadn’t appeared?” By comparing imagined outcomes with real ones, or with other imagined possibilities, the Narrator refines the organism’s understanding of cause and effect, optimizing decision-making in ways far beyond simple trial-and-error.

Episodic Memory: This is our ability to mentally re-experience specific past events, complete with their sensory details, emotions, and chronological order—the “what, where, and when.” This is distinct from simply remembering facts or skills. It’s the Narrator’s capacity to weave these elements into coherent personal “stories” that allows us to learn from our unique past experiences, predict future scenarios based on similar events, and build a rich, continuous sense of self. It is this suite of capabilities that elevates intelligence to truly remarkable levels.

In contrast to the Narrator’s sophisticated mental simulations, much of the daily action of organisms, and indeed, much of your daily actions today, particularly habituated actions and simple stimulus-response behaviours, are governed by older, more primitive brain structures (like the basal ganglia and amygdala).

These systems operate largely “model-free”, relying on a vast ‘scorecard’ of learned associations: ‘Situation X -> Action Y -> Reward Z (Good!)’ or ‘Situation A -> Action B -> Penalty C (Bad!).’

When we scroll social media seemingly in a trance, or automatically tie our shoelaces, it’s often these older processes running the show.

This ‘scorecard’ is built directly from real-world trial and error, creating fast, reflexive ‘gut feelings’ or automated responses without the need for conscious deliberation or simulation – this is what we were discussing in the previous section on reinforcement learning.

The Narrator (our conscious, simulating neocortex) is largely disengaged during these times, simply allowing these highly efficient, hard-wired behavioural programs to execute.

This is where the Narrator comes in, actively filling his notebook by figuring out the rules and dynamics of the environment, allowing him to mentally explore possibilities without actually having the body perform them.

Like a seasoned chef, who after years of trial-and-error (model-free learning), understands ingredients, reactions and heat – thus, can cook a new dish in their head without entering the kitchen. They’ve figured out the “rules“.

The notebook contains entries like: “If I am in Situation P and I take Action Q, then I will typically end up in Situation R”.

This explicit internal model of the environment’s dynamics – this is the simulation you live in.

The Narrator actively consults and uses this section for mental simulation (VTE), planning, and counterfactual learning. Where it plays out possible future actions.

The “surprise” for our friend, the Critic, now comes not just from actual external rewards, but from these internal simulations predicting a better outcome than previously expected.

The Narrator actively consults and manipulates this section for planning, problem-solving, and adapting to novel situations, enabling counterfactual thinking – learning from “what if” scenarios that only exist in its mind.

The neocortex, is divided into two halves. While the sensory neocortex (at the back) is dedicated to rendering a detailed simulation of the external world, the front half is the frontal neocortex.

This frontal region in humans comprises three main subregions: the motor cortex (involved in planning and executing movements), the granular prefrontal cortex (gPFC), and the agranular prefrontal cortex (aPFC).

For a deeper dive into the intricate architecture and evolutionary journey of these remarkable brain regions, I highly recommend exploring Bennett’s book.

The aPFC’s configuration is quite interesting. You see, just as the sensory neocortex receives input from external senses (eyes, ears, skin) to build its model of the outside world, the aPFC receives its primary input from internal brain structures: hippocampus (our spatial map and memory hub), hypothalamus (regulating internal bodily states like hunger and thirst), and amygdala (processing emotions and valence).

This suggests something remarkable: the aPFC appears to treat inputs of our own internal states, emotions, and past experiences – our “inner world” – the same way the sensory neocortex treats sequences of external sensory information. The aPFC is attempting to explain and predict the animal’s own behaviour, just as the sensory neocortex explains and predicts the flow of external reality. I cannot overstate how ground breaking this discovery has been for neuroscience.

You may think: “It was preposterous enough to suggest that the brain simulates reality and then compares it to sensory input, but here this guy is saying that the brain simulates the animal itself and then compares it with the actual actions taken by the animal. What? What about free will? What about intent?“

Just as the brain predicts what it should see, it also predicts what it should do, or what its intentions are.

The aPFC, by modelling these behavioural sequences and goals, essentially creates an internal “simulation of future intent.”

If your aPFC “predicts” you’ll reach for water (based on your internal state of thirst and learned goals), but your body (via basal ganglia/motor cortex) starts moving towards food, this creates a “prediction error” about your own behaviour/intent.

Just as the sensory cortex is predicting how the outside world should behave, the aPFC is predicting how the animal should behave.

But unlike the sensory cortex that updates the simulation to conform to the world, the aPFC tries to make the animal conform to the simulation.

That is, if there’s a prediction error, the aPFC drives action to resolve it, not just updates its model. It tries to make its own predictions come true.

This is what you may call “will“.

The way it does so, in simple words, is by essentially playing “trailers” of alternative futures for the older brain (basal ganglia) to “see”.

This is nothing but the Vicarious Trail and Error (VTE) I explained above – But for the older brain, imagined futures and experienced futures both result in “votes” for or against the actions that led to those futures (reinforcement learning).

And if the votes for the actions that lead to a certain future cross a threshold, then the older brain takes those actions.

As Bennett explains: “… neocortex simulates sequences of actions, but what makes the final decision … [the] basal ganglia … [enabled by the aPFC the] process of vicarious trial and error unfolds … votes [accumulate] for each choice in the basal ganglia—the same way it would if the trial and error were not vicarious but real. If the basal ganglia keeps getting more excited by [option ‘A’] than by [‘B’] … then these votes … pass the choice threshold … [and the] basal ganglia will take over behavior … the aPFC vicariously trained the basal ganglia …”.

Bennett (and the predictive processing framework) suggests that “intent” or “will” is not some ethereal, non-physical entity, some “uncaused cause”, but rather a computational construct.

A high-level prediction the aPFC generates about future behaviour that it then tries to make happen

His view doesn’t necessarily eliminate free will, but it redefines it as an emergent property of a complex, predictive, self-organising system.

Our “will” might be the brain’s highest-level prediction of what it’s going to do, which then biases the system to make that prediction a reality.

The aPFC is “generating” it all the time, but the times we are not acting out of this “will” is, like said before, when the habits are ingrained in us. But when the basal ganglia’s habit system doesn’t have a strong, clear answer, when there’s uncertainty or a prediction error, or the stakes require deliberation, or a novel situation arises – aPFC engages.

Note, however, that the aPFC does not “come online” immediately at birth – instead, it comes online slowly and learns about the world from the basal ganglia.

In development, infants start with more reflexive, stimulus-response behaviours (basal ganglia/model-free dominant).

As the prefrontal cortex develops, children gradually gain more goal-directed, flexible, and deliberate control. The basal ganglia “teaches” the aPFC about basic rewards, and then the aPFC learns to predict and pursue more abstract goals, subsequently “teaching” or guiding the basal ganglia’s execution.

When it comes to the other component of the frontal cortex, the motor cortex, the situation is similar – as Bennett points out “… aPFC learns to predict movements of navigational paths, whereas the motor cortex learns to predict movements of specific body parts. The aPFC will predict that an animal will turn left; the motor cortex will predict [where] the animal will place its left paw …”.

And similar to the aPFC’s workings, when you have the make a novel, complicated move, the motor cortex first helps you imagine that move, which vicariously trains the basal ganglia on the configuration of muscle movement. Over time, through real and vicarious trial and error, you become “a natural” at the move and no longer need (as much) the motor cortex.

So, it seems the neocortex just simulating all the time.

Whether it is simulating the outside environment or it is simulating the organism’s thoughts or it is simulating the organism’s body movement.

Whatever it is doing, in the end it is always just stimulating. It seems.

We’re now underway to seeing the brain evolve to what it is today – there is still some time to go, but we’re getting there.

Bennett talks about the era of mammals that followed the meteorite that struck Earth.

He talks about the subset of mammals that humans belong to, primates, who shifted from nocturnal to diurnal, developed opposable thumbs, became arboreal frugivores, lived in groups and whose brains exploded in size.

He talks about that while mammals all experienced brain size growth, primates experienced the most (besides maybe dolphins and elephants).

He tries to understand why such a large brain was needed, and says that while we think it is about the size of the social groups we live in but there is more to it.

“It isn’t group size in general but the specific type of group that early primates created that seemed to have required larger brains.“.

While many mammals decided it was better to live in groups and paid the costs that came along with it (food scarcity, competition, infighting), for the benefits (safety, protecting the young) – but it was the primates that show sophisticated understanding of the “theory of mind“. It is the ability to put yourself in someone else’s shoes and see the situation from their POV. And not just understand their intentions, but also their knowledge. To know that other intelligent beings exist, and who can have different knowledge and intent than you.

That primates are extremely sensitive to violations of social hierarchy, and “Unlike most other social animals, for primates, it is not only physical power that determines one’s social ranking but also political power … even young children will regularly challenge adults of lower-ranking families, but they won’t challenge adults of higher-ranking families.“.

Basically, which family you belong to determines your swagger in society (true for humans as well).

And not just the family you belong to, but the affiliations you carry as well – a lower ranking primate can have associations (say, due to unique skills) with a higher ranking individual, and thus inherit parts of the latter’s social status.

Bennett says the “Low-ranking individuals with powerful grooming partners get harassed much less, even when the high-ranking ally is out of sight; everyone in the group knows ‘Don’t mess with James unless you want to deal with Keith’ … ability to forge such allyships is one of the primary determinants of an individual’s rank …”.

The growth in primate brain size was not just about the old parts getting bigger – some new parts also emerged during this time.

Specifically, the granular prefrontal cortex (gPFC) – I mentioned this briefly a few pages earlier. And something Bennett calls the “primate sensory cortex” (PSC). These two are deeply interconnected and like the rest of the neocortex, these are also simulators.

The question is, naturally, what are these things simulating?

A hint comes from the fact that the gPFC sits atop the aPFC (which itself sits atop the basal ganglia and other structures).

If the basal ganglia enables reinforcement learning, and the aPFC simulates this reinforcement learning (VTE as we discussed), hence, avoiding expensive real world reinforcement. Then the gPFC is likely to be simulating the aPFC’s simulation.

You read that right, it’s a simulation on top of another simulation.

What was the aPFC’s simulation? Build models (of the animal and its world) and manipulate them to imagine different futures.

Basically, Think.

So, when the gPFC simulates the aPFC’s simulations, it is simulating thinking itself – metacognition – it is thinking about thinking.

While the movie “The Matrix” was about an external party putting all of us in a simulation, it appears from this that all of us are living in a sort of “Matrix” after all.

If external reality is a simulation, and our internal experience of intent, action, and self is also a simulation.

Then we live in a self-generated, self-referential simulation.

Building a complex model of a “mind” (a system that generates intentions, holds beliefs, and processes information) is incredibly challenging.

It’s far more efficient for evolution to develop one general-purpose “mind-modeling” mechanism and then use it repeatedly.

The most accessible “mind” to model is your own, as you have direct, continuous access to its “inputs” and “outputs.”

Enter: Future you

Once your gPFC powered brain simulating your thinking in the present moment – “What am I thinking about right now?” – it is a short step from there to “What will I be thinking about in the future?”.

This ability to plan for future, differing motivational states is closely linked to episodic future thinking (mentally simulating specific future events) and episodic memory (mentally re-experiencing specific past events).

Enter: Others

And once your gPFC powered brain is simulating your own thinking – “Why am I thinking this way?” – it is a short step from there to “Why is he thinking this way?”.

Your own mind provides the best blueprint. If you want to understand why someone is doing something, you can simulate what you would do it.

This ability to simulate others, was advantageous in this new social world that the early primates enrolled themselves in. Having political savvy was as important as having brute force. In previous eras, intelligence grew to navigate the world, but in this new era, intelligence grew to navigate other intelligent beings.

There isn’t a single “language organ” that emerged de novo. Language recruits and repurposes existing brain structures (like the neocortex, basal ganglia, etc.).

Bennett states that the ability to learn language is, at least in part, a consequence of a simpler genetically hard-coded instinct to engage in conversation.

He gives examples of babies engaging in “proto-conversations” (babbling, turn-taking, joint attention, pointing) with parents before they actually learn a language. They are also able to attach descriptive labels to things.

He also posits that humans may have also evolved a unique hardwired instinct to ask questions to inquire about the inner simulations of others.

Asking questions is a fundamental human drive. If our gPFC is simulating others’ minds, actively seeking to clarify or expand that simulation via questions (“What do you think? Why did you do that?”), makes sense. It’s a direct way to reduce prediction error about other minds. This is a plausible and insightful extension of Theory of Mind.

In this way, Bennett says evolution gave humans a curriculum to learn language – because complicated concepts cannot be given in one go to bio-intelligence (or even AI).

He is saying that the hardwired need in human infants to engage in proto-conversations, and our curiosity about people’s inner simulations provides a sort of curriculum for language to develop in later years – “… language emerges in the human brain through a hardwired curriculum …” – just like flying emerges in birds through a hardwired need to jump.

Continuing, Bennett says that language is a way for us to transfer our simulations to others. And while the same happens when animals communicate with each other (indeed, in absence of this, the other party will not be able to understand you), “… these kinds of transfers are undetailed and inflexible, capable of transferring information only with genetically hard-coded signals … always few in number and cannot be adjusted or changed …”.

Perhaps “transfer” is not the right word, rather, I will say that language provides us a way to create such conditions in other’s minds that we think will lead to the same or similar simulations as ours.

In this way, humans can learn not only from their own imagined actions (simulating), not only for other’s actual actions (imitation learning), but also from other’s imagined actions.

This allows myths (or as I like to call them: hallucinations) to become “common” – powered by language, an entire society is now able to share in hallucinations like the concept of a “nation“, “money“, or “religion“.

Such societies are able to cooperate in flexible ways as Harari popularised in “Sapiens“.

Bennett points out that while the theory of mind enabled initial politicking and group behaviour, this ability alone could not scale the size of the groups beyond the capacity of brains to keep track of relationships, but with language that enabled common myths, groups could be infinitely large.

In this way, Bennett points out that just like nucleotide structures enable the transmission of genes, and useful genes tend to propagate over generations; language enables the transmission of memes, and useful memes tend to propagate over generations.

In essence, both genes and memes are useful information – or as I said – useful configuration, and this configuration, if adaptive, persists over time.

It seems that humans are not very selective in ascribing a mind to others – we readily assume things are way more intelligent than they actually are, even in the face of clear evidence to the contrary – like with AI models like ChatGPT.

As of the time I am writing this, we’re living through times where people are developing more than merely “user – tool” relationships with their AI chatbots _{[1], [2], [3], [4]}.

But the way these popular forms of AI (classified under “Generative AI”) learn today is a brute force method (vs. human learning). When the power of computing is added to learning, you get something that seems intelligent.

AI is narrowly “intelligent”, no doubt about it – Current LLMs excel at specific tasks (like language generation) within a defined domain, making them narrowly intelligent. But beyond that, popular forms of AI (esp. LLMs) merely produce an illusion of a mind, which, I believe, is really our own doing.

We trick ourselves into thinking there is someone intelligent on the other side.

Bennett points out: “Humans don’t learn math the way GPT-3 learns math. Indeed, humans don’t learn language the way GPT-3 learns language.“.

True learning in humans does not happen by rote – To truly understand something is to have a robust internal model of it. Humans learn through simulating.

To me, this difference between what humans are doing and what LLMs are doing is a question of Simulation vs Prediction. Both predict but the latter is mathematically predicting a sequence while the former is playing through a simulation and language is used as a transfer mechanism. To us, language is a means; to AI, it is the end.

Language is a powerful tool, but there is a reason it emerged after the Theory of Mind: Language needs to tread a fine line between exactitude and usefulness.

If the language you use is too precise, like a legal contract, you will spend far too much time on it – speaking/writing – this is not only tedious for you but also for the listener/reader. Language here would cease to be practically useful. That is why we gloss over any “Terms & Conditions” page and hit “I Agree” without reading

But if you use language that is too vague, the person you are trying to communicate with may not understand you or may even misunderstand you. Again, language loses its practical use.

As a communicator, Theory of Mind enables you to peer into the mind of someone else and understand the appropriate level of detail for them. And as the communicated, ToM allows you to understand the intend behind the message. It is this inability of AI (as of today) that leads to apocalypses with Paper-clips.

That is why, I think, Bennett is right when he says “… if we want true humanlike AI systems, theory of mind will undeniably be an essential component of that system.“

Non-fiction authors sometimes get philosophical at the end of their books. For Bennett, this means talking about breakthrough #6 and its differences from the rest.

He says that ASI is likely to be the sixth breakthrough, and while it was evolution that had been doing the work all this while, the sixth breakthrough will come from us.

This will be the time “… when intelligence unshackles itself from these biological limitations.“. Whatever we create, it will be influenced by the five breakthroughs. And whatever we create will not just be a test of our intelligence but also of our values.

Bennett says that the “… universe has passed us the baton.“.

But I don’t agree, the universe will continue its unceasing march – whether it is a march towards the Big Chill or the Big Crunch.

Really, it is evolution that has passed us the baton, will we – the children of evolution, intelligent and powerful – be wise enough to carry on its legacy.