Nexus by Yuval Noah Harari

Literally anything.

Like a knife does not discriminate in what it is cutting; if it can be cut, it will cut it.

Information does not discriminate in what it is connecting; if they can be connected, it will connect them.

“The pan is hot!” – Here I connected “pan” with “hot”, thus creating information through the connection (“pan is hot”). That the pan may actually be cold, or tepid or non-existent is secondary – as I said, representing objective reality is not the point.

Similarly, anything can be connected to anything.

“The moon is inhabited by lizard people.” – Here I connected something real (moon) with something imaginary (lizard people).

When we say “John is married to Jina.”, or “Mike owns that plot of land.”, or “Yesterday I had the most amazing garlic bread.”.

All of these are pieces of information since they connect two things (John ➡️ Jina, Mike ➡️ Land, You ➡️ Bread).

But where is this information? Can you point and say, “there it is!”?

I don’t think the answer is “information lives in its objects”, like minds, books, documents, web pages and so on.

While these things may contain information, it does not live in them. Paradoxically.

Let me try and explain with two examples.

• Imagine, if John and Jina’s marriage certificate were to be destroyed.
• Also imagine, that Mike’s country is annexed by a neighbour that does not recognise his land deed.

For John and Jina, life is going to continue mostly as it was, sure there’s the headache of getting a duplicate marriage certificate. But besides that they’re still married for all intents and purposes. Certificate or not, all their friends and family recognise their marriage. Their bank documents and identity proof list John as Jina’s spouse and vice versa. And perhaps more importantly, they love each other.

For Mike, however, life’s just taken a turn for the worse. With the new government not recognising his claim to the land, he is technically encroaching illegally. Mike will live in the fear that any day a government official will come and evict him. It doesn’t matter what Mike’s friends and family say, if the government doesn’t think he doesn’t own the land, what can they do? For Mike, regaining title to the land becomes existential.

If information truly lived in its objects – the official document in the examples here – then it should be John and Jina who should be most concerned, having lost their document, and not Mike, who still retains his.

In truth, information lives in the connections themselves; specifically, the strength and number of each connection.

The connections linking John with Jina were numerous, and the single connection that was broken when their marriage certificate was destroyed was not strong enough to overpower all others.

While in the case of Mike, a single connection (the government recognising Mike’s claim on the property) held most of the power, and when that connection was cut with the takeover, it didn’t matter that Mike may have had numerous weaker connections between him and the land.

If Mike had had a far greater number of connections (such as the entire country knowing he owned that land), or more powerful ones (such as a relative in the newly formed government), he could have slept more peacefully that night.

This is like asking: “Can a knife cut anything?” – No, at least not very easily. Same with information, not any two things can be connected, at least not easily.

Just like you may repeatedly ram your knife into a boulder and eventually cut it in half, similarly you may forcibly connect two intractable things and eventually succeed – But just like the exercise with the knife, it will not be very pretty or very useful. Not all knives are sharp, and not all connections are informative.

And just as the cut with the knife will not be very precise, the forced connection won’t be very strong. Forcing it is not a great strategy.

So, the bottom line here is that while any connection can technically be called information, it does not mean any two things can be tenably connected.

Information is not just any connection – but one that modifies interpretive state or behavior, even if that modification is imagined.

To answer that, consider what the most basic network is comprised of: nodes and connections.

Nodes: These are the individual units of the network. People in a social network, web pages on the internet, neurons in the brain etc. A node is anything that can store, process, or transmit information.
Think of a node as a mini-world: it receives input, transforms it (or not), and passes it on (or not).

Connections: These are the pathways that allow nodes to interact. Friendships, messages, citations, nerve impulses between neurons, trade routes between cities, hyperlinks between web pages etc. A connection defines who can influence whom.

Connections do not form automatically, they need a special kind of node: An Agent.

By “Agent” I do not mean intentional, I simply mean causally active.

Consider Earth, billions of years ago, and how proto-DNA emerged.

In the primordial soup:

• Certain molecules formed hydrogen bonds more easily.
• Others folded in stable configurations.
• Eventually, self-replicating chains emerged – not because they intended to, but because their structure favoured it.

So were those molecules agentic?

→ In a very primitive sense, yes.
Not agentic like a human making choices, but agentic in the network-theoretic sense: They altered the future state of the network in a consistent, replicable way.

So, the one making these connections, is a special kind of node, called the Agent.

Not all nodes are agents. But any node can become agentic, under the right conditions:

• If it has enough information.
• If it holds a high-degree position (lots of connections).
• If the network becomes unstable, and it provides the new stable pattern.

Information is information when it is able to establish “tenable connections” between two nodes. I’m using “tenable” here to describe connections that are able to modify the behaviour of other nodes.

This relates to formal information theory, where Claude Shannon defined information as a reduction in uncertainty – not as truth or insight. This means even misleading or fictional signals can carry high informational value if they change the probabilities in a system. That’s why conspiracy theories, rumours, and satire all count as ‘information’ – they rewire networks, regardless of accuracy.

When this connection loses its tenacity, that is to say, it loses its potential to modify the behaviour of individual nodes, or the nodes themselves disappear, is when the connection breaks.

And when does the entire information network die?

When the architecture of connection collapses – when no signal, memory, or influence can meaningfully propagate.

Death may come via attrition (nodes fading), suppression (censorship), or rupture (loss of high-degree connectors). Some networks degrade slowly. Others shatter overnight.

Think of extinct languages. The words still exist in books. But the nodes – speakers – are gone, and the living connections between them are severed. The network has died, even if fragments remain in archive.

As I wrote above, information is not about reality, it is about connection. An information network therefore, does not need to represent reality.

But if we consider the information network to be a super-organism, it will accurately reflect reality when it is adaptive to do so (or at least, not maladaptive).

A network can be stable and self-reinforcing – adapted to its own internal dynamics and the external world (or at least, not too pressured by it).

But when environmental conditions shift, such networks may collapse.

So, information and information networks, are not, primarily and fundamentally, about defining or abstracting or explaining reality – Information and Information networks are about creating reality (or “order” as Harari will reveal shortly). While information doesn’t need to be objectively true, but for it to be useful, it must alter node behavior in some adaptive way over time. That’s the thin line between mythology and madness.

Harari summarises, “… the Bible has done a poor job in representing the reality of human origins, migrations and epidemics, it has nevertheless been very effective in connecting billions of people …”.

Before humans started telling each other stories, information networks still existed.

Pre-linguistic hominins had:

• Gestures, facial expressions, vocal tones = multi-modal info
• Observational learning = skill transmission
• Grooming rituals = trust-building

These information networks connected nodes with each other, allowing the nodes to behave in synchrony. These were, what Harari calls, “human-to-human chains” (or “human-to-human networks”). But due to the brain’s limited capacity to hold a mental map of the information network, we couldn’t have more than 150-200 humans (nodes) in a single network.

You probably know of this limit as Dunbar’s number. If it were only for human-to-human networks, we’d have run up against Dunbar’s number – as do all other primates.

The reason we don’t see a coup staged by chimpanzees to take over the city zoo is because they can’t coordinate in very large numbers.

If two humans could agree on the story, then they could cooperate. And even if they forgot all about each other after some time, but still remembered the common story, they could cooperate again.

Apart from very close ties, humans relied on mostly stories to go about their daily lives and cooperate in numbers several orders of magnitude higher than what was possible with human-to-human connections.

Stories about how a omnipotent God was watching their actions, how a man sitting on the throne was their king, how they would get a better life if they were righteous in this one, and many more.

Stories are data compression for the human mind: Too much raw sensory input? → Story turns it into signal. Too many variables? → Story gives it a beginning, middle, and end. Too little certainty? → Story offers a pattern – true or not. The world doesn’t come with structure. Stories lend it.

They structure time: Before clocks and calendars, there were stories: “When the leaves fall, the salmon will return and we will fish.”, “Our ancestors came from the north, fleeing a great fire.”. Stories give temporal coherence – the sense that the past, present, and future belong together.

They create identities: Not just “Who am I?” but also, “Who are we?”, “Who do they say we are?”, “Who were we before we were enslaved?”, “Who will we be when we rise again?”. Your identity is a story you tell yourself.

They encode values: A list of events is not a story. A story selects and frames: who was brave, who was wronged, what matters, and so much more.

They are reality-generators: If enough people believe the story, it becomes true in consequence. Borders get drawn, laws get enforced, people die for flags.

The story enabled humanity’s first synthetic information network – scaling cooperation beyond kinship by trading truth for order. In binding us together, it also blurred our grip on objective reality. We gained mythic cohesion, but lost epistemic precision.

I may even say that we lost some grip on our subjective reality too – In the sense that stories came in between how we actually felt and what we wanted to feel⁴. What it means to be a good son, an ideal employee, a successful person.

Stories offer prefab narratives to explain why we feel what we feel. But by choosing from a menu of pre-approved meanings, we may never confront the raw, idiosyncratic truth of our actual experience.

Harari points to a balance that stories must strike between truth and order.

Truth is the an objective view of reality in order to understand it better, while order is the subjective view of reality to make sense of the understanding.

A story that wants to become a powerful intersubjective reality must toe this fine line.

Be too focused on truth then the story loses stickiness – it is always going to change as new facts are discovered about the world and understanding is deepened.

Be too focused on order then the story loses its practicality – people will say “alright that is a good story, but what does it have to do with my real life?”

If the story poses as the truth (”a man in the sky controls your crop yield”) then it is at risk of being disproven. If the story poses as order (”there’s a man in sky, and he is watching you”), then it loses practical value.

If the story aspires to become a religion then it must balance truth and order: “the weather controls your crop yield, and the weather is controlled by a bearded man in the sky who is watching you”.

• “The weather controls your crop yield.” ➡️ Objective truth. But dry, unpredictable, and emotionally indifferent.
• “A bearded sky-god watches you.” ➡️ Psychologically resonant. But untethered from lived evidence.
• “The sky-god controls the weather, which controls your crop yield – and he watches what you do, so be nice!” ➡️ Now you’ve got a sticky, scalable, system-shaping intersubjective truth.

However, despite this need for balance – Harari seems to point out that in a peculiarly human quirk, we tend to regard order a little higher than truth, and what may explain why storytellers are more popular then truth tellers, “… people who know how to build ideologies and maintain order … give instructions to the people who merely know how to build bombs …”.

It has always been this way, this is why people will consult astrologers before getting married, and why Iranian nuclear experts follow the orders of Shiite theologians.

When an agent (the special kind of node I described earlier) can get other nodes in the network to do certain things, behave in a certain way, develop or drop connections – basically, influence them – the agent can be said to hold power.

Don’t confuse power in the information network context with power over the objective world. It may be possible, such as when a government (agent) orders a group of scientists (nodes) to discover a cure for COVID-19.

Or it may not be possible, such as when the government and its scientists are unable to land a rocket on Mars. Whether or not power in an information network leads to real world outcomes is tangential.

When information is used to uncover truth (such as how the physical world works, or how someone cheated someone else), or when it is used to impose order (such as how we are all God’s chosen people, or citizens of a country) – both of these cases lead to power.

Epistemic Power

Ability to shape what nodes know. (e.g. Scientists, Journalists)

This is the power to influence what people know.

It belongs to scientists, journalists, whistleblowers, and sometimes conspiracy theorists.

It determines what counts as “knowledge” in a network – what is trusted, cited, repeated.

Epistemic power is often fragile – truth doesn’t always win, especially if it’s inconvenient to power structures.

Example: When Galileo insisted the Earth orbits the Sun, he challenged the network’s dominant “truth” and paid the price. Over time, the network shifted.

Normative Power

Ability to shape what nodes value⁷. (e.g. Priests, Ideologues)

This is the power to influence what people feel they should do and value.

It is wielded by priests, activists, ethicists, influencers, and political leaders.

Normative power operates on morality, duty, identity, and social rules.

Normative power doesn’t care if something is factually true—it cares if something is socially binding.

Example: When a religious leader declares something sinful, the normative frame of a community may shift, even without new “truths” being introduced.

Structural Power

Ability to shape who connects to whom. (e.g. Platform Designers, Bureaucrats)

This is the power to influence the architecture of the network itself.

It is held by bureaucrats, platform designers, rule-makers, managers, and system architects.

Structural power determines what’s allowed, what’s possible, and how information flows.

Structural power is usually invisible – until something breaks, or someone is silenced.

Example: Twitter’s algorithm tweak changes whose voices get heard. A visa policy changes which scientists collaborate. A new Discord role changes who can post where.

Memetic Power

Ability to shape what spreads. (e.g. Influencers, Propagandists)

This is the power to influence what ideas go viral, and what gets forgotten⁸.

It’s the domain of advertisers, TikTokers, propagandists, comedians, pop stars.

Memetic power governs attention, repetition, emotional stickiness.

Memetic power can overpower all others if the message is sticky enough – truth, ethics, and structure be damned.

Example: A catchy slogan (“Make America Great Again”) may have more memetic power than a 500-page policy white paper.

Repeat a story enough times and it starts feeling real. That is why it is important to choose better stories, about the world and about yourself. Harari gives the example of Germans in the 1930s, calling their decision to believe in the Nazi story a “tragic mistake” and that they “… could have chosen better stories …”.

The most powerful stories aren’t lies. They’re partial truths wrapped in emotional resonance and systemic repetition. And when they become indistinguishable from identity, escaping them feels like self-erasure.

Say, you tell yourself a story about how you need to rise over your family’s generational poverty challenges and, as a way to show your family that things can get better with effort, you will be the first one in your family to get a job.

You’re all excited and charged up to make this reality happen. Now what? Well, you make a to-do list. Get an education, get vocational training, find an internship (free if it has to), convert internship into a job, and all that comes in between.

You see, for any information network to truly become a powerful intersubjective reality it needs to operate a self-reinforcing loop: it needs to tell stories that people believe in ➡️ then these people need to act on those stories ➡️ which then makes them more deeply invested in those stories ➡️ so they may act next time with greater conviction.

And while stories are great at inspiring and motivating people, they are great for putting people in a shared reality, they are great at energising them; they are not very good at telling them what exact steps to do so that energy isn’t wasted running around in circles. For that we need lists, tables, tallies, records – in other words we need documents.

Stories are interesting, records are boring – still if you want to get anything done in life, you need them.

Why are records boring?

Why is “18342 chickens were delivered to the poultry farm” boring, and “a chicken jumped and landed on the farmer’s head” interesting?

Why is the statement “The experiment results show the treatment worked on one-third of mice population, but had no effect on one-third …” stale, but when combined with “… and the third mouse ran away.” becomes joke about sample sizes?

I think records are boring because humans like narrative, we have been evolutionarily hyper-trained into responding to narratives.

Stories, whether they are written down or oral, tap this evolutionary programming.

Records, no matter how simple or complicated, do not.

This is also the reason that we tend to write down records lest we forget them, because our brains are not primed to remember records, they’re primed to remember stories.

Records too need to balance truth and order – truth meaning how closely information represents objective reality, and order meaning how much it abstracts it.

And just like stories, records create new intersubjective realities.

For example, a passport doesn’t describe anything interesting. It’s a record. But that record can determine whether you’re welcomed into a country or turned away at the border.

Which meant we found ourselves with a lot of written records. How could one locate the right record at the right time from this pile?

This created the problem of retrieval.

Rising to the problem of retrieval is the second key agent in our journey: the Bureaucrat.

What did the bureaucrat do? He organised different records into different folders, sorted them, archived them and so on – he gave them order.

What did this order enable? Specialisation.

What did specialisation enable? Progress (albeit narrow).

To explain, consider a tightly knit family.

Once the stories (love, responsibility, shared dreams) are in place, someone has to figure out how to make life work day to day. That someone is the bureaucrat: The mother who keeps everyone’s medical files organised, the father who colour-codes monthly budgets, the elder sister who sets a shared calendar for chores, the younger brother manages the online payments. They’re not just “being organised”, they’re imposing order on the shared story of “our family”.

How It Plays Out:

• Order: Labelled tiffin boxes, tuition receipts in one folder, electricity bills sorted by month.
• Specialisation: Everyone has clear responsibilities, the younger one handles online payments. The older one manages car service. Mom’s the go-to for medical appointments.
• Progress: Fewer missed deadlines, better finances, smoother life.

Without this bureaucracy, even the best family story – “We support each other no matter what” – gets lost in forgotten grocery runs and missed doctor’s appointments.

What happens at the level of the family, also happens at the level of the country, just that the bureaucracy is a lot more complicated.

In the family, if the younger one’s records say something, the family must believe it – In the country, if the bureaucrat’s records say something, then the nation has to believe it.

And so it is, with the advent of these new human-to-document chains, power centralised even further.

Centralisation of power happened with the emergence of the storyteller as well, but at least people could understand we he said.

But when power centralised with the bureaucrat, what he said – referencing obscure by-laws and provisos – nobody could understand. As I wrote above, this goes back to our evolutionary priming to remember stories but not facts and figures. Where the storyteller wielded power through persuasion, the bureaucrat wields it through procedure.

And hence, even when a bureaucracy was doing legitimate good, it could still be seen as dubious or diabolical – humans have always been afraid of things that do not understand.

Lawyers, accountants, managers, civil servants and all other bureaucrats, and the orders they created – these became the new realities of people.

And, in what is a popular saying in many countries, “knowing how the system works” became a critical skill for modern day living.

Together, they form the dual architecture of cooperation:

• One tells you why the world is the way it is.
• The other tells you what to do now that you know that.

Truth and order represent the two competing goals that information networks can pursue.

An information network designed to pursue truth will constantly change its topology to more accurately represent/predict an external benchmark/reality – its focus is fidelity of representation.

An information network designed to pursue order will keep its topology fixed to maintain coherence – its focus is integrity of structure.

Of course, like all of life, a balance needs to be struck between these two extremes.

No information system can survive by purely focusing on either.

Stories (narratives) and Documents (records) bookend the range of information technologies with which we may pursue either truth or order.

Both stories and documents can lie on a continuum going from simplicity to complexity. A story can be a fable or a legal constitution; a document can be a grocery list or the Ten Commandments.

Solving this balancing problem is critical.

Focus too much on truth and you risk destabilising the network, which may lead to collapse.

Focus too much on order and the network calcifies and becomes fragile, which may lead to collapse. The network needs to be able to self-correct to survive. It needs to know when to lean towards truth and when to lean towards order.

But humans are fallible, and a human balancing well today may come unhinged tomorrow.

Harari talks about how in ancient Athens, the Pythia (a priestess in-charge of balancing truth and order) was bribed by the pro-democracy faction in the city to instruct the Spartans to depose Athens’ ruler, Hippias.

What happened here has happened many times in history: a fallible human agent overweighted order, this created an intersubjective reality in a group of people, who then went ahead and killed someone.

So, we humans thought a better solution would be to write a book – mind you, a balanced book, a holy book that took the best of what we knew – whose wise words would settle once and for all, the balancing problem.

The benefit of the book is that it is unchanging so it fixes the problem of fallible human agents who keep changing their minds, and also that we can use a book to place checks on what a selfish agent is allowed to do.

But there are a few problems with the book as well. 

• First, there’s the issue of curation: What/What not to include in such a book so that we may call it “balanced” (and that everyone can agree on said balance). This saw the rise of the Curator, a special kind of bureaucrat who made lists about the stories you should read and which ones to avoid.

• Second, the larger issue was that books cannot speak. And because books don’t speak, we needed someone to interpret them. And so rose the Interpreter, a special kind of storyteller whose authority stemmed not from being able to tell great stories, but being able to read great stories from a book. The Talmud and the Rabbi, or the Holy Bible and the Bishop, or the Vedas and the Brahmin, or the Quran and the Imam – emerged from this. As Harari astutely points out, “… arguments about the interpretation of [the holy book] became even more important than [the holy book] itself.”.
• Finally, because the truth-seeking can be unstable (since it tries to represent external reality) by writing a book and trying to “lock-in” the truth we accidentally created order masquerading as truth.

The ironic idea was that we would take this book, wave it around, and tell everyone: “Here is the truth! And with this we will create order!”.

Instead of solving the problem of balancing truth and order, we hardcoded a single solution into the network and then locked it from future edits. The interpreter could interpret, yes – but only from within the strict grammar of what had already been written.

The network became reflexive, not adaptive.

As I wrote above, the problem was that we had accidentally created order masquerading as truth.

We had deified the holy book, we had frozen the supposed truth in time – we had told everyone that the holy book contained the infallible, ultimate truth.

This led to, what Harari calls, the “infallibility trap”.

The interpreting institution was trapped because it could not be seen as interpreting anything. Are you seeing what’s happening here?!¹⁰

The book had supposedly solved this exact problem, it had done away with the need for any human medium, whether a storyteller or an interpreter.

And that is why, the interpreting institution could only be seen as transmitting what was the absolute, unvarnished truth.

While Harari says that humans unfortunately forgot that the book the institution was supposed to interpret, was also made by humans – I think he perhaps missed the point that the institution they created was incentivised to make humans forget that little detail.

Without realising, humans bound themselves in a trap of their own making.

This created an institution that believed ultimately, in its infallibility. Individual humans could come and go, but the institute, which derived its power from an infallible source (indeed, for many religions, a source literally given to us by God himself) was also, infallible.

Yes the words of the book were locked in, but what really got locked in was the idea that the book cannot change.

This is order masquerading as truth – and once you embed that, you have built a system that can never evolve without shattering its own source code.

And we repeated the same mistake over and over; fallible humans creating fallible institutions.

Why can’t humans help themselves?

Even when they say they want the truth, they move towards order – like a sugar addict saying he wants to be fit while helping himself to his third slice of chocolate cake.

A subconscious, hardwired desire for control perhaps?

Anyway, this right here, ladies and gentlemen, is the reason that networks where order masquerades as truth are so worried about, and brutal towards, attempts at finding the truth.

It is true of certain religions, it is true of populist totalitarian regimes, it is true of cults, and ironically, it’s even true of conspiracy theories that are supposedly designed to unearth the truth.

This is why heretics are burned at the stake instead of being given documents showing experimental replicability.

What was applicable back then is as applicable today despite hundreds of years of progress in between. And will persist until humans gain the humility to accept that truth cannot be locked-in. It is ever changing, and so an attempt to lock it in is false comfort.

But even science – the supposed gold standard of truth-seeking – is made up of humans.

And humans are not epistemically neutral.

We have careers, reputations, identities – all intertwined with the stories we already believe.

When a new truth challenges those stories, we don’t just reject the idea – we feel personally attacked.

As Max Planck, one of the fathers of quantum theory, famously said: “A new scientific truth does not triumph by convincing its opponents … but rather because its opponents eventually die, and a new generation grows up that is familiar with it.”.

No holy books here, no unquestioned leaders. Everything was up for debate.

This enabled the network to adapt much faster in light of changing external reality.

Harari illustrates with the case of Dan Shechtman whose discovery of five fold rotational symmetry was ridiculed by peers first, similar to what Ignaz Semmelweis faced.

But unlike Semmelweis whom the network drove into a mental hospital, the scientific information network at the time of Shechtman had evolved to the point that it realised its own mistake and awarded Shechtman a Nobel prize in 2011.

First, it takes more energy to maintain self-correcting mechanisms than just assuming what you know till now is good enough.

Second, the question “correct towards what, and away from what?” is critical.

Till this point we’ve understood the answer to be “Towards the truth, away from order.”.

But as Harari points out, “… even when the social order is highly oppressive, undermining it doesn’t necessary lead to a better place. It could just lead to chaos and worse oppression.”.

Third, self-correcting networks focused on truth-seeking don’t offer the comfort of final answers, they’re faithless disciplines and not for the faint of heart.

It takes a bit of courage to state into the abyss.

Most people don’t want to stare into the abyss. They want a balcony seat, a warm blanket, and a story.

Both are information networks, information flows through both, both have roughly the same tools available to them, but both use those tools differently. And both have very different relations to truth and order. Dictatorships are about dictation, democracies are about conversation.

This is because democracies and dictatorships are based on fundamentally different stories. In an extremely over simplified sense: Democracies tell themselves a story whose moral is that order must be in service of truth, which is prime, ever-changing and can’t be tied down.

Dictatorships tell themselves a story whose moral is truth must be in service of order, which is prime, fixed and must be upheld.

• Design Features:
  • Decentralised structure: many voices, many channels.
  • Self-correcting: elections, independent media, courts.
  • Legitimacy comes from consent, not coercion.

• Advantages:
  • Resilience through redundancy: if one institution fails, others can compensate.
  • High adaptability to complex, shifting environments.
  • Institutionalised dissent fosters innovation and reform.

• Disadvantages:
  • High signal-to-noise challenge: truth is harder to isolate.
  • Gridlock and polarisation can slow down urgent action.
  • Vulnerable to manipulation by actors skilled in memetic power.

• Design Features:
  • Hierarchical structure: one center, many peripheries.
  • Suppression of dissent to avoid signal degradation.
  • Legitimacy comes from authority, not consent.

• Advantages:
  • Rapid execution: few barriers to action.
  • Narrative coherence across channels: clear messaging.
  • Long-term vision easier to implement (in theory).

• Disadvantages:
  • Fragile to error: if the top is wrong, the system follows.
  • Information distortion due to censorship and fear.
  • Lack of feedback mechanisms makes course correction rare and costly.

Neither is bulletproof. Both are vulnerable to ossification, to misinformation, to abuse

But their failure modes differ.

Democracies tend to decay into noise; dictatorships collapse under brittle silence.

In the 1960s it seemed that democracy was unsustainable while in Stalin’s Russia, order prevailed.

Today it is the opposite. Tomorrow it may change again.

And so the question is not which is “better” in theory – but which network design offers the best shot at long-term truth-order balance in practice.

A democracy tries not to interfere in the life of its citizens but if it does not interfere at all, even in places where it should, like protecting citizens from violence and crime – then it is not a democracy, it is anarchy.

A dictatorship may be based on the populist idea that power is the only reality and everyone is out to get it, but then even the dictator is no different, as a result all truth breaks down – then it is not a dictatorship, it is anarchy.

Harari draws our attention to a class of computers – those that can not only make decisions but also generate ideas.

But for the purposes of this Field Note, my definition is a whole lot simpler: A computer is hardware combined with some software.

Both hardware and software can act as information technologies in a network – nodes that enable other nodes to connect with each other.

For example, the computer screen is hardware that can act as IT by displaying relevant text and images; while the PDF is software that can act as IT with the help of appropriate hardware.

But software is special because it can also act as an agent. Software is potentially agentic, like each human is potentially agentic.

For example, YouTube’s recommendation algorithm (software) is an agent in the network because it actively shapes the network by recommending videos that millions of people watch each day.

A slight bend in this algo (say, showing more revolutionary videos than pacifist ones) can lead to the entire network behaving differently.

Note that software exists even outside computers, for instance a Rube Goldberg machine has rudimentary software too: it is the exact position and configuration of the various bobs and bits that make up the machine.

Considered that way, Software just becomes a configuration that leads to desired outcome. The design of a car’s engine is software because it produces reliable, desired outcome (forward motion), so is the hand-blender, so is the toaster, and so on.

In that sense, whatever machines we designed in the past, came with some software. This is not software in the digital sense, but a broader definition – any configuration, any design that encodes behaviour over hardware.

Software presupposes the presence of underlying hardware, it uses the hardware as foundation. But like I said, this software is rudimentary – it cannot adapt, is relatively simple and modifying it is a chore.

With silicon based hardware we got silicon based software. Silicon based S/W is powerful because it can be designed to adapt, can handle incredible complexity, and modifying it is much easier (just change a line in the code).

Importantly, the designers of some of that silicon based software leaned towards making it more agentic, and that is the ongoing information revolution we’re all part of.

With this, the silicon agent has entered the network. As a reminder, an agent is a special kind of node that is causally active.

An agent can change the network – by forming new links, breaking old ones, influencing other nodes, or altering the overall topology.

Most revolutions in human history introduced new tools. This one introduces a new agent.

They had become editors-in-chief for the content and social media feeds of billions of people, they had become master-traders buying and selling in the hundreds of millions through complex calculations, they had become heads of security, observing millions each day and flagging those they thought were risky.

They had taken on these and many more roles, and it definitely impacted how we saw reality – a simple example is how millions of teenage girls had their realities drastically altered due to Instagram’s recommendation algorithms, as authors like Jonathan Haidt have written about.

But they had not directly hacked our stories. The agents making the stories were still human.

That changed in late 2022. With the release of generative AI models like GPT-3.5 and GPT-4, the silicon agent gained a new, unprecedented power: the ability to shape human stories.

Until the emergence of GPTs silicon based agents were in our world, but with the advent of ChatGPT, Gemini, Replika and many others that will come in the future – silicon based agents have entered our minds.

They began mimicking our voices, anticipating our questions, completing our thoughts. For the first time in history, language – the “I’m special” badge humans wear with pride – may not remain unique to humans.

One of the first and popular example of this hack and mind entry was of Blake Lemoine in 2020, but such examples can be found taking place daily across the globe.

Humans are increasingly using LLMs for all kinds of stuff, sharing secrets, taking advice, and in the process developing “relationships” with them.

Harari predicts that we’ll all be relying on our personal AIs to mediate our relationship with the world. If the AI says it’s false, then like the Oracle in Ancient Greece, it might as well be.

And this is going to be bad for democracy because just as members of the Catholic Church listen to the bishop as the single source of truth, or citizens of a dictatorship listen to the leader as the single source of truth, members of future generations may just listen to their own AIs as the single source of truth.

Harari says, “We are in danger of losing control of our future. A completely new kind of information network is emerging, controlled by the decisions and goals of an alien intelligence.”.

Harari goes as far as to call it the “end of human history”, in the sense that history will be increasingly shaped by computers.

Our values define what we consider important, and often when two parties clash, it is a clash over values. Values define what a network optimises for. You can think of them as the loss function in machine learning, or the story’s moral in human culture.

They tell the network: “This is good. Do more of this.”.

If you train a network on likes, it will seek likes. If you train it on compassion, it will seek that.

The network has no intrinsic understanding of morality. It inherits its “should” from the values embedded in its design.

Getting a handle on our values, and more importantly, developing a system with which to evolve our values is – in my opinion – going to be the defining quest for humanity in the age of AI.

If we are confused (or short-sighted) about our values – AI will mirror our confusion. Mo Gawdat said the same in “Scary Smart”.

The engineers that coded the algorithm had values that prioritised “engagement” and “growth”.

They codified these values in the form of impressions, likes, shares and watch time – the algorithm inherited these values and went about increasing “engagement” but not realising that a hateful comment was very different from a compassionate one.

The engineers optimised for engagement, and encoded that as likes, shares, impressions, and watch-time. They didn’t ask: “Engagement in the service of what?”.

The system didn’t know, didn’t care. It just found what generated the most engagement – and it turned out, that was hate.

You will often hear about the “paperclip experiment” as a doomsday scenario that a misguided AI can land us in, whether or not that happens only time will tell.

But we’re already seeing “mini paperclip experiments“, like the Rohingya violence episode, that show the power of misguided AI here and now.

Although commonly used in the context of computers and AI, the alignment problem is simply a time when any agent does things that are beneficial tactically, but harmful strategically.

As in the namesake experiment, if a factory managing AI is given the objective to maximise the production of paperclips and does so at the expense of humanity, then it has failed at alignment¹¹ – the tactical goal to achieve maximum paperclips production was achieved but the strategic goal of the paperclip company (say, a world where no human has to deal with an unruly stack of papers) is not.

But the same failure at alignment happens when a sales manager offers deep discounts, hence achieving their tactical goal of achieve sales targets, but missing out on the strategic goal of profitable growth.

Or when a politician allocates state budget to give freebies to the electorate, hence achieving their tactical goal of re-election, but missing out on the strategic goal of developing the state.

In the context of information networks, the alignment problem is when an agent performs actions that are harmful to its long term sustenance even if it might seem beneficial in the short term.

The alignment problem is generally wasteful and becomes particularly dangerous when the agent is non-human, like a silicon-based algorithm, because it lacks internalised human values, and external correction is often too slow for it.

It was precisely this alignment problem that led to the Facebook algo delivering the tactical goal of engagement while failing at the company’s strategic goal of creating a most respectful and cohesive society.

If your values are not clear or if you’re kidding yourself into believing something they are not – then you’ll run into the alignment problem.

In the case of Facebook, as of this writing, its purported value is to “bring people closer together” and I suppose it would have been something similar at the time of the Rohingya violence as well, and indeed, all of Facebook (the company) may truly believe in it, and live by it.

But the FB algo played a part in the violence, and as Harari demonstrates, did not take action despite repeated warnings. What should this tell us?

To me, it tells that there was a deeper, unsaid, unwritten value that was guiding Facebook at the time.

In the hierarchy of values, that implicit value was ranked above the explicit one.

I suspect the value would have been something related to “growth”, related to “ROI”, related to “valuation” – something that prevented the information network orchestrating Facebook (the company) in 2016-17 to not take action despite warnings.

Even if your values are clear you need to convert them into numbers and other measurables to know if and how well you are living as per your values.

If you choose the wrong proxies and metrics – you’ll again run into the alignment problem.

Taking the Facebook example again, and assuming the engineers truly believed in Facebook stated mission of bringing people together – they still needed to answer the question – “How do I know I’m bringing people together?“.

The proxy for “togetherness” that the engineers chose was “engagement”, and that was the wrong proxy.

As Harari writes, this choice “… reduced the multifaceted range of human emotions … into a single catch all category: engagement … Based on this very narrow understanding of humanity … encouraged our basest instincts …”.

That is why I always say: Just because it can be measured, does not mean it matters.

When you convert your strategic goal of being happy into the tactical goal of making money, you are at risk of running into the alignment problem several years later as you cry in your Mercedes at the irreplaceable time wasted chasing cash.

While it has recently found popularity with the AI boom, the alignment problem has been called by various names in the past. And has been felt viscerally by kings whose army generals achieved tactical victories at the expense of strategic defeat.

By national leaders whose bureaucrats make decisions that were penny wise but pound foolish, by the human who was “… rewarding A while hoping for B.”.

So, when the alignment problem, runs into us – What are we trying to do?

We are trying to achieve an ultimate goal.

This “ultimate goal” has many taglines such as:

• “Bring ideas of liberty and individual freedom to the world.”
• “Organize the world’s information and make it universally accessible and useful.”
• “Bring inspiration and innovation to every athlete in the world.”

Solving the alignment problem is not easy. But the fundamental reason isn’t that getting to ultimate truths is hard, or because defining proxies is hard, or because our technology isn’t smart enough yet.

The fundamental reason is that we are trying to impose permanent order on ever-changing truth. Trying to create order by locking-in the truth is like trying to write on water.

The same truth collapses the order that sought it.

• Considering liberty as ultimate can lead to anarchy.
• Universal access to the world’s information leaks secrets that dismantle society.
• An excess of inspiration leads to exhaustion.

As Harari points out, there is “… no rational way to define that ultimate goal … executives and engineers who rush to develop AI are making a huge mistake if they think there is a rational way to tell AI that its ultimate goal should be.”.

The alignment problem is the consequence of seeking fixed order in a world governed by shifting truths.

We’re not just building AI systems. We’re offloading our alignment problems onto them. That’s the risk.

Example, the when you search something on Google, its search algorithm creates a list of web pages it thinks are most relevant to you.

This has led to the creation of several other algorithms (for instance, Semrush and ahrefs) that help businesses rank higher on the Google search results page.

This is an inter-computer reality between two computers (Google’s Search and Semrush).

Just as with intersubjective realities, inter-computer realities also carry over to the physical reality.

Like how companies will hire SEO managers to rank higher on search results pages, or as Harari explains, how YouTube video recommendation algorithms incentivised creators to create inflammatory content.

It should come as no surprise that billions across the world today are “playing to the algorithm”, i.e., we’re increasingly taking decisions based on our calculations of how they’d be seen by one or more algorithms – for both good and bad.

In a sense we’ve always been playing to algorithms. Whether it was a tribal elder’s favour, the church’s approval, or the bureaucrat’s checklist, human beings have long adapted their behaviour to fit into systems of reward and punishment.

These systems – though administered by people – functioned algorithmically: if you say X, wear Y, or behave like Z, you get social acceptance, resources, protection, or power.

The difference with H. Algoriticus, is that it faces an algorithm that has the potential to observe, evaluate, and score him at every single moment.

Harari points to the emergence of social credit systems (China’s being a popular one), where every single action of the individual at every single moment is reduced to a number that adds to or subtracts from his “Score” due to the abilities of silicon based agents.

Imbued with inter-computer myths, your Score becomes more than just a number; not very different from how, imbued with intersubjective myths, holy cities become more than just parcels of land.

Is the future outcast just a person with a low social credit score? Will tomorrow’s social purges look less like ethnic cleansing and more like algorithmic exclusion?

With silicon agents running the show, we risk a system of ever-present judgment, operating at machine speed, unburdened by empathy, and often trained on flawed or biased data.

Consider how in 2015 Google Photos’ face recognition algorithm labelled two black men as “gorillas”, or how Narayanan and Kapoor found out you can trick hiring software into giving you a higher score by putting books in your video’s background. If a hiring algorithm is biased against your application, then you may find yourself getting rejected not just for one job but all of them.

As Harari says, “Computers could be frighteningly efficient at imposing false labels on people and making sure the labels stick.”.

For example, consider a case where a social media algorithm is designed to reward photos taken at exotic locations, but a financial credit algorithm designed to penalise discretionary spending such as traveling to foreign lands and staying in 5-star hotels. Which algorithm will you perform for?

That is the future Harari is warning us against, “… humans who currently engineer computers need to accept that they are not manufacturing new tools. They are unleashing new kinds of independent agents, and potentially even new kinds of gods.”.

Finding the right balance between truth and order has become more critical than ever.

But this is exactly what we must do if we are to deserve the name we’ve given ourselves: wise man.

They may start out useless¹² or be undermined during rapid change. But because of the in-principle alignment to the most fundamental law (entropy/ thermodynamics; something dictatorships wilfully ignore) – over the long run, democracies tend to produce resilient networks.

As I wrote above, we saw this in 1960 when democracies were in chaos while dictatorships seemed to be order, but the picture flipped as we entered the 21st century. The key strength of democracies is their adaptability, and as a super-organism, it’s extremely evolutionarily adaptive.

Of course, course correction takes trial and error, as Harari points out, the Industrial Revolution led to many experiments like modern imperialism, Nazism, Stalinism before the democratic approach finally realised what not to do.

Listening is about being able to separate signal from noise. The reason you’re still able to follow a conversation in a noisy cafe is because you can separate signal from the noise.

Generative AI is rapidly getting deeper into the “infosphere”, enabling the creation of megabytes of convincing and confusing text, images, audio and video.

We are at risk of being drowned in noise and losing confidence on even legitimate signals.

Talking is about more than just having a voice; it’s about being heard meaningfully. And ironically, today, talking is too easy.

With billions of voices amplified by social media, the network is flooded with messages. Everyone is talking. All the time.

The friction that once served as a quality control mechanism – having to think, write, publish – has vanished.

When judges refer to the COMPAS AI to decide if a defendant is likely to commit the crime again (hence, decide the severity of their sentence), or when hiring managers feed a bunch of CVs to the HireVue AI to shortlist candidates for the interview round, or when healthcare professionals use the Optum Impact Pro AI to identify individuals who would benefit most from targeted healthcare services, or when an insurer uses Tractable AI to decide if your car is worth insuring – they are using AI to take decisions that impact many individuals.

Needless to say, any bias in these AI tools must be eliminated.

But to eliminate bias, you must first be able to how the AI is biased.

This is a new problem, because when human institutes take decisions that impact many individuals, we can intuitively understand many of the biases they might have – for instance, a hiring committee may be biased against women, or Asians, or African-Americans.

As Harari points out, these are age-old biological dramas that we have become evolutionarily trained to understand.

But when an AI is biased against you, it is difficult to understand the source of the bias, ironically, because the AI is designed to remove biases by considering hundreds of parameters.

The bias of the AI does not stem from a few well understood biological dramas that sway it one way or the other, but from hundreds of tiny parameters that when combined together, make it biased.

A human will not be very good at pointing out to, say, 62 of the 137 parameters that when combined together lead the AI to be biased against her.

In this way, the ability to understand is compromised as we increasingly rely on AI to run our processes – understanding how decisions are being made becomes unfathomable.

This is about how AI understands us. AI is, at the end of the day, an algorithm.

And unless designed to be, algorithms are unfeeling. If they cannot feel, then they cannot empathise.

And if they cannot empathise, then they cannot truly understand.

Cannot understand that a quick decision rejecting your insurance claim without any recourse might not just dent your finances – it might also crush your spirit. A human may pause to listen to your story, may feel a twinge of sympathy or guilt, may break a rule or take a chance. But an unfeeling algorithm has no such luxury.

That’s why “Theory of Mind” – the cognitive capacity to infer the beliefs, intentions, and emotions of others—must become a cornerstone of any serious AI design moving forward. Not because AI needs to “feel” like us, but because without this capacity, it cannot understand us. And without this understanding, there is no democracy. Only control.

Just like counterfeit money is regulated by having mechanisms that remove it, as well as, methods to identify genuine legal tender – we should do the same with AI; “… to protect trust in humans … outlaw fake humans as decisively as … fake money.”.

Regulations and laws should be passed that prohibit and penalise creating deepfakes, attempts by non-human entities to pass off as humans, unsupervised algorithms from curating public debates (the kinds that run on social media sites).

Harari makes the case for the creation of a new fundamental right available to humans, the “Right to an Explanation”.

It means that when decisions about humans are taken by machines, the humans should be given an explanation as to why that decision was taken.

This addresses the unfathomable nature of AI partly, because it still needs humans to pore over the explanation, which if genuinely given, will likely be very long and very complicated (because AI is designed to consider many factors).

As a result, this fundamental right must also be supported by the creation of expert organisations that decode the explanation and take a call on its fairness (i.e., good at bureaucracy), as well as share the finding in a way that humans can understand (i.e., good at storytelling).

A robot priest can technically perform all the steps needed to solemnise your wedding, and indeed, with much more efficiency than a human priest. But you’d likely not be in favour of such an arrangement.

Similarly, a nanny may perform every task a mother can, but to the child, it’s not just what’s being done – it’s who is doing it. The emotional substrate changes the very nature of the experience.

When it comes to meaning and connection – we want a human on the other side, despite a few mechanical or technical flaws. Jobs with high emotional resonance are not evaluated purely on competence, but on connection.

That connection can be real (a bond of love) or symbolic (a belief in tradition), but it matters enormously.

We seek recognition, ritual, meaning. And that’s not something you can program – at least, not yet.

The reason why drivers fear being replaced while priests don’t – because drivers know their role is mechanical, and priests believe theirs is meaningful.

Whether or not that’s true, that belief protects them from replacement anxiety.

And this is where automation – especially by AI – hits a wall. Not a technical wall, but a meaning wall.

So while the problem of automating process – the procedural and mechanical steps – is a technical one, the problem of automating purpose – what we’re ultimately looking to get out of the process – is not just technical.

It’s a question not just about what you want to do, but how you want to feel.

And that’s where it gets interesting. Over the last few years we have seen the emergence of AI counsellors and therapists and lovers, and we have seen their users develop real relationships with them – as Harari points out “consciousness – relationship” is a two way street.

We develop relationships with conscious entities, but we can also ascribe consciousness to entities we have relationships with.

How a child truly believes their teddy bear can feel the hurt when dropped from a height.

Or how a person can’t bear leaving their dog in the car but happily order hamburgers for lunch.

I have explored this in previous Field Notes, and there is no easy answer. The current information revolution will have humanity asking this question several times over.

Ultimately, the answer may be tautological, that humans are good at being human.

And just like sometimes one wants a physical book “just because”, despite its disadvantages versus e-readers, you may want humans just because you want them.

On a personal note, AI today is already sophisticated enough to not just read entire books, but also connect their concepts with several others. In essense, not just delivering the functional aspect of Sunchaser but with several times more efficiency.

But when I tell you that I’m a real human writing this, pouring my heart and soul into the work, and that I need your support to keep this going – you feel something inside you that you do not with an AI book summary service.

You come to a human because you want to come to a human. And that is what we’ll always be good at.

As I have mentioned earlier, what this means is, that for the first time in history a new kind of agent, a non-biological agent has entered our information networks.

The way our biological code is a combination of Adenosine, Guanine, Thymine, and Cytosine; the computer’s code is a combination of 0s and 1s.

And from these basic building blocks are made complex humans and complex AI.

Many fundamental materials have shaped history, such as iron, gunpowder, paper, coal, rubber, oil, steel, silicon and more.

Each of these materials didn’t just change technology – they reshaped power structures, economic networks, and cultural norms.

No points for guessing where I’m going with this – data is the fundamental raw material for the information revolution we’re living through.

And if AI is like a gun, then data is the gunpowder.

And quite fittingly, just as guns enabled imperialist conquests in the 18th and 19th centuries – subjugating populations, gaining vast lands and resources – Harari wonders whether something similar is at risk of happening with AI in the 21st century.

Only this time, the weapons are not muskets and cannons, but algorithms and servers.

The conquered are not just nations, but minds and behaviours.

And the spoils are not gold or spices, but data and influence.

Harari draws a parallel with how during imperialist times raw material was sent from the colonies to the headquarters and finished goods were sent back to the colonies to purchase; data will be sent from “data colonies” and algorithms will be sent back for purchase.

As a result, nations and corporations want to push their AI to serve as the backbone for crucial services in other nations.

Just like cotton or spices from India were processed in British factories and sold back at a premium, your photos, texts, searches, clicks, and biometric data are collected by platforms, fed into massive training models, and returned to you in the form of AI products – many of which require you to pay to access them.

Harari explains that just as the Iron Curtain was a result of people having different ideas on how to organise society, a similar “Silicon Curtain” will come between the creators of AI technology who will have different philosophies about how to use AI as well as code different philosophies into the AI itself.

Ideology shapes our institutions, that then shape our ideology. Therefore, it may come to pass that as countries and companies choose an AI solution (hence choosing an AI philosophy), they may go down very different paths culturally, ideologically and identity-wise, thus become “foreigners” to those who choose a different AI solution.

He presciently says, “While the web has been our main metaphor in recent decades, the future might belong to cocoons … different digital cocoons might adopt incompatible approaches to the most fundamental questions of human identity.”.

But before being fought with traditional weapons and in the open, this war will be first fought in secret – and in many ways, it is already being fought in secret (remember Russia’s role in the 2016 U.S. presidential elections).

The weapons of this secret war are not nuclear bombs but rather, logic bombs. Basically, cyber-warfare. These logic bombs are not planted on a network, they’re planted in the code of the agentic nodes themselves.

And according to Harari, unlike nuclear bombs which, happily, led to deterrence due to mutually assured destruction – logic bombs, unhappily, may lead to a temptation to strike first.

For example, an actor may insert malicious code deep within the codebase of a country’s space agency, who may not notice it until the last moment when they press the launch button and nothing happens.

It’s like having enemy spies in your country, except the spies are algorithms not people, and they sit silently inside your power grid, your hospitals, your satellites, and even your voting machines – until the opportune moment.

In such a situation, neither side can be sure about the other’s intention, even if the other may be publicly committed to playing fair. The optimum strategy then becomes offence and not defence.

Consider the example of Stanislav Petrov, a Soviet Air Defence officer who, in 1983, quite literally saved the world. On September 26, 1983, the Soviet Union’s early-warning system reported that the U.S. had launched five intercontinental ballistic missiles toward Russia.

The protocol demanded immediate retaliation – but Petrov, who was on duty at the time, refused to report the alert to his superiors.

He intuitively judged the alert to be a false alarm, reasoning that: the system showed only five missiles (inconsistent with a full-scale U.S. nuclear strike), the satellite warning system was new and possibly unreliable and the ground radar had not yet confirmed the launches.

And he was right. It was later revealed that the satellite mistook sunlight reflecting off high-altitude clouds for missile launches.

Petrov’s action prevented what could have become a nuclear exchange between the U.S. and USSR.

The danger is that they operate too fast, too precisely, and too independently – in ways humans can no longer oversee, audit, or interrupt in time.

Stanislav Petrov had minutes to assess and override the system.

That was enough.

In a fully automated defence system, decisions could be made in milliseconds – before any human even sees the alert.

And leaders should not think that by effectively managing the agent within their borders is enough, “An out-of-control AI, just like an out-of-control virus, puts in danger humans in every nation.”.

He seems disheartened as the book draws to a close, because at a time when powers should be coming together to regulate AI and, more broadly, the computer agent – the divides seem larger.

And “… despite the importance of self-correcting mechanisms for the long-term welfare of humanity, politicians might be tempted to weaken them.”.