AI – The Alien Among Us

AI is everywhere, and it seems to know everything. From solving climate change to curing cancer, it promises to overcome humanity’s most pressing challenges. But there’s one big problem: we hardly know anything about it.

That’s not an accident – it’s by design

In 2024, the creators of the AI model Alphafold2 were awarded the Nobel Prize in Chemistry. Alphafold2 had succeeded in predicting the structures of over 200 million proteins, accomplishing more in a few months than scientists had in the previous 150 years.

There was only one catch: not even Alphafold2’s creators knew how it worked.

AI models like Alphafold2 are built to learn by themselves. Once they have figured out how to do something, they are essentially black boxes: billions of interconnected parameters whose inner logic is impenetrable.

The world’s top experts acknowledge that AI remains a mystery.

As veteran AI researcher Melanie Mitchell says: “It’s not at all easy to understand how these networks make their decisions.”² After training a very small neural network to recognize written numbers, she realized she had no idea why it was so effective.

And the largest AI models now are tens of millions of times larger and much more powerful than Mitchell’s simple experiment.

Historically, this isn’t so unusual. When Alexander Fleming invented penicillin, he didn’t know how it worked³. When Marconi helped develop the radio, he had the wrong idea about electromagnetic waves⁴.

Scientific understanding often lags behind practical knowledge.

But in the case of AI, this pattern is especially concerning. This is because:

1. AI is not just a tool. It is an agent capable of making decisions. And it is already playing a role in nearly every major aspect of human life. AI determines who gets loans, who gets hired, who gets health treatment, and who gets sent to prison. In warzones, it is deciding who gets killed⁵.
2. We are actively seeking to make AI exceed human intelligence. We want it to understand what we cannot understand and do things that we could never even imagine.

Which brings us to…

How does it feel to be left behind?

Every other species on the planet has gone through this. They watched a weak, hairless primate become Earth’s dominant species in a very short amount of time.

Humans have never had to live through such a transition, but we may be about to.

Major transitions in intelligence create an insurmountable gap between those who master a transition and those who don’t. Today, animals exist on a planet dominated by human beings.

The best they can hope for is to become our pampered pets. Should another major transition occur, could humans also be left behind?

To figure out how close we are to such a scenario, we need to ask:

How do major transitions in intelligence happen?

The recipe for human intelligence has three interrelated ingredients:

• Cognitive capacity⁶
• Information
• Energy

Increasing the amount of one ingredient can significantly affect the others. Once this runaway process begins, it can lead to irreversible and profound changes.

Here is how it more or less played out in human evolutionary history:

No other human species survived this transition. Not even the Neanderthals, who had larger brains than ours.

Since then, an information and energy cycle has taken over. Rather than relying on increases in brain size, our cognitive capacity was supplemented with tools for thinking and communicating (writing, drawing, mathematics, telescopes, the internet, etc.)⁷.

The latest technology under development in that cycle is AI.

But AI has its own intelligence cycle. It has similar ingredients:

• Computational capacity⁸
• Data
• Energy

As with human intelligence, each of these ingredients affects the others. More energy and computational capacity allow for the deployment of larger datasets. In turn, more complex algorithms – armed with massive amounts of data – can uncover more efficient ways to use energy and computational capacity.

Better AI models attract more resources, more resources allow for bigger AI models, and bigger AI models lead to better performance.

The results of this feedback loop have been dramatic. Between 2012 and 2023, the amount of computational capacity dedicated to AI increased by a factor of 100 million.⁹ In the same time period, the complexity of the largest algorithms increased by around 1,000-fold.¹⁰ According to one estimate, by 2030, the data centers that power AI could account for over 20% of global energy demand.¹¹

In the history of our species, gains in intelligence were fueled by the meat of mammoths and the trunks of trees burned for fire. Today, AI is feeding on the conversations of billions of people and the plentiful electricity we generate.

But whereas the human transition occurred over a period of half a million years, a leap in artificial intelligence could happen any day now. We are living through an AI revolution rather than an AI evolution.

Researchers typically compare AI’s performance on certain tasks to human capabilities.¹² But this is misleading. AI is not like human intelligence – it is an alien intelligence.

It is not constrained in the same ways that we are. AI can read every book ever written without pausing to eat or sleep. Looking at AI models today and trying to guess what they will be able to do in the future is like looking at a primate half a million years ago and predicting that its descendants would send probes into space and split the atom.

Biology is slow. Culture is fast. AI moves at warp speed.

Which is a problem because…

What if it happened tomorrow?

Imagine the scene – A tech giant announces: “superintelligent AI has finally arrived”.

The superintelligence automates research and development, racing ahead of every other AI lab. Competition is futile.  The AI uses its capabilities to engineer flash crashes on the stock market, causing rival companies to collapse. As government regulators move to contain the tech juggernaut, they find that they are too reliant on its services. Every piece of digital infrastructure can only be secured with its help.

A new phrase is coined in the media to describe this AI’s status:
“Too intelligent to fail.”

Back to reality – Anthropic’s CEO Dario Amodei has described the next phase of AI as a “country of geniuses in a data center.” But these geniuses won’t be like Isaac Newton, Marie Curie, or Albert Einstein. They won’t think like them or face similar limitations.

Who should decide how this country of alien geniuses can be used?

Weapons manufacturers cannot sell to whoever they wish. What they manufacture has too much power. In the same way, decisions about advanced AI cannot be left solely to company executives and shareholders. It is a matter of public safety.

We are all shareholders in the future of AI.

So what can we do about it?

Currently, there is no international authority to regulate AI and ensure its safe use around the world.

In the United States, efforts to reduce the risks of AI have been sacrificed in the name of winning the AI arms race. In Europe, the enforcement of new AI regulations remains untested. In China, concerns about AI safety are a relatively recent development and at risk of being sidelined by strategic competition with the United States.

The leaders of the AI revolution recognize the dangers that AI presents. Yet they are charging forward. To most observers, it looks like an AI arms race is inevitable.

But people have come together before to regulate a disruptive and dangerous technology. In the 1950s, efforts to prevent the spread of nuclear weapons resulted in the formation of dedicated organizations that monitored nuclear materials and promoted the peaceful use of nuclear technology. Thanks to these efforts, humanity has avoided nuclear war and benefited from nuclear power.

Containing the risks of AI will require international collaboration.

An institution tasked with promoting the safe and peaceful use of AI must prioritize three urgent issues:

1. We need to understand how AI works

Progress in science means increasing understanding. For AI companies, progress means increasing profits.

The U.S. National Science Foundation invests about $700 million a year in AI research.¹³ OpenAI, by contrast, has attracted an investment of $500 billion over four years.¹⁴

Thanks to the massive resources at their disposal, AI companies can hire the best AI researchers and afford the most computational power. This stifles scientific research. If we’re going to find out how AI really works, we need our scientific knowledge to catch up to technical achievements.

We need CERN for AI

Politically, international collaboration is at a low point. But scientists are working across borders more than ever before. To pave the way for a safe and prosperous AI future, we should leverage these global scientific networks.

In 1953, 11 countries began the foundation of the European Organization for Nuclear Research (CERN). The CERN Convention said that the organization will be devoted to “nuclear research of a pure scientific and fundamental character”. CERN’s relatively large budget (about $1.4 billion¹⁵) and ambitious goals have since led to groundbreaking discoveries that have accelerated the development of quantum computing and transformed our understanding of physical reality.¹⁶

A similar international organization¹⁷ could fill the funding gap and attract brilliant scientists to study the fundamental questions of AI research and develop less risky AI technologies that benefit humanity. This body would attempt to develop a general theory to explain the emergent properties of AI models. They would also pursue alternative AI designs that would result in safer, more predictable AI models. Such technologies could serve as a benevolent ‘AI police’ to monitor the growing population of AI agents–something that we can no longer do ourselves. 

This organization doesn’t need the vast sums of money required to build the biggest data centers. It only needs to attract the brightest minds. Basic research for AI will benefit everyone, and every nation should help to build a home for it.

Let’s learn how to build safer AI before it takes us by surprise.

2. We need to anticipate what AI can do

AI evaluation research is still in its alchemy phase.

Alchemy was a practice without a theory. Between the 16^th and 18^th centuries, the transition from alchemy to chemistry involved making testable predictions with theoretical rigor. When it comes to evaluating AI models, we’re still in the stage of magical thinking. We still don’t know what we should be measuring.

To anticipate what AI can do, we need better tools to evaluate its capabilities. The performance of AI models on certain tasks is routinely compared to that of human beings.¹⁸ But shouldn’t we be measuring AI in a way that is not tethered to our own ways of thinking? If we measure AI’s performance against our own, we risk missing the things that make its intelligence unique.

If we don’t measure the right things, we could create superintelligence without even knowing it. A CERN for AI would develop a paradigm of AI evaluation that would offer theoretically-rigorous and reliable assessments of a given model’s capabilities. And it would look out for concerning signs of autonomy that would allow an AI agent to escape human control.

We need an institution dedicated to testing AI for dangerous behaviors and communicating those discoveries to the public.

AI evaluation needs to become a science.

3. We need to agree on a legal framework for AI

If the warning lights come on, what should we do?

Before the next AI breakthrough arrives, we need a legal framework in place. If you think scientific understanding is slow, try legislating. But the problem of AI demands international cooperation – if a superintelligent AI gets out in one part of the world, it won’t respect national borders.

Even at the height of the Cold War, countries still worked together to try and mitigate the threat from nuclear weapons. The International Atomic Energy Agency, established in 1957, offers legislative assistance on nuclear issues to all 180 of its member states. The organization establishes safeguards for the Nuclear Non-Proliferation Treaty and verifies adherence to it. This has helped limit the number of nuclear-armed countries to just nine.

We can’t afford to allow rogue superintelligent AI to emerge even once

A CERN for AI would give world leaders and citizens everywhere the tools to steer AI development towards peaceful purposes. The decisions about how to navigate the next major transition in intelligence should be made by all of us. The current political climate is poor, but we have no choice.

Ronald Reagan said that the only thing that could unite humanity is the sudden appearance of aliens. Well, an alien intelligence has arrived.