By Jeff Caruso | April 29, 2024

In a 2012 essay for Aeon, physicist David Deutsch wrote of artificial intelligence: “I cannot think of any other significant field of knowledge where the prevailing wisdom, not only in society at large but among experts, is so beset with entrenched, overlapping, fundamental errors. Yet it has also been one of the most self-confident fields in prophesying that it will soon achieve the ultimate breakthrough.”

The self-confident prophesying Deutsch wrote of has since continued, except it is now accompanied with the warning that a rogue artificial intelligence could end life on Earth as we know it, and the technology should be categorized as an existential threat.

Artificial intelligence has come into the spotlight since 2022’s release of OpenAI’s ChatGPT, which was followed by several other text and image generation models. The applications have already been incorporated into businesses, healthcare settings, and even military operations. When given prompts, these systems respond by statistically predicting words or other forms of output based on the massive amounts of data they were trained on. In other words, they cannot think or rival human cognition. They are, at least for now, just mathematical models.

While generative AI can wreak havoc in many ways—such as amplifying existing biases or dispensing misinformation at scale—it’s not an existential threat any more than computer code is. It’s merely a tool that could, like many other tools, facilitate a bad outcome. It is, therefore, imperative that the public and policymakers understand this distinction to devise appropriate regulations that address the real risks of these applications and not fictional ones.

The thermonuclear bomb would never have been invented had it not been for an innovation in computing called the Von Neumann architecture whereby a computer could store data and run a program in memory. To successfully achieve that innovation, a single mathematical operation on a computer ironically named MANIAC at Los Alamos National Laboratory ran non-stop for 60 days. Which, then, was the weapon of mass destruction, or existential threat, the thermonuclear bomb, or the innovation in computing that brought it into existence? Obviously, it was the former.

Even in the case of lethal autonomous weapons or AI-controlled drone swarms, the designation belongs to the weapon, not to the tech that helped create, or control, it. But when the discussion centers on AI as an existential risk, the proponents are not speaking of an AI-controlled weapons platform. They’re speaking of a superintelligence—an entity whose cognitive abilities surpass human intelligence—that could wipe out humanity simply by doing what it was asked to do.

According to AI philosopher Nick Bostrom and computer scientist Geoffrey Hinton, the capabilities of a superintelligence would be exponentially greater than a human being in every way. If life were a game like the one in the television show Survivor, an artificial general intelligence (systems that could rival human intelligence) would outthink, outwit, outlast, or outright kill humans in short order because of two theoretical dilemmas: misalignment and instrumental convergence. Misalignment is what happens when an AI completes a task or operation in a way that is detrimental to humans or the environment. Instrumental convergence theorizes that an AI, when given a task, would maximize all available resources towards its completion.

One famous example of misaligned AI and instrumental convergence is the paperclip maximizer thought experiment that Bostrom came up with in 2003. “Suppose we have an AI whose only goal is to make as many paper clips as possible. The AI will realize quickly that it would be much better if there were no humans because humans might decide to switch it off. Because if humans do so, there would be fewer paper clips. Also, human bodies contain a lot of atoms that could be made into paper clips. The future that the AI would be trying to gear towards would be one in which there were a lot of paper clips but no humans.”

Another theory, the Orthogonality Thesis (a term Bostrom took from mathematics and applied to artificial intelligence), says that distinct AIs with different goals will share similar values such as self-preservation, cognitive enhancement (software), technical enhancement (hardware), and resource acquisition (time, space, matter, and free energy).

These thought experiments and theories recall HAL, the antagonist and superintelligence featured in 2001: A Space Odyssey, a collaboration between iconic film director Stanley Kubrick and writer Arthur C. Clarke.

Kubrick, who used a familiar science fiction trope—that humans can no longer control the technology they created—provided some background to HAL in a 1969 interview, the year after the movie was released: “One of the things we were trying to convey in this part of the film is the reality of a world populated—as ours soon will be—by machine entities who have as much, or more, intelligence as human beings, and who have the same emotional potentialities in their personalities as human beings.”

In the movie, astronaut Dave Bowman, the protagonist, asks HAL to open the pod bay doors. To that, the supercomputer eerily responds, “I’m sorry, Dave. I’m afraid I can’t do that…this mission is too important for me to allow you to jeopardize it.”

That science fiction template of a runaway machine found its way into science as well. In 1964, RAND, a non-profit policy think tank, hired Hubert Dreyfus, a professor of philosophy at MIT, to review the work of AI researchers Allan Newell and Herbert A. Simon. Newell was one of AI’s earliest scientists who in 1957 predicted that in 10 years a digital computer would accomplish three remarkable achievements: become a world champion in chess, discover a new mathematical theorem, and write music that will be acclaimed by critics. That level of optimism energized his work at RAND and, since he and Simon’s research presumably had defense implications, the think tank was looking for an objective review by an outside expert. In 1965, Dreyfus shared his findings in Alchemy and Artificial Intelligence and refuted the thesis that human and computer intelligence operate in similar ways. Both Newell and Simon called the paper nonsense and fought against its release. They eventually lost that battle, and Dreyfuss’s analysis became one of RAND’s most popular papers.

Artificial intelligence research has come a long way since then. Today, the world is transfixed by what AI can do. Google DeepMind AlphaGo’s momentous victory against World Go champion Lee Sodel in 2015 was instrumental in Elon Musk and Sam Altman’s decision to launch OpenAI with the intention of achieving the benefits of artificial general intelligence safely while mitigating the existential risks. ChatGPT became the fastest growing app in history with 100 hundred million active users just two months after its release. Soon, AI startups popped up everywhere as venture capitalists dumped billions of dollars into new players including Anthropic, started by former OpenAI employees that raised $7.3 billion in its first year. Everyone was, and still seems to be, drinking the Kool-Aid and the AI innovation race is on.

Despite all that investment and progress, there is no evidence AI is closer to achieving superintelligence than it was in 1965 when Dreyfus wrote: “An alchemist would surely have considered it rather pessimistic and petty to insist that, since the creation of quicksilver, he had produced many beautifully colored solutions but not a speck of gold; he would probably have considered such a critique extremely unfair. But if the alchemist had stopped poring over his retorts and pentagrams and had spent his time looking for the true structure of the problem, things would have been set moving in a more promising direction.”

Predictions of future catastrophe rely upon the ability of AI to apply knowledge it learned in one area to problem solving in a completely different area that it hasn’t been trained on. This broad generalization capability is the key ingredient to achieving human-level intelligence. That not only hasn’t happened, but there’s no clear path to achieving it.

The AI must also be strongly goal oriented to the point where it will take independent action to prevent any interference in the achievement of its objective. There’s no evidence that such a thing is even feasible.

Additionally, any suggestion of a rogue AI showing ill-intent requires that it be conscious and self-aware. That is the stuff of alchemy and other esoteric magical traditions, not science.

In Why general artificial intelligence will not be realized, Norwegian physicist and philosopher Ragnar Fjelland points out that Dreyfus’s critiques still hold today; that human intelligence is partly transmitted in ways that are not taught but are communicated tacitly; ways that cannot be duplicated using algorithms or code.

Turing Award winner Yann LeCun, the Chief AI Scientist at Meta and the Silver Professor of the Courant Institute of Mathematical Sciences at New York University explains the problem this way: “A 4-year-old child has seen 50x more information than the biggest LLMs [large language models] that we have.” If you do the math, that’s 20 megabytes per second through the optic nerve for 16,000 wake hours, and that doesn’t include how much data is received through the other senses.

Considering the lack of any substantive evidence supporting the existential risk theory, it’s puzzling why many in the fields of computer science and philosophy appear to believe AI is an existential threat. It turns out, there aren’t that many who have bought into the theory. A recent poll of more than 2,000 working artificial intelligence engineers and researchers by AI Impacts put the risk of human extinction by AI at only five percent.

Also, despite the fact that thousands of experts signed petitions on AI risk, it doesn’t necessarily mean they agree with those statements, as was discovered by two MIT students who took a closer look at the Future of Life Institute’s 2023 open letter to pause AI. The undergraduate students, Isabella Struckman and Sofie Kupiec, discovered that “very few claimed to sign because they agreed with all or even most of the letter.”

These concerns might have gained traction because AI safety is an extremely well-funded segment of the tech industry, and developers have an interest in securing a voice on AI policy in the United States and European Union, especially when it comes to future regulation of the industry. Fear mongering tends to be more successful and work faster than objective reasoning in convincing an underinformed group of people, which legislators can be, to adopt policy recommendations. In this case, the AI safety lobby hopes governments regulate the use, rather than the development, of artificial intelligence. In other words, the lobby wants to put the onus on the user, not on the developer. If that sounds familiar, it’s because the tech industry has been beating the same drum for the past 40 years: “If you regulate us, you hurt innovation. We’ll just have customers accept an end-user licensing agreement and all will be well.”

The specter of a rogue superintelligence is just a distraction by the tech industry to stave off the regulations that they know are coming their way.

The Bulletin elevates expert voices above the noise. But as an independent nonprofit organization, our operations depend on the support of readers like you. Help us continue to deliver quality journalism that holds leaders accountable. Your support of our work at any level is important. In return, we promise our coverage will be understandable, influential, vigilant, solution-oriented, and fair-minded. Together we can make a difference.

Keywords: AGI, AI, ChatGPT, artificial intelligence, existential threat, regulation, superintelligence, tech
Topics: Artificial Intelligence, Disruptive Technologies

It’s becoming clear that with all the brain and consciousness theories out there, the proof will be in the pudding. By this I mean, can any particular theory be used to create a human adult level conscious machine. My bet is on the late Gerald Edelman’s Extended Theory of Neuronal Group Selection. The lead group in robotics based on this theory is the Neurorobotics Lab at UC at Irvine. Dr. Edelman distinguished between primary consciousness, which came first in evolution, and that humans share with other conscious animals, and higher order consciousness, which came to only humans with the acquisition… Read more »

Jeff Caruso is a security researcher and author. His book Inside Cyber Warfare, now in its third edition, will be published this fall by... Read More

Copyright © 2024 Bulletin of the Atomic Scientists. All rights reserved. Terms of Use Privacy Policy

1307 East 60th Street, Chicago, IL 60637 | 773.702.6308

Introduction