PauseAI Meetup Opening Speech

In May 2023, shortly after the launch of ChatGPT, Geoffrey Hinton, one of the pioneers of Artificial Intelligence, publicly announced quitting his job at Google to sound the alarm about the dangers of the field he helped found.  This concern was echoed by countless other experts in an open letter asking for a global pause on AI development.  The CEOs of the top three AI labs have publicly stated that their technology could lead to nothing less than human extinction.  And a recent survey of thousands of people working in AI has suggested the same, where nearly half of the respondents believed that there is a 10% or greater chance of human extinction from AI.  And this isn’t some distant problem “future generations” will have to deal with.  Timelines for AI becoming powerful enough to be this dangerous vary wildly, but the people who are building it tend to say it could come about in just 10 years.  And these estimates are constantly getting shorter.

What on Earth is going on?!

Let’s step back and take a moment to understand how AI works.  Traditionally, computers work by following instructions (called “code”).  Those instructions can be very complicated, but they’re fundamentally not much different from a recipe you might use in cooking.  Modern AI is not like that.  It’s based on something called neural networks, which were originally designed to mimic the human brain.  You take millions of artificial neurons and connect them together.  What happens in each neuron is very simple; the complexity comes from the connections—by strengthening some and weakening others, you get different results.  Then, you train the system by feeding in data, getting results, and applying a feedback signal based on how good those results are.  This feedback causes all of the connections to change—some get stronger and some get weaker—so that, next time, the system gives you a better result.  Not only that, but when given enough examples, the system finds patterns in the data so that it can give good results for data it has never seen before.  

What is most remarkable about this whole approach is that the system gets more powerful just by making it bigger.  Add more neurons, feed it more data, run it longer.  In fact, that’s where most of the progress in AI has come from over the last several decades.  One result of building AI this way is that it is very difficult to know what the system is actually learning.  Some studies have shown that AI figures out how to recognize images in more or less the same way you might expect a human would—starting with simple shapes and working up to complex features—but the field of interpreting trained systems is lagging far behind that of creating new ones.  What’s more, AI has lots of abilities it doesn’t show unless you ask in just the right way, which is why scientists are still discovering abilities of systems that were built over a year ago.  And it is nearly impossible to know what new abilities will emerge on each new training run.

The most advanced type of AI today are Large Language Models, or LLMs.  These are what power ChatGPT.  They train by taking a piece of text, covering up part of it, and trying to guess at what got covered up.  This makes it so you don’t have to hire people to add labels to all the data.  More data is better, so they train on all the text of the Internet.  This is the main part of the process and it costs tens of millions of dollars for a single training run.  Then companies like OpenAI apply some fine-tuning and human feedback in order to make the system talk like a helpful robot assistant and also to stop it from saying things that would get the company in trouble, like giving you instructions on how to build a bomb or that it has become self aware and is suffering.  Let me say that again, AI has to be specifically trained not to say that it is conscious.

So why are people so concerned about AI?  There are a lot of reasons.  First, AI has been shown to discriminate against women and racial minorities when used for hiring and loan decisions.  People are biased, AI learns from our data, and so it has picked up our biases.  Second, AI is a strain on the environment, it uses tons of water and energy.  Coal plants have even been reopened to power new data-centers.  Third, AI can be used to create fake images and misinformation.  With under a minute of voice recording, AI could clone a child’s voice, call their parents, and say they are in trouble and need to be sent some money right now.  Be warned, scammers are already using AI in this way.  Also, there’s an election coming up.  We’re used to fake news, but AI allows it to be created on a massive scale and be tailored to each person for maximum persuasiveness.  Fourth, AI can take our jobs.  In the past, automation led to economic growth which led to new jobs.  But this relies on humans being better at learning, so that the new jobs are done by people and automation is always playing catch-up.  AI, however, intends to automate the process of learning itself.  The explicit goal of AI companies is to automate most work.  In theory, this could be a good thing—but do we really trust tech billionaires to share their wealth?  And there are countless other problems: enabling cyberattacks and the creation of bioweapons, wrecking the education system…but I’m trying to keep this short.

All of this is bad enough, but what about those statistics I mentioned at the beginning about experts being worried that AI could kill us all?  For that to make sense, we need to look beyond AI as it exists today towards the AI of the future that is embodied in the world, learning from experience, and making decisions on its own.  Make no mistake, AI companies are working on all of these things and making progress.  AI is already being used by militaries around the world, including autonomous drones in Ukraine, robot dogs with machine guns in the Chinese military, and Israel using AI to identify Hamas in Gaza to decide who lives and who dies.

But AI is not only dangerous because it empowers dangerous people, it also has the potential to become dangerous on its own.  The basic problem is that AI works by being given a goal and then it figures out the best way to achieve that goal with no way to know whether its strategy involves nasty side effects.  Train an AI to play a video game and it finds ways to cheat.  Train it to write good answers to questions and it tells people what they want to hear.  Build a social media algorithm to maximize engagement and it discovers that the best way to do that is to make people angry and the best way to make people angry is to make them politically polarized in a way that undermines democracy.  Now imagine the kind of systems that drive social media, make them a thousand times more powerful, task them with managing the economy and the military, speed up the decision-making process to where human oversight is just a rubber stamp and what could possibly go wrong?

One thing that could go wrong is that the AI discovers that the best way to achieve whatever goal we have given it is to accumulate power and resources, stop anyone from shutting it down…and that it’s actually better off without us.  Now it could be that if we give AI the right goals, train it the right way, give it the right guardrails, then we will be able to trust it even after it becomes too intelligent and too powerful to control.  This is called the “alignment problem,” it is nowhere close to being solved, it might not be solvable at all, but there is no way to know because the people with money are not really trying.

I want to remind you that all of the concerns I have been speaking about have been raised by the people who are building AI.  Not all of them, mind you.  There are some who brush off the immediate concerns as bumps in the road to progress...as if the real lives being disrupted didn’t matter.  There are some who dismiss AI takeover as science fiction…as if being able to have a conversation with a computer wasn’t sci-fi.  And there are some who try to pretend that the immediate and future dangers are somehow opposed to each other…knowing that if the public is divided, we won’t be able to fight back.  But where there is disagreement there is uncertainty and where there is uncertainty—and the stakes are this high—the only sane response is caution.

So if so many people in AI believe their work is dangerous, why don’t they stop?  A few of them believe that AI represents the next stage of evolution and we should not resist being replaced.  Others believe that AI could be used to reverse aging and grant them eternal life.  But the most common reason is that they don’t trust anyone else to build AI safely.  “AI is dangerous, but I get it, I know how to make it safe.  So I need to build it first, because if I don’t, someone else will.”  And then everyone else is saying the same thing and the result is an out of control race.
Those of us at PauseAI understand that the tech industry cannot be trusted to regulate itself and so we are calling for a global pause on new, large-scale training runs until there is some kind of democratic oversight on how or whether to proceed.  There are only a few organizations in the world that can build cutting-edge AI so this is a natural choke point.  And the pause needs to be global so that one country slowing down doesn’t become irrelevant if someone else races ahead.

Our strategy is in three parts.  First, we lobby our local politicians about the risks of AI, the need for a pause, and urge them to take the lead in getting the diplomatic process started.  Second, we build a mass movement of regular people to speak out and pressure those politicians to listen to us.  And third, we build up an organizational structure, understanding of the problem, and set of realistic solutions so that when the reckless decisions of the AI companies starts triggering a larger public backlash, we are ready to direct that energy somewhere that is actually helpful.  PauseAI is an international group of citizen volunteers primarily organized through a website called Discord but to really have an impact we need to start building connections with real people, face-to-face.  And that’s where you all come in.
​
My hope with inviting you here—besides having an important conversation about our future—is to set up a local chapter that engages in ongoing activism.  This includes letter writing campaigns to Congress and local newspapers and empowering you to make your voice heard.  This includes starting conversations with your friends and neighbors and building community.  And this includes listening to your voices about what should be done, so we don’t become what we are opposing: another small group of people deciding what’s best for everyone else.  Thank you all for being here.