Godfather A.I. have some remorse (2023)

This transcription was created using speech recognition software. Although it has been checked by the transcribers, it may contain errors. Please watch the episode audio before quoting this transcript, and if you have any questions, please email transcripts@nytimes.com.

From The New York Times, I'm Sabrina Tavernise. And this is the Journal.

[MUSIC]

As the world begins to experience the power of AI, there has been a debate about how to mitigate its risks. One of the sharpest and most urgent warnings came from the man who helped invent this technology. Today my colleague Cade Metz talks to Geoffrey Hinton, who is considered by many to be the godfather of AI.

It's Tuesday, May 30.

Cade, welcome to the show.

Happy to be here.

A few weeks ago, you interviewed Geoffrey Hinton, the man considered by many to be the godfather of artificial intelligence. Aside from the obvious fact that AI is taking control of all conversations all the time, why talk to Geoff now?

I've known Geoff for a long time. I wrote a book about the 50-year development of ideas that are now driving chatbots like ChatGPT and Google Bard. And it could be argued that he is the most important person in the development of artificial intelligence in the last 50 years. And in the midst of all these chatbots, he emailed me and said, "I'm leaving Google and I want to talk to you, and that he wants to discuss where this technology is going, including some serious concerns."

Who better to talk to than the godfather of AI?

Exactly. So of course I got on a plane and flew to Toronto -

- sit at the dining table and discuss.

Geoff is a 75-year-old Briton, educated in Cambridge, currently living in Toronto. He has been working there since the late 1980s. He is a university professor.

So how did Geoff become the godfather of artificial intelligence? Where does his story begin?

It starts in high school. He grew up the son of a scientist, but he always tells a story about a friend describing the theory of how the brain works.

This friend was talking about how the brain stores memories and how he felt it was storing those memories as a hologram. The hologram is not stored in one place. It is divided into small pieces and then spread on a piece of film. And this friend believed that the brain stores memories the same way—that it breaks those memories down into pieces and stores them in the brain's network of neurons.

It's actually very beautiful.

This is it.

This piqued Geoff's interest. Since then, he has spent his entire life trying to understand how the brain works.

So how does Geoff begin to answer the question of how the brain works?

So he goes to Cambridge and studies physiology, seeking answers from his professors. Can you tell me how the brain works? And his physiology professors can't tell him.

And no one can tell him how the brain works.

We still don't have detailed information about how the neurons in our brain communicate with each other when we think and learn.

It then moves to a relatively new field called artificial intelligence.

[PLAY MUSIC]

The field of artificial intelligence was created in the late 1950s by a small group of scientists in the United States. His goal was to create a machine that could do everything the human brain can do. And at first, many of them thought they could build machines that would act like a network of neurons in the brain - what they called artificial neural networks.

But after 10 years of work, progress was so slow that they decided it was too difficult to build a machine that worked like neurons in the brain. And they gave up on that idea.

So they thought very differently about artificial intelligence. They embraced what they called symbolic artificial intelligence.

You'd take everything you and I know about the world and put it on a list of rules - like you can't be in two places at once, or if you're holding a cup of coffee, you're holding the end open. The idea was that you would write out all these lines step by step, line of code line by line of code, and then run it on the machine. And that would give him the power that you and I have in our own brains.

So basically tell the computer all the rules that govern reality, and the computer makes decisions based on all those rules.

Right.

But then Geoff Hinton shows up in 1972 as a student in Edinburgh. And he says wait, wait, wait. It will never happen.

[LAUGHS]: There are many rules.

You will never have the time, patience and personal power to write down all these rules and put them in a machine. No matter how long you put it off, he says, it's not going to happen. And by the way, the human brain doesn't work that way. That's not how we learn.

In this way, it returns to the old idea of ​​​​a neural network, which was previously rejected by other AI researchers. And he says that's how we should build machines that think. We let them learn from the world the way people learn.

So instead of giving the computer a set of rules like the other guys did, you're giving it a handful of information. The idea was that the computer would gradually figure out how to make sense of it all, just like the human brain.

You would give examples of what is happening in the world. I looked at these examples and looked for patterns in what was happening in the world and drew conclusions from these patterns.

But Geoff embraces an idea that has been largely rejected by much of the AI ​​community. Did he have any evidence that his approach would actually work?

His only evidence was that this was essentially how the human brain worked.

At the time, many of his peers thought he was stupid for trying.

Okay, so what happened next?

Then, after graduation, Geoff moves to the United States. He is a postdoctoral researcher at the University of California. And he starts working on an algorithm, a piece of math that could make his idea a reality.

And what exactly does this algorithm do?

Geoff basically builds the algorithm on the image of the human brain. Remember that the brain is a network of neurons that exchange signals. That's how we learn. So we see. That's what we hear. What Geoff did was so revolutionary that he recreated this system on a computer. He created a network of digital neurons that exchanged information, much like the neurons in the brain.

So the question he wanted to answer so many years ago - how the brain works - he answered only for computers, not for humans.

Normal. He built a system that allowed computers to learn on their own. In the 1980s, this type of system could learn in small ways. He couldn't learn the complicated ways that could really change our world. But fast forward a good three decades. Geoff and two of his students built a system that really opened the eyes of many people to the possibilities of this kind of technology.

He and two of his students at the University of Toronto have built a system that identifies objects in photographs. A classic example is a cat. They took thousands of pictures of cats and put them into a neural network. By analyzing these photos, the system learned to recognize the cat.

He identified patterns in these photos that define a cat's appearance: the tip of the whiskers, the curve of the tail. Over time, by analyzing all these photos, the system could learn to recognize a cat in a photo that it had never seen before. They could do this not only with cats, but also with other objects - flowers, cars. They built a system that could identify objects with a precision no one could have imagined.

So this is actually image recognition. Normal? This is supposedly why my phone can search for photos of my family and return entire photo albums of just my husband or just my dog ​​and cuddling or beach photos.

Normal. All Geoff and his students did in 2012 was publish a research paper describing this technology and its capabilities.

This has led to a rush for this technology in the tech industry.

There was a literal auction of Geoff and his two apprentices and their services.

Part of the auction was Google, Microsoft - another giant of the world of technology - Baidu, often called the Chinese Google.

For two days, they bid on the services of Geoff and two of his students to the point that Google paid $44 million for the three people who had never worked in the tech industry.

What is Geoff doing at Google after this service war?

He is working on increasingly powerful neural networks. You see this technology making its way into all kinds of products, not just Google, but the entire industry.

This is what drives Siri and other digital assistants. When you speak commands to your mobile phone, the neural network allows it to recognize what you are saying. When you use Google Translate, it uses a neural network for that. There are many things we use today that use neural networks to work.

So we can see that Geoff's idea is really changing the world by using things that we use all the time in our daily lives without even thinking about it.

Absolute. But this idea, on Google and elsewhere, is also used in situations that make Geoff a little uncomfortable. The main example is what we call Project Maven. Google took a job with the Department of Defense and applied the idea to trying to identify objects in drone images.

Humming.

If you can identify the objects in the drone footage, you can build a tracking system. If you combine this technology with a weapon, you have an autonomous weapon. This caused concern among Google employees at the time.

Geoff never intended to use his work for military purposes. He raised these concerns with Sergey Brin, one of the founders of Google, and eventually Google withdrew from the project. And Geoff was still working at the firm.

But at the same time, the industry started working on a new application for the technology, which worried him even more. He started applying neural networks to what we now call chatbots.

Basically, companies like Google have started feeding massive amounts of text into neural networks, including Wikipedia articles, chat logs, and digital books. These systems started learning how to build a language the same way you and I built this language.

For example autocomplete in my email.

Of course, but on a large scale.

As they added more and more digital text to these systems, they learned to write like humans. This is how chatbots such as ChatGPT and Bard were born.

And what made Geoff think of all this? Why was he so worried?

Well, I still feel that these systems are not as powerful as the human brain. And they are not.

But in other ways, he realizes that they are much more powerful.

How exactly stronger?

Geoff thinks so.

You and I have a brain that can learn a certain amount of information. And after reading this information, I can pass it on to you. But it's a slow process.

With these neural networks, as Geoff notes, they can be connected. A small network that can learn some information can be connected to all sorts of other neural networks that have learned from other parts of the internet. And these can be combined with other neural networks that learn from additional parts.

Everyone can learn together and exchange what they have learned with each other in an instant.

This allows them to learn from all over the internet. You and I can't do it individually, and we can't do it together. Even though each of us is learning a piece of the internet, we cannot easily share what we have learned. But machines can. Machines can work in ways that humans cannot.

[UNHOLY MUSIC]

What does all this mean for Geoff?

In a way, he considers it the pinnacle of his 50 years of work. He always assumed that if you rolled more dice in these systems, they would learn more and more. He had never imagined that they would learn so much and become so powerful so quickly.

we're coming back.

APPROX. So what exactly is Geoff afraid of when he realizes that the AI ​​has this turbo ability?

He worries about many different things. At the lower end of the scale are hallucinations and prejudices. Scientists talk about these astonishing systems, which means they make everything up. When you ask a chatbot for a fact, it doesn't always tell you the truth. And it can react in a biased way towards women and people of color.

But, as Geoff says, these problems are just a by-product of the way chatbots mimic human behavior. We can tell stories. We can be biased. And he believes that all this will soon become clear.

When he starts to say that these systems are getting scary, he is primarily addressing the issue of disinformation.

These are systems that enable organizations, nation-states and other malicious actors to spread disinformation on a scale and effectiveness not possible in the past.

They can also create photorealistic images and videos.

Deep fakes.

Right.

He, like many people, fears that the internet will soon be flooded with fake texts, fake images and fake videos, to the point where we won't be able to trust anything we see online. So this is a short term problem. There is also another medium-term problem, which is job losses. Today, these systems often complement employees. But he worries that as these systems become more powerful, they will replace jobs in large numbers.

And what are the examples?

No wonder that Geoff - a computer scientist - refers to the example of computer programmers. These are systems that can write computer programs themselves.

These programs today are not perfect. Programmers tend to take what they produce and incorporate the code into larger programs. But as time goes on, these systems will get better and better at the work people are doing today.

And you're talking about professions that, so far, are not perceived as technology-sensitive. Normal?

Exactly. For years, it was thought that artificial intelligence would replace the work of workers - that robots, physical robots, would perform manufacturing and sorting tasks in warehouses. But what we're seeing is the emergence of technology that can replace white-collar workers, people working in offices.

Mm-hmm.

So this is medium term. Then there are more long-term concerns. And let's not forget that as these systems get more powerful, Geoff becomes increasingly concerned about how this technology will be used on the battlefield.

He casually calls them robotic soldiers.

Like basically soldiers who are robots?

Yes, actually soldiers who are robots.

He talks about how this technology can lower the threshold of war - making it easier for nation states to go to war.

So it's like drones. People who kill sit in an office with a remote control, away from people who die.

No, it's actually a step further. It's not people who run the machines. More and more machines make their own decisions. That's what Geoff's worried about.

He worries that if we give machines specific goals - asking them to do something for us - when they try to achieve those goals, they will do things that we don't expect them to do.

So he worries about unintended consequences.

Unintended consequences. And here we begin to venture into the realm of science fiction.

We've seen this happen in books and movies for decades.

If anyone has seen the great Stanley Kubrick movie "2001"—

Mm-hmm.

We've seen HAL 9000 spiral out of control because of the people who made it.

This is the scenario, believe it or not, that involves Geoff. And he's not the only one.

In fact, robots are taking over.

Exactly.

Whether these technologies are used on the battlefield, in the office or in a computer data center, Geoff is concerned that people will increasingly hand over control to these systems.

So it seems unlikely to be honest. But okay, let's play it like it's not. What would this doomsday scenario be? Paint a picture for me.

Think about it in simple words. If you ask the system to make money for you - which, by the way, people start to do - can you use ChatGPT to make money in the stock market? When people do this, think of all the ways you can make money. And think of all the ways it could go wrong. That's what he's talking about.

Remember, these are machines. Machines are psychopaths. They have no emotions. They have no moral compass. They do what you ask them to do. Make money for us? Okay, we'll make money for you. Maybe you hacked into a computer system to steal that money.

If you have Central African oil futures, you may be able to cause a revolution to raise the price of these futures and make money from it. That's the scenario Geoff tells me and many others I've talked to. However, at this point, I must say that, like today, this is hypothetical. A system like ChatGPT will not destroy humanity.

[LAUGHTER]:

Punt.

Bom.

And when you talk to a lot of experts in the field about it, they get annoyed when you mention it. And they point out that today it is not possible. And I actually confronted Geoff about this.

So basically he's arguing that this is Pandora's box, that it's been opened, and since people are people, they're going to want to use what's inside. But I guess I wonder, I mean what do you think here - what importance should we attach to his warnings? Yes, it has a certain level of authority, godfather AI and so on. But he is surprised by his past evolution and may be wrong.

Normal. There are reasons to trust Geoff and there are reasons not to trust him. About five years ago, he predicted that all radiologists would be obsolete, but that hasn't happened. You can't just take everything he says. I want to emphasize that.

But you have to remember that this is someone who lives in the future. Since the age of twenty, he has been living in the future. Then he saw where these systems were going, and he was right. Now he is looking to the future again to see where these systems are headed. And he's afraid they're going where we don't want them to go.

Cade, what steps do you propose to take to ensure these doomsday scenarios never happen?

Well, he doesn't believe that people will just stop developing this technology.

This technology is developed by some of the largest companies in the world - public companies created to make money. Now they compete.

And he sees this happening not only to companies but also to governments in other parts of the world.

So in a way it's like a nuclear weapon. Normal? We knew they would destroy the world, but we started an arms race to get them anyway.

Absolute. He uses this analogy. Others in the field use this analogy. It's powerful technology.

He wants to make sure we strike the right balance between using this technology for good and evil.

So, Cade, you've put together a rather complicated puzzle here. On the one hand, it is a technology that works completely differently and perhaps much better than one of the most important inventors envisioned. But then again, it's the technology that has also made this inventor and others anxious about the future with such startling and sudden changes. Did you ask Geoff if looking back he would have done anything differently?

I asked him this question several times.

It's a complicated situation he's in.

He paraphrased the British philosopher Bertrand Russell:

- saying that you can make a wise decision and still end up unhappy. And that's basically how it feels.

This reminds me, Cade, of Andrei Sakharov, who was obviously the Soviet scientist who invented the hydrogen bomb, witnessed its invention, was terrified and tried to fight it for the rest of his life. Do you see it this way?

I am doing.

[PLAY MUSIC]

He's someone who helped build powerful technology. And now he's very worried about the consequences. Even if you think the doomsday scenario is ridiculous or implausible, Geoff points to so many other possible outcomes. And that is reason enough for concern.

Cade, thank you for coming to the show.

Happy to be here.

we're coming back.

[MUSIC PLAYING] Here's what else you need to know today. After a marathon of crisis negotiations, President Biden and Speaker of the House Kevin McCarthy reached an agreement on Saturday night to suspend the government's debt ceiling for two years - enough to get through the next presidential election. The deal still has to go through Congress. Both McCarthy and the Democratic leaders spent the rest of the weekend giving full-fledged presentations to members of their respective parties. The House plans to consider the deal on Wednesday, less than a week before the June 5 deadline, when the government will no longer be able to pay the bills.

And in Turkey, President Recep Tayyip Erdogan overcame the biggest political challenge of his career on Sunday to secure a runoff victory that brought him another five years in power. Erdogan, a fickle leader who has angered his Western allies while tightening his grip on the Turkish state, will deepen his conservative influence on Turkish society, which will rule for a quarter of a century by the end of this term.

Today's episode is produced by Stella Tan, Rikki Novetsky and Luke Vander Ploeg, with help from Mary Wilson. Edited by Michael Benoist, with help from Anita Badejo and Lisa Chow, it features original music by Marion Lozano, Dan Powell, Rowan Niemisto and Elisheba Ittoop, and was engineered by Chris Wood. Our theme song was written by Jim Brunberg and Ben Landsverk of Wonderly.

It's all in the Journal. I'm Sabrina Tavernise. See you tomorrow.