How worried—or excited—should we be about AI? Recode Media with Peter Kafka

Support for the show comes from Odo. Running a business is hard enough, so why make it harder with a dozen different apps that don't talk to each other? Introducing Odo.

It's the only business software you'll ever need. It's an all-in-one, fully integrated platform that makes your work easier.
CRM, accounting, inventory, e-commerce, and more. And the best part?

Odoo replaces multiple expensive platforms for a fraction of the cost. That's why over thousands of businesses have made the switch.
So why not you? Try Odo for free at odo.com. That's odoo.com.

Support for this show comes from Upwork. If you're overextended and understaffed, Upwork Business Plus helps you bring in top quality freelancers fast.

You can get instant access to the top 1% of talent on Upwork in marketing, design, AI, and more, ready to jump in and take work off your plate.

Upwork Business Plus sources vets and shortlists proven experts so you can stop doing it all and delegate with confidence.

Right now, when you spend $1,000 on Upwork Business Plus, you get $500 in credit. Go to upwork.com/slash save now and claim the offer before December 31st, 2025.

Again, that's upwork.com/slash S-A-V-E, scale smarter with top talent and $500 in credit. Terms and conditions apply.

Hello, I'm Kara Swisher. And I'm Scott Galloway.
And today we're featuring an episode of Recode Media from our friend Peter Kafka.

In this episode, Peter tries to answer the question, is AI going to change everything we know or is it just another tech hype cycle?

He speaks with folks from Google, the New York Times, and others to get the answer. Scott, what's your biggest question around AI? You're still messing around with it, right?

I'm on that thing every night. Which one? Chat GPT or Bob?

I'm on Notion. You got to do Notion AI for writing.
I'm on Dolly. I'm on, I'm Can't Stop.
And every time I watch TikTok, it tells me about a new one. Oh, really?

Is there any voices you're listening to that like pull up pro and con? Anybody? Well, one of my colleagues, Gary Marcus, is pretty interesting on it.

You know, I hate to say it. I'm part of the, I, most of my AI, I'm learning from TikTok and reels.

Oh, all right. There you go.
Real's fascinating. I, what I tell everybody is, AI is not going to take your job.
Someone who knows AI is going to take your job. You need to learn about this stuff.

Yeah. Yeah.
Do you have a worst case scenario for AI? Yeah, I'm living it. Yeah.
Because everything is so catastrophizing or catastrophis, I'm, I'm actually, I, you are, which is unusual for me.

I'm an AI optimist. I actually think it's going to be, yeah, I think it's going to unleash incredible economic productivity, huge benefits around health care.
Yeah. I'm really excited about it.
Yeah.

Yeah. You be an optimist.
That's good, you sunny optimists.

Hello, sunshine.

Hello, rainbow's coming out of my ass.

What do I think of you? There is a worry. I mean, I know the writers, strike people are worried about it.
They take all the fun jobs.

But no matter what, Scott, you still have to take out the trash. Yeah.
Hey, I have a question. Do you or your wife take out the trash?

No, Carmenia takes out the trash. Hello, douchebag.

Hello, douchebag. Who do you think between a man and I take out the trash? We do not have Carmenia.
Who do you think takes out the trash? Not the AI. Well, this comes down to who's the man.

So who handles a really tough parking spot? Is that wrong? Is that wrong? I want you to guess who takes out the trash.

The fact you ask means you.

It is. It's me.
I take out because I ain't over 10 people. It's called critical thinking people.
When someone asks who's a bit of a narcissist, it means they're setting themselves up to look good.

That is correct.

I love taking out the trash. It's one of my favorite things.
Anyway, an AI cannot replace me and it shall not. Perhaps a robot, killer robot will, but we'll wait on that.
More on that.

Here's Peter Kafka and Recode Media. Enjoy.

Joshua Browder is bad at driving. This isn't a judgment.
It is a fact, and it's verified by the police.

I like to make the excuse that they give out tickets a lot, but in reality, I was a terrible driver and probably got about 30 tickets. But Browder is good at getting out of tickets.

The trick he figured out was writing letters to appeal the tickets. He did this a lot, and it worked.

And a rumor spread that I was the guy who could get my friends out of tickets, and people would just send me photos of their tickets.

And after about the 10th ticket, I thought there must be an automated way. And so I created it just for fun.
Browder researched the top 12 reasons that tickets were dismissed.

Things like the signage or there being a mistake on the ticket. And then he wrote a piece of software that matched your ticket with one of those excuses.

Then takes down all the details, inserts those details into a letter, and then faxes or mails it to the right place. Browder turned this talent into a company.
Do not pay.

So you can pay him to automate you out of tickets or bank fees, free trials, stuff like that. This is cool, but not mind-blowing.
Not for a tech startup.

But then Browder, like a lot of people across Silicon Valley, started to get excited about artificial intelligence. He wanted do not pay to use it because he wanted to do more than write letters.

He wanted to send AI software to court to replace a human and act as someone's lawyer.

Over 80% of people who need legal help can't afford it, and a lawyer can't read 10,000 documents in one second, but AI can.

And so it can actually craft pretty good defenses in some of these areas So I put out an offer in December saying we want to actually have AI whisper in someone's ear for a speeding ticket case and Over 300 people took me up on our offer because I said even if you lose we'll pay the ticket for you so how did you imagine this was gonna work So the person will wear an earpiece or these glasses where they kind of have the audio through the glasses.

You'd walk into the courtroom with some device that had a microphone. Exactly.
That then would talk to something on the internet.

It would listen to what's being said in the courtroom, process it with the AI, and then whisper back to the person what to say.

So it literally would be like having software tell you, poke you on the shoulder and say, say,

I object.

Exactly. It's like having a really good lawyer whispering to you.
Real lawyers did not agree. Browder's offer caused quite a stir with human professionals.

So these lawyers started phoning up state bar associations. And there are these laws sometimes from the 1800s and 1900s saying that you need a license to practice law.

And it's actually a criminal offense to practice law without a license.

And they were making these new arguments to suggest that by having a robot lawyer in the courtroom, it's illegal and I should go to jail for helping inspire that.

And you couldn't just say this is the same thing as bringing a calculator into the courtroom?

That's what we were trying to do, but some of the state bar associations were saying, well, we'll arrest you and find out. Browder backed down.
He did not like the idea of going to jail.

And now his company is facing a lawsuit for the unauthorized practice of law. I think you can call this a mixed result.

Browder isn't happy about getting sued, but he also knows that guy who brings robot lawyer into court is a good way to get attention. The kind of attention we're paying to him right now.

So you did this thing in part because you wanted to generate attention. You generated attention and that got you sued.
Is that a fair summary? Yeah, we put a target on our backs.

And if you have powerful crazy lawyers with a target on your back, it's a scary situation.

Browder thinks that eventually this stuff will stop being a novelty and that we're going to use AI in court just like we're going to use it in all kinds of situations, so often that we won't think it's remarkable.

I think it's inevitable and all these lawsuits, it's just the dinosaurs suing to stop the ISH. AI is being used in the US right now to generate sentencing reports for judges.

Other judges, like a Colombian judge, are already using ChatGPT to write their briefs. And so, why can't this same technology be given to consumers?

There are a flood of people like Browder. People who say this tech is not just coming around the corner, it's here, now, and it's going to have massive effects on everything.

Business, culture, science, the way we live, the way we do or don't make a living.

This seems like it all came out of nowhere. You didn't hear people talking about AI like this a year ago.
Now it's all anyone is talking about.

It's hard to tell what's bullshit, what's a stunt, and what's real. But it feels all-consuming.
People at big companies are spending a lot of money on this tech.

Entrepreneurs are creating new businesses. And there are people watching all this happen who are getting really, really worried.

I want to talk to all of them. So, for the next three episodes of Recode Media, that's what I'm doing.
I'm going on an AI tour while I try to figure this out, and you're going to come along with me.

Our first stop, Microsoft. Yep, Microsoft, the people who bring you Excel spreadsheets and Word documents have bet more than $10 billion that AI is the future.
They put that money into OpenAI.

That's the company behind ChatGPT, which is the chatbot software you've been hearing a lot about the last few months.

And now Microsoft is bringing ChatGPT into their own products like Bing, Microsoft's search engine. ChatGPT can do amazing things.
It can write software, it can make you a menu plan.

It can also spout out alarming ideas and flat-out untruths. Microsoft CTO, Kevin Scott, thinks this is the future.
Yeah, I think it's a pretty big deal.

And anybody who's making predictions about where AI is going needs to approach it with a little bit of humility because things are changing so fast.

But the sense that I have, having lived through three of the big technology platform shifts over the past 50 years.

So I was a 12-year-old, 10-year-old kid when the PC revolution was happening, when I learned programming. I was in grad school when the Internet Revolution happened.
And then I was

a pre-IPO employee at Google. And then

I was running a mobile startup right at the very beginning of the mobile revolution, which sort of coincided with this massive shift to cloud computing.

This feels to me very much like those three things.

So that's very big picture. I want to narrow it down for a minute or at least explain some terms to the listeners and to me as well.
People throw around AI as a term.

We hear a lot about generative AI. It seems to be the particular flavor that we're very interested in the moment.
There's also discussion about... Age AI.
Can you sort of help us set the table here

and emphasize what we're talking about right now when we're talking about AI? Yeah, so artificial intelligence is the, or AI is the name of this field that we're all practitioners in.

The term was coined in the 1950s.

And in a certain sense, like it's a terrible name for what we're doing because it invites all of this comparison to biological intelligence, human intelligence, and like the way that AI software works is very, very different from how a human brain works.

And like what capabilities AI has are very different from what a human brain.

Like, there are a bunch of things that are easy for AI that are hard for humans, and there are a bunch of things that are just trivially easy, even for

like an infant human being, that are nearly impossible for AI to do.

And so, you just have to be really careful about drawing those connections between what human intelligence is and what artificial intelligence is.

So, those other two terms that you mentioned, so generative AI is the thing that is getting everyone excited now. So these are very large-scale AI models.

So big pieces of software that were trained on lots of data with huge amounts of compute that can synthesize things.

Like they can, if you embed them in a chat bot, for instance, they can have a conversation with you. So they're generating language.

If you are, for instance, like wanting to do something creative, they can generate visual images. And so like we'll have more and more types of generative AI over time.

Right. I mean, you use the word, you hear compute a lot.
And I just translate that as powerful computers. Correct.

Yeah. Very, very, very powerful computers.
And then AGI is the thing that everyone says, maybe we'll get there one day, maybe we won't. And that sounds the most science fictiony.
What is that?

So AGI is

an acronym for artificial general intelligence.

And this is, you know, sort of the thing that the founders of the field of AI were going for from the very beginning.

So it's a thing that in a, you know, by some definition is close to human-like intelligence.

So where is this going in the near term? We, you know, we've sketched out like there's this idea that maybe one day the computer is actually going to sort of replicate human thinking.

That seems to be a far way off. If ever we have this stuff now that's really novel and interesting, it seems to be moving very quickly.

If you're talking to a lay person,

again, me,

how should we think about this tech impacting our lives within the next couple of years?

Well, the vision that we have and that

our partner OpenAI has is that we believe that you can think about this AI as a platform. Like a platform allows people to build things on top of it.

And so people will be able to pick these platform components up and build software with it. And so what I think everybody should expect to see over the coming months is,

you know, you've seen this progression of what Microsoft and OpenAI are doing with the software.

So, you know, it's like GitHub Copilot, which is a coding assistant, Bing Chat, which is a assistant to help you do a search.

We announced the Microsoft 365 Copilot, which is a set of AI assistants to help you get your work done.

So, you know, we're going to have more of those ourselves, but other people will be able to build that sort of software for whatever it is

the important problem is to them.

And this is why you hear people talking about this being sort of akin to the iPhone slash App Store moment.

Like you have this technology and then you allow everyone else to build stuff on it and you get fart apps and you also get Uber or Piccadilly. 100%.
Pick up rates.

It's great you mentioned the fart apps because I think one of the interesting things that happens with every one of these new technologies as they emerge is it takes a while for people to figure out what the interesting things are to build.

So it's often the case that the easy things to build at the beginning are not the interesting things, which is why you get fart apps.

But like eventually you do get something like Uber, which is a very complicated business that couldn't exist without that smartphone platform.

And then right now people are also intentionally testing this stuff out, both the OpenAI version, the stuff that you have, and saying, what can we do with this?

What can we do with this that the makers didn't intend? Can we get it? Where is it going to fall flat on its face?

Sometimes they're accidentally stumbling into some exchange where the chat bot seems to be saying rude or even worse things to them.

And you guys are going back and modifying that as you get the feedback. When you use Bing right now, there's a banner on it that says, let's learn together.

Bing is powered by AI, so surprises and mistakes are possible. Make sure to check the facts, share feedback so we can learn and improve.
So I think I understand the intent behind that disclaimer.

On the other hand, you guys are a giant, giant company,

both enterprise and consumers. How are you thinking about rolling out a product that is, by definition, not finished at all?

Yeah. And so if you think about the product as a platform, like it will never be finished, like it sort of has infinite possibility for what people can do with it.

The way that we have been thinking about things is we've spent,

you know, from 2017 until today, rigorously building a responsible AI practice.

You just can't release an AI to the public without a rigorous set of rules that define sensitive uses and where you have a harms framework and where you like even are transparent with the public about what your approach to responsible AI is.

And so like we publicly disclose two versions of the responsible AI standard that we use inside of the company. All of these things that we've done run through this responsible AI process.

That's people imagining what could we do with this that would be bad and let's prevent it from doing that. Yeah.
But even in the best case scenario, they're not going to think of everything.

And even in the best case scenario, someone's going to foil those efforts. And I'm not even talking about that yet.
I'm just talking about the basic fact that this stuff is still pretty raw.

It seems, I understand if you were a home hacker when you were building your own computer, you'd say, Oh, it doesn't work, you got to get in there with a soldering thing and get into the circuit boards, but this is going out into the public.

Yeah, so look,

the point of actually launching

the software is because it actually, for the first time, I think, really has given a very broad swath of the population an opportunity to see the software, to see what it's useful for.

And like they're seeing all sorts of things that

the software is useful for that we hadn't imagined, but also giving us feedback.

One of the things that has been really useful for us is

they're obviously this clear set of things that we vetted beforehand

that like we haven't seen anybody demonstrate a behavior or one of these AI systems that people are talking about like oh like how do we get it to not we want to make sure it doesn't tell people how to make chemical weapons is the thing people are talking about.

You know and among other like many, many, many, many other things. I mean,

there's so much stuff that we've done to try to make sure that

the harms that we all, like any reasonable human being could sort of look at a thing and say, yep, this is bad. Like we don't want the system to do it.

And then you're going to have this category of things where

you know, you're going to randomly choose two reasonable human beings from 8 billion folks on the planet. And like you're going to have disagreements about what's acceptable and what's not.

And like this is, you know, just sort of how you build products in general. Like people have different preferences depending on what part of the country they're from.

They have different preferences depending on what part of the world. Like, you know, some places one thing's acceptable and another, it isn't.
And

part of, you know, the, the, the thing that we're learning right now by launching all of this stuff is, you know, where those lines of preference are and like how we can quickly conform the product to give people the thing that they want.

If you do any amount of even, you don't have to read, you can just browse some of this stuff and people will you know about AI, you inevitably get people talking about doomsdays scenarios. Yeah.

And again, part science fiction and part a thing we ought to be thinking about. I'm sure you've seen the survey of AI experts from last year.

And depending on how you phrase the question, the question says something like, what percentage chance do you think AI has of wiping out humanity?

And depending on how you phrase the question, it's five to 10 to 14 percent. Those seem like unacceptable odds to me.

What am I missing? And how do you think about playing with tech that by definition, you don't really know where it's going to go and you don't know what its full capacity is going to be?

It is absolutely useful to be thinking about these scenarios.

It's more useful to think about them grounded in where the technology actually is and like what, you know, the next step is and the step beyond that.

I think we're still many steps away from like the thing that people are worried about. And like you would have to deliberately as the developer of these platforms go seek out some of this stuff.

And like there, there are people who disagree with me even on that assertion. Like they think that there's going to be some, you know, uncontrollable emergent behavior that happens.

And like we, you know, we're careful enough about that where we have research teams thinking about the possibility of these emergent scenarios.

But like the thing that you, you would really have to have have in order for some of the weird things to happen that people are like concerned about is like real autonomy.

So like a system that could participate in its own development,

you know, and like have that feedback loop where you could get to some superhuman fast rate of improvement. And that's not the way the systems work right now.

Not the ones that we're building at least. So you're assuring me that you have things under control.
I'm going to take your word for it. If we

do. And look, we shouldn't, I mean, this is one of the things, like we're, we're going to figure out over the next handful of years, like, where

governments and society want to like be able to look over the shoulders of the people developing this technology to make sure that everything is being done responsibly.

And I think that's a reasonable thing for societies and governments to ask for. I don't feel particularly

confident that government's going to be particularly good about controlling tech. There's the sort of lock it down.

If we look at how governments around the world, including the U.S., are dealing with just social media tech, which is pretty simple, all things considered,

they seem kind of baffled by it and they can shut it down or they throw their hands up. And this is much more complicated.
Trevor Burrus, Jr.: Well, look, so I think it is and it isn't.

Like, the interesting thing about social networks is like they, you know, sort of get right to

this very complicated thing about like what happens when very large numbers of people are interacting with one another in a technology facilitated way. And like,

you know, maybe you could argue that that technology is not as complicated as these AI things, but like that dynamic is maybe even more complicated than what we're talking about here.

Let me ground things in the here and now. What's the business model for Microsoft with AI? Who's paying for this?

Because as we all learned over the years, even when you're using a free service, someone's paying for it on one, and that shapes the way the product is made.

Aaron Powell, Jr.: Yeah, I think just like all of these other technology shifts that we've seen over the past 50 years, like we will figure the business model out as the technology matures.

So the obvious ways that we're paying for it right now are sort of three-way.

So

for Bing, it gets paid for with advertisements.

And so like the ad model itself is likely going to change over time. But right now, it's it's a search engine, it's free and it's ad supported.

For things like Chat GPT, it's sort of a freemium model. So

you you have a free tier that you can sign up for and like that gets you one level of service and you can pay for ChatGPT Pro, which is a subscription product that gets you another level of service.

And then there is the, you know, can we integrate it with existing products like Microsoft 365 that has like a pricing model and like there, there's maybe some different way to price it than

an E5 subscription or whatnot, which is what we do right now for big businesses. You've been very responsible in this conversation and you have not been promising me the world.

On the other hand, you've been saying, this is as big a deal as...

major technological revolutions you live through in your life. What are you most excited about when you're playing with this stuff? Yeah, so like I like, I'll give you a couple of examples.
So

my brother,

who lives in rural Central Virginia with my 74-year-old mother, is immune compromised and got COVID last fall for the first time.

And,

you know, in rural Central Virginia, he doesn't have access to the same quality of health care that I have access to here in Silicon Valley. And

the responses that he got to his doctor about what he should do after having tested COVID positive just weren't great.

And, you know, I just sort of think about the possibility for these AI systems to help people

get access to

high quality of medical care, like high quality of educational resources, high quality of like a bunch of things that you think of as skilled guidance that you get from people.

Like, I think

these systems can make those sorts of things accessible to folks for whom it's not accessible right now.

And, like, I think that's a really exciting thing to me.

So, that's a good place to maybe wrap this up because it's one of the big questions and debates that I've found here is that scenario you've described, you hear people who are excited about AI saying we could extend knowledge and resources to people who wouldn't have it normally, whether it's medical or legal, all kinds of stuff that you had to be rich or live in a certain place or know someone to get to, and we can extend that out.

And then other people say, you're describing a dystopian scenario where the rich people get real human doctors and everyone else has to work through a bot. That sounds terrible.

So you've sort of described the upside of your vision of it,

but help convince me and others that this is a better solution than going to my GP and asking him for Pax Lobid. Well, like, I think it just sort of depends on what the actual delivery mechanism is.

So, like, what I would say is, like, you absolutely don't want a world where all you have is some substandard piece of software and no access to a real doctor.

But so, so I have a concierge doctor, for instance,

and like, I interact with my concierge doctor mostly by email. And so, like, that's actually a great user experience.
Like, it's phenomenal.

It saves me so much time, and I'm able to get access to a whole bunch of things that, you know, just my busy schedule wouldn't let me have access to otherwise. And so

I have thought for years, wouldn't it be fantastic for everyone to have the same thing? Like an expert medical guru that you can go to that can help you navigate a very complicated

system of insurance companies and medical providers and whatnot.

And yeah, sure, you want to be able to get access to human beings at the right point, but but having something that can help you deal with the complexity of a very complex system, like, I think is a good thing.

When the first iPhone came out, and this was my last question, there were people lined up to buy it. And then I would see these people in New York.

They'd walk around showing the iPhone off at parties. And people go, oh, my God, let me check out that iPhone and pinching and zooming.
And it was really great.

X number of years now, no one ever looks at you and wonder when you pull a supercomputer out of your pants pocket.

Is that the track we're on here for AI where it becomes embedded and we don't think about it and it's fully part of our lives and we take it for granted?

Well, look, I think it's either going to be that or it's going to be useless. So like

every major transformative technology thing in human history gets past this sort of hump of adoption where like there's lots of skepticism and like people don't really know what it's good for and like enters the domain of ubiquitous.

So either this is like one of those major platform things,

in which case at some point it will be absolutely ubiquitous and we will completely take it for granted, or it just isn't. And you know, like.
folks who are excited about it are

like in some kind of crazy hype cycle and it's going to flame out and we will see which one it is over the coming years.

In a moment, we're going to hear from someone who says Kevin Scott and most of the other AI optimists are wrong.

Support for the show comes from Odo. Running a business is hard enough, and you don't need to make it harder with a dozen different apps that don't talk to each other.

One for sales, another for inventory, a separate one for accounting.

Before you know it, you find yourself drowning in software and processes instead of focusing on what matters, growing your business. This is where Odoo comes in.

It's the only business software you'll ever need. ODU is an all-in-one, fully integrated platform that handles everything.
That means CRM, accounting, inventory, e-commerce, HR, and more.

No more app overload, no more juggling logins, just one seamless system that makes work easier. And the best part is that Odo replaces multiple expensive platforms for a fraction of the cost.

It's built to grow with your business, whether you're just starting out or you're already scaling up. Plus, it's easy to use, customizable, and designed to streamline every process.

It's time to put the clutter aside and focus on what really matters, running your business. Thousands of businesses have made the switch, so why not you? Try Odo for free at odoo.com.
That's odoo.com.

Support for this show comes from Upwork. So you started a business, but you didn't expect to become the head of everything.
Now you're doing marketing, customer service, and IT with no support staff.

At some point, doing it all becomes the reason nothing gets done. Stop doing everything.

Instead of spending weeks sorting through random resumes, Upwork Business Plus sends a curated shortlist of expert talent to your inbox in hours.

These are trusted, top-rated freelancers vetted for skills and reliability.

And with Upwork Business Plus, you can get instant access to the top 1% of talent on Upwork in marketing, design, AI, and more, all ready to jump in and take work off your plate.

Upwork Business Plus can take the hassle out of hiring and the pressure off your team. That way you can stop doing everything and instead focus on scaling while the pros at Upwork can handle the rest.

Right now, when you spend $1,000 on Upwork Business Plus, you get $500 in credit. Go to upwork.com/slash save now and claim the offer before December 31st, 2025.

Again, that's upwork.com/slash S-A-V-E.

Scale smarter with top talent and $500 in credit. Terms and conditions apply.

And now, an anti-hyper. My next guest believes in AI, but not the way it's going right now.
His name is Gary Marcus, and he's a cognitive scientist.

He's the author of Rebooting AI, and he's got a new podcast coming out called Humans versus Machines.

Marcus believes the things that companies like Microsoft and OpenAI are building aren't real artificial intelligence, but what they are is deeply flawed tech.

You know, a lot of us dream of artificial general intelligence, a system that you could throw any problem at, kind of like the Star Trek computer. You say, solve this for me.

The Star Trek computer could play chess, but it could also answer questions about dilithium crystals or whatever it is they needed at that time.

GPT-4 gives the illusion of doing that, but it's not actually that great at it. It can talk about anything, but it's not reliable.
Some people say it lies.

Literally, it doesn't lie because it has no intent, but it makes stuff up fairly regularly. And so you can't really trust its output.
For GPT-4, you have to use just the right prompt.

If your prompt is slightly off, then you get a weird answer.

This is not something that happened with the Star Trek computer. I sort of started having this fantasy, like the enterprise is about to crash.

And Captain Kirk says to Scotty, you know, we need to fix this problem. And Scotty is like, well, I just can't get the right prompt to work.

And then, you know, maybe it gets the right prompt or maybe it gets the wrong prompt and the enterprise crashes. We don't have reliability here with these systems.

They're a hint at what artificial general intelligence would look like. They don't work that well.

Even if this is not the supercomputer that we're promised in science fiction, there are a lot of pretty rational people, some of whom have a vested interest in this, but others who don't, who say this seems like a really big moment.

We just heard from Kevin Scott, the CTO of Microsoft. He does have a vested interest in this, and he described this moment as being something akin to the development of the personal computer,

the development of the internet, the development of the iPhone,

maybe all three of those bundled together. And he is not promising that this is a transporter beam or it's going to manufacture food for us, but he does think that it's a really big deal.

Is there a sliding scale where we can say, look, this is not science fiction fantasy, but it is a big deal. It is interesting and it is going to help.
us in some way. Is that more reasonable for you?

I mean, it's absolutely interesting. I would not want to argue against that for a moment.
I think of it as a dress rehearsal for artificial general intelligence, which we will get to someday.

But I think right now we have a trade-off. You know, there's some positives about these systems.
You know, you can use them to write things for you. And there's some negatives.

So this technology can be used, for example, to spread misinformation. And to do that at a scale that we've never seen before, which may be dangerous, might undermine democracy.

And I would say that these systems aren't very well controllable. I think of them as like bulls in a china shop.
They're powerful, they're reckless, but they don't necessarily do what we want.

And ultimately, there's going to be a question kind of akin to the driverless cars, which is, okay, we can build a demo here. Can we build a product that we can actually use? And what is that product?

And I think in some places, people will adopt this stuff. They'll be perfectly happy with the output.
In other places, there's a real problem.

So in search engines, for example, people are trying to replace traditional search engines with what you might call chat search, like Bing.

And, you know, it kind of works and it kind of doesn't. The most recent version that I saw is Bard.

Somebody put in my name and had it write a biography. Bard is Google's version of Bing.
Bard is Google's version

of Bing. And, you know, there's some differences between the two.

But what Bard did, and we've seen this with Bing, is it confabulated. So it got the subtitle of my book wrong.
It fabricated two quotes.

It said that I made arguments there that I didn't because the book was published in 2019 before large language models were popular.

And so it kind of like gave a flavor of a Gary Marcus argument, but it got all the details wrong. I don't really want my search engine to do that.

When I use a search engine, like when I'm writing a book, I want the facts, not like a flavor of the facts.

So if, you know, we've gone, we've had this discussion about driverless cars in the past, and you've got people like Elon Musk saying, sure, that the driverless Tesla is going to hit something occasionally, and maybe

someone might even die. But people die in cars all the time.
Human drivers are terrible. The computer is going to be better than X percent of them, and that's a worthwhile trade-off.

Most people aren't comfortable. with that analogy, but I'm also thinking of earlier versions of technology being directionally correct, but not right at the beginning.

And people are comfortable with that, and it gets better. I remember with the early Wikipedia, when that was brought up as an idea that it could replace an encyclopedia, and that seemed ridiculous.

And the initial you know, Wikipedia results were often very bad because they were biased, et cetera. And now Wikipedia is still flawed in lots of ways, but is generally a pretty good resource.

Are we on that kind of trajectory with this kind of tech? I think we might be, but I think it's a critical question of time and cost. So costs come in different

dimensions.

With driverless cars, I warned in 2016 that there was an outlier problem and that it might not be practical to actually use them anytime soon.

And not long after that, we were told, hey, we're going to have a driverless car go from Los Angeles to New York. Elon Musk said that in 2016, promising it in 17, if I recall correctly.

And we've gotten these promises pretty much every year since, that they're imminent. And the reality is they're still demos.
They're not really products.

So, yes, you can take some rides in San Francisco, for example, but it has to be at certain hours on certain roads. You can't take it wherever you want.

And so it's been many years of promise, not really realized. We could wind up in the same place with chat search.

I mean, people are certainly running demos right now, but whether we can count on it, how long it's going to take for us to count on it, I don't know.

I think driverless cars will eventually replace humans. I think we will reach a point where driverless cars are, in fact, better than people, but I don't think we're particularly close to that.

It might be another decade or two.

And so a question is whether chat search is more like driverless cars, where we see a taste of it, but takes a long time to deliver, or are they more like Wikipedia, which I think is actually pretty practical right now.

Right. And so I just want to push a little bit on that.
If we say, look, we don't want the AI to drive us from

Los Angeles to New York or really anywhere, because we're just not comfortable with it, and we're not comfortable with it being wrong.

But if it gives you search answers that are usually correct or mostly correct, and you have to do a little bit of extra work to make sure it's entirely correct, you're not going to kill anyone

if this gets wrong. Well, if it's medical misinformation, you might actually kill someone.
And that's actually the domain where I'm most worried about erroneous information from search engines.

People do search for medical stuff all the time, and these systems are not going to understand drug interactions. They're probably not going to understand particular people's circumstances.

And I suspect that there will actually be some pretty bad advice. Now, they may try to lock these systems down so they don't give medical advice, but then people may go back to Google.

And so it might not be Microsoft's great desire to do that. So I mean, there, there's a lot of like commercial questions and user questions we don't really understand.

So we understand from a technical perspective why these systems hallucinate. And I can tell you that they will hallucinate in the medical domain.
Then the question is like, what becomes of that?

What's the cost of error? How widespread is that? How do users respond? And we don't know all those answers yet.

I want to ask you to define some stuff. Let's start with hallucinate.
It means something kind of akin to what we think of when it comes to AI, but something a little bit different.

What does that mean? So what we mean by that is that these systems, large language models is what we're talking about, have a tendency to make stuff up.

They don't have a built-in system for validating things, although people are trying to put extra machinery on top. But inherently, they don't actually track facts about the world.

They don't build mental models of the world. They don't build mental models of medicine, of physics, et cetera.
They just really are autocomplete on steroids.

They predict text, what might come next in this circumstance. And they do that over clusters of words.
So they're like, this cluster of words predicts this other cluster of words.

It's not even conceptual. It's really relationships between words.
And so you get really weird

things coming out of it. So if you actually look as recently as November of 2022, Facebook had a system or Meta had a system called Galactica, and it made up some really wild stuff.

It, for example, said on March 18th of 2018, Elon Musk died in a car crash. And he didn't actually die in a car crash.
It's not actually in the data that the system was trained on.

There's lots of data that contradicts that. So that's what we call hallucination.
It makes stuff that is plausible, but it doesn't know the difference between things that are true and not.

And it takes months to train these things. GPT-3 was trained only on 2021.
It didn't know what happened in 2022. Didn't know that Elon Musk was the CEO of Twitter, for example.

And I got mansplained by,

I think it was ChatGPT, told me that Twitter was a public company. It wasn't private.
Elon Musk wasn't the CEO. Basically said that I made all this stuff up, which I hadn't, right?

We sometimes call that gaslighting, right? So if you use a pure language model, it simply won't be up to date.

It sort of strikes me as, and I've been in this position in the past and at various grades,

where you're presenting a

talk or writing a paper and you use lots of words that have come up in class and because they you think they sound right and you're hoping that no one's going to sort of pay too close of attention and it might get you through the quiz.

Yeah, I mean, GPT-3 is a lot like that. It doesn't have the intention.
It's not trying to bluff you.

It's not trying to please you, but it is at some level bluffing because it doesn't really understand what it's talking about. I mean, that's kind of being anthropomorphic about it.

I mean, really, it's just predicting words. You wrote an essay in the spring of 2022, about a year ago, called Deep Learning is Hitting a Wall.

Can you tell us what deep learning is and why you thought it was hitting a wall?

Deep learning is the major technique in machine learning, which is part of artificial intelligence, that's getting used all the time. It underlies large language models.

We use these things for image recognition. They're in image synthesis.
They're in these chatbots now. They're very widespread.

And what I said in this essay, which was in March of 2022, was that they were hitting a wall. I didn't say there's not any progress anymore, but I said they have particular problems.

I said that they're not reliable, they're not truthful, they confabulate things, and they have a tendency to produce misinformation.

People got really, really upset that I should dare to say that their baby was hitting a wall. There was a whole satirical campaign almost of memes to try to attack me.

And what's amazing to me about it is that it didn't take that long before a lot of people changed their views.

Sam Eltman, he's the CEO of OpenAI, he said, give me the confidence of the mediocre deep learning skeptic.

And then, you know, in December, he put out his own tweet that said, these systems still have a lot of work to do. We can't really trust them.
They have problems with truth, with reliability.

Exactly the same things that I had said in that article. So I think in a way, the field has grown up.

And in a way, like there's something deeply problematic about a field that so tries to silence its critics that there's like this relentless campaign against them. And it's also a sign of like

if you're going to do science, you have to actually listen to people who have opposing views. If you try to silence them, you're not going to get to

doing good science faster.

You published that paper in March 22.

In the fall of that year, there's this just wave of excitement around AI, largely from what I can tell, because it was now sort of something that regular people could now see in the case of images like DALI or image generators like DALI and then chat GPT that you could play with without having to know how to use anything other than how to turn a browser on.

Was there any technological change that also spurred that on or is that or sort of if we if we look in the guts of what's happening there is that the same conditions that you were talking about in March?

And it's just a fact that it's just the fact that it's now accessible to people that has made the excitement that has generated that excitement? There's a few things.

So so the technology in it is not that different from its predecessors but they added these guardrails you could still so-called jailbreak it and make it say really terrible things with without that much trouble you could use it to generate misinformation but they had these guardrails that were good enough even though they weren't great that people could just try it out and that made a big difference and even open ai didn't expect it to go viral the way that it did but it did in part because it was so much fun to play with

you're saying if I'm getting this right, that the folks at OpenAI took tech that was basically pre-existing, that they already had, made it more accessible to humans, added software to make it less prone to error, and people got very excited about it.

This seems like

the good version of tech progress. I mean, you take a thing and you modify it, and it becomes better and useful.
They made it a better product.

I mean, so Jan Lacun has pointed out, and other people, also yashua benjio these are two of the pioneers in deep learning have pointed out i think rightly that it wasn't big technical progress it was good product progress so it wasn't some new form of ai that we didn't have before but yes it was a better product it was easier to use and some of it was just like making it cute like it types out words one by one instead of giving you the whole thing somehow that makes it feel more human so like from a product side, it was definitely a better product than we had before.

From a technical side, those of us who are interested in so-called artificial general intelligence and more generally, reliable and trustworthy artificial intelligence are like, yeah, we still have the same problems that we did before.

It's just more fun to play with.

So, I mean, there's a tension between like, what do people actually want to use? And it's better at that.

And on the other hand, like, how do we make this reliable so that if you type in your medical questions, you can actually trust the answer.

I have a broad and shallow view of technological progress, but it essentially goes like things get better over time. You have unexpected results.

You don't always know where it's going to go, but generally things get better, faster, cheaper, smarter.

We know about Moore's Law and chips get used to get faster all the time, and that sort of slowed down.

And what you hear from AI proponents is these neural nets and other things that are the engines for this are getting better all the time.

We're adding more data, we're computing more, we're learning more.

They are in the aren't why won't that work here? They are in the arms. It's the first thing I'll say.
And then I'd like to to give you a historical parallel.

So on misinformation, for example, they're not really getting better. On truth, they're not really getting better.

A couple days ago, NewsGuard released something showing that GPT-4 was actually worse than GPT-3.5 on discriminating truth. So they are getting better at being more plausible.

They're not getting better at being more truthful. And that really matters a lot.
So that's the first thing I would say.

The second thing is There are a lot of interesting historical cases where people got obsessed about a particular technology and chose wrongly.

And it took a little while to back down from the popular thing to the thing we needed to do. So dirigibles were really popular in the 1920s and 1930s.

Until we had the Hindenburg, everybody thought that all these people doing heavier-than-air flight were just wasting their time. They were like, look at our dirigibles.
They scale a lot faster.

You know, we built a small one, now we built a bigger one. Now we built a much bigger one.
It's all working great.

And those poor guys with the heavier-than-air airplanes are still stuck, you know, barely bigger than the Wright brothers. So, you know, sometimes you scale the wrong thing.

In my view, we're scaling the wrong thing right now. We're scaling a technology that is inherently unstable, unreliable, and untruthful.
We're making it

faster, has more coverage, but is still unreliable, still not truthful. And for many applications, that's a problem.
There are some for which it's not, right?

GPT's sweet spot has always been making surrealist pros. It is now better at making surrealist pros than it was before.
If that's your use case, it's fine. I have no problem with it.

But if your use case is something where there's a cost of error, where you do need to be truthful and trustworthy, then that is a problem.

Because I think the way a lot of people look at it is, yes, they're doing surrealist prose today,

and tomorrow, or six months, or a year from now, we can't project when, it's going to get much better at much more sophisticated.

Well, I mean, this is where I'd say people have been telling me that for 30 years. I first pointed out the hallucination problem in 2001, and so far nobody has solved it.

Even though you can scale some things, you're not going to scale truth. And in fact, the NewsGuard result shows that they're not scaling truth.
So to me, it feels like appeal to hopeful monsters.

And let me pause here. It's really important to distinguish between what we can do with large language models and what AI might someday be able to do.

So I am not making an argument that we will never be able to build an artificial intelligence system that we can trust, that is reliable,

that can keep track of truth. I'm saying this is the wrong technology for that problem.
And until we broaden our search through the space of hypotheses, that we're a little bit stuck.

So this is not artificial general intelligence. That's something you remain interested in and think could arrive one day.
But we have this technology. It is flawed in lots of ways.

Let's stipulate that.

Still seems like it ought to be useful for lots of present-day applications.

Are there things that you are interested in using the current version of this technology for? I mean, me personally, no. Other people have to write much more boilerplate stuff than I do.

And for them, it might make some sense.

I would warn them about accuracy and things slipping through because our human tendency is to take things that are grammatical and well-formed and assume they're okay.

You may have seen how CNET got themselves in trouble. They published 70 articles written by machine.
It turned out 40 of them had mistakes in them. It's very hard for people to pay enough attention.

We have a similar problem with driverless cars, which is a human sits in a driverless car for like an hour and thinks, all right, this is fantastic, and doesn't realize that the machine doesn't think like they do.

And the machine, for example, could run into a jet airplane because it's not in the training set.

And so these systems are not as reliable as they might seem in the first few minutes that we play with them. Fundamentally, the outputs just are not 100% reliable.

And so you have to decide: is that okay for my task?

Aaron Ross Powell, do you think this technology ends up replacing people replacing workers because it can do it better faster cheaper more efficiently or it is a tool that people who know how to use this work use in their in their day-to-day i suspect it'll be some of both we don't really know for sure yet we may find that like where you have 10 people on a job now you can do it with two and that the two kind of oversee the systems and make sure there aren't mistakes we may see a bunch of that like it's it's a little bit too early to say for sure There are other things like that we haven't talked about as much, the generative systems that create images.

And there are limits there. So they may not always give you exactly what you want.
It may be hard to edit to get the thing that you want.

But if you just need a rough thing, let's say for an illustration for a talk,

they work pretty well. And so I think they're going to cut there into the jobs for commercial artists by a lot.

I do expect this stuff will have impact on employment. I think it's as yet a little bit hazy about what that impact is going to be.
It may be small, it may be dramatic.

I think it's going to matter domain by domain. So another place where we've kind of seen this movie before, but might have a different ending, was with radiology.
In 2016, Jeff Hinton said that

might as well stop training radiologists. That's almost a verbatim quote.
And here we are in 2023.

We don't have enough radiologists because it turns out that the job of radiology is not just to look at an image, which deep learning is pretty good at, but also to look at a patient's file and understand the relation relation between that image and a person as a whole and diseases.

And you need to know about anatomy and physiology. And it's going to depend in every profession.
Like, can you do a certain piece of it?

If you can do a certain piece of it, does that mean maybe we can reduce the staff by 10%?

There are a whole lot of questions that really depend on the details of what the job is and, you know, the cost of error and all kinds of different things.

So I don't know if there's like a blanket answer there. I think there's going to be a lot of discovery process in the next few months, but everybody is interested in this question.

Is there a rubric that you can share with us that's useful for an interested layperson to sort of say, all right, this person is telling you about this exciting new AI, whatever, inflected thing they're rolling out.

How could a regular person who has no capacity to look into the code, et cetera, understand whether the thing they're getting is actual AI, whether it's fake AI, whether it's

or it's just hype entirely? So, I mean, look, how you define AI is actually up for grabs. There's no fact of the matter kind of in the universe, independent of people.
We use the terms as we do.

There is no artificial general intelligence right now that just doesn't exist yet. We have AI for certain problems that works great.
So

route planning,

use your GPS to go from A to B works great. That's AI.

It's not large language models, not very sexy anymore because we've habituated to it, but it's actually kind of amazing that that can be done and, you know, all stick on the phone.

Chatbots are real, they're just not reliable. They can talk about anything, but they're often out of their depth.

So you give them a math problem and sometimes they get it right and sometimes they get it wrong.

You give them almost anything in any domain and sometimes they get it right, sometimes they make up stuff. Most of the things we're talking about right now are based on that.

And if you want to ask like, am I in trouble because of that? You have to ask questions like, how well can it do the thing that you do in your job? How much creativity do you need to do in your job?

How systematic is it? How much does it cost if you make a mistake? Who's going to catch the mistakes if the AI does it? Is that even feasible?

For a rubric, there are a whole bunch of different questions that you might ask around that.

How often do you come up with some unusual case that a system that only kind of knows the usual cases might fail on? And it's going to be different for every profession. Gary, this has been great.

Thank you for your time. Thanks a lot.

Thanks again to Joshua Browder, Kevin Scott, and Gary Marcus.

Next week, AI has a lot of critics, but there are a lot more people in Silicon Valley happy to ignore them because they want to cash in. So next up on our AI tour, we're going inside the gold rush.

This is a special series, so we got extra help. None of them are robots.
They're all humans. They're excellent humans.
Thanks to Matt Frassica for producing this episode. Megan Kunan is our editor.

Jelani Carter is our engineer. And Brandon McFarlane scored and mixed this episode.
We got brand new human-made music for this. We'll see you next week.

Thanks for listening. You can subscribe to Recode Media wherever you listen to podcasts.
We'll be back on Friday with more pivot unless the AI takeover starts early. I did not write that.

Support for the show comes from Odoo. Running a business is hard enough, and you don't need to make it harder with a dozen different apps that don't talk to each other.

One for sales, another for inventory, a separate one for accounting.

Before you know it, you find yourself drowning in software and processes instead of focusing on what matters, growing your business. This is where Odoo comes in.

It's the only business software you'll ever need. Odoo is an all-in-one, fully integrated platform that handles everything.
That means CRM, accounting, inventory, e-commerce, HR, and more.

No more app overload, no more juggling logins, just one seamless system that makes work easier. And the best part is that Odo replaces multiple expensive platforms for a fraction of the cost.

It's built to grow with your business, whether you're just starting out or you're already scaling up. Plus, it's easy to use, customizable, and designed to streamline every process.

It's time to put the clutter aside and focus on what really matters, running your business. Thousands of businesses have made the switch, so why not you? Try Odo for free at odo.com.
That's odoo.com.

Support for this show comes from SC Johnson. We've all been there.

Choosing not to wear your new white shoes because there's a 10% chance of rain, bending awkwardly over the tiny coffee table to enjoy a sip of your latte, not ordering the red sauce.

Those feelings of dread are what we call stainsiety.

But now you can break free from your stainsiety with Shout's Triple Acting Acting Spray that has stain-fighting ingredients to remove a huge variety of stains so you can live in the moment and clean up later.

Just breathe and shout with Shout Triple Acting Spray. Learn more at shoutitout.com.