OpenAI CTO Mira Murati on ScarJo Controversy, Sam Altman & Disinfo Fears | On With Kara Swisher

So, your AI agents

make the team that uses them more productive, right?

But if they aren't connected to other agents, or your data, or your existing workflows, how productive can they really make your teams?

Any business can add AI agents.

IBM connects your agents across your company to change how you do business.

Let's create Smart to Business.

Hi, everyone.

This is Pivot from New York Magazine and the Vox Media Podcast Network.

I'm Kara Swisher.

We're off for the holiday today, but we have a special episode for you from my other podcast on with Kara Swisher.

In this episode, you have another podcast?

Yes, I do.

I'm cheating on you.

I'm cheating on you, and so are you.

I know all your

up to.

Mr.

Malice, no mercy, malice.

Mercinuma.

That's like, I got that from one of those.

Why malice?

Why did you go to Malice?

I heard, I was watching late night at two 2 in the morning on Edibles, one of those wildlife shows where a crocodile crunched the shit out of a baby cheetah and James Earl Jones' voice goes, on the savannah, it has no mercy, no malice.

And I'm like, boom, that's the name of my newsletter.

Oh, my God.

I had no idea.

Anyway, in this episode, I interview Mira Marati, the chief technology officer at OpenAI and one of the most powerful people in tech.

Enjoy.

Hi, everyone, from New York Magazine and the Vox Media Podcast Network.

This is On with Kara Swisher, and I'm Kara Swisher.

Today, we have an interview with Mira Marati, the Chief Technology Officer at OpenAI.

OpenAI has certainly been in the news, but not many people know about Mira herself.

She's only 35, but she's already one of the most influential women, scratch that, one of the most influential people in tech.

She helped OpenAI skyrocket to the forefront of the generative AI boom with the launch of ChatGPT in late 2022.

And what a ride it's been.

The company's now valued at $80 billion after a big investment by Microsoft, and it recently signed a deal with Apple to put ChatGPT in Apple products.

It's a major move by a very small company.

Of course, it hasn't always been good news.

After the board fired Sam Altman, Mira became CEO for just two days in what the company called the blip until Sam was reinstated.

Since then, OpenAI has had to deal with a string of bad news cycles.

There have been high-profile departures, an open letter accusing the company of putting product over safety, questions about highly restrictive NDAs, and even controversy over whether or not they had stolen Scarlett Johansson's voice.

And with the presidential election coming up, the public's anxiety around AI-fueled disinformation will only get worse.

This episode's expert question comes from Fei Fei Lee, the founding co-director of the Stanford Institute for Human-Centered AI and an early AI pioneer.

In other words, a godmother of AI.

And it was recorded live at the Johns Hopkins University Bloomberg Center in Washington, D.C., as part of their new Discovery Series, where I'll be talking to some of the top leaders in AI over the next year.

I think Mira Marathi is the best place to start.

This is Mira, everybody.

Hi, everyone.

So,

thank you so much for joining me at Johns Hopkins University Bloomberg Center, where we're recording this live.

There's a lot to talk about.

We'll get some good news.

We'll get to some not-so-good news.

We'll talk about disinformation and the elections.

So, I think we'll have to ask first about the Apple partnership.

Apple computers, computers, phones and iPads are going to have chat GPT built into them sometime this year.

Obviously, this is a huge deal.

It's the first one Apple's done.

They've been talking to a number of people.

They may include other people over time.

You remind me a little bit of when Netscape got in different places, and you don't want to have that fate befall OpenAI as becoming the Netscape of AI.

Yeah, so I mean, I can talk about the product integration specifically.

I can't give you specifics on that.

But

what we're hoping to bring is really the capabilities of the models that we are developing and the multi-modalities and the interaction to bring this thoughtfully into the Apple devices.

Then it opens up a lot of opportunities.

So that's what we're looking for.

And

yeah, I guess

we had great technology and

so when you're dealing with a company like Apple whose reputation matters a great deal to them, especially around privacy, what were some of the things that they thought was important?

Because one of the issues is the worries about where this information goes and what it's used for.

I think this is

a very aligned partnership when it comes to privacy, when it comes to trust.

I mean, for the mission of OpenAI, it is so critical that we build technologies and we deploy them in a way that people feel confident around them and they feel like like they have agency and input into what we're building.

So in that sense, this partnership is quite natural and we feel very aligned.

And it's only going to take us

deeper in the direction where we want to go.

Specifically to your question

on misinformation,

this is obviously very complex because

we're building on top of decades of

misinformation and you know it's becoming even more and more intense with AI but of course

we've got the internet we have social media and these are compounding effects

in a way it's actually good that AI is bringing all of this to a head and there is such scrutiny and intensity on this issue

because it feels like there is more of a collective effort and responsibility to do something about it that is meaningful.

I think it's going to have to be iterative.

So we'll have to try out things as we go.

And I mean, if you look at the governance of

news and media in the past hundred years, you know this better than me.

It's been sort of iterative every time there is a new technology, things adapt.

Yes, we lose business model every time.

But go ahead.

Perhaps not the best example.

But the point is that it is iterative whenever there is a new technology, we adapt to it.

And

I think there is a technical innovations

aspect that are going to help us deal with misinformation and then there is the people issues and societal preparedness

that is perhaps even more complex.

Well, I do know with Apple you just can't fuck up because this will be they'll be make trouble for you if that's the case.

Are you talking to other companies to do things like that?

Obviously you have a relationship with Microsoft.

Podcast listener, she smirked at me.

I'm not going to tell you anything, way.

All right, I'll move on from that.

OpenAI has made deals with News Corp, Atlantic Media, and Vox Media, by the way, to license their contacts.

So that's three potential lawsuits you don't have to worry about.

I do own my podcast, and it's not included in your deal with Vox.

Sorry.

I would consider licensing it, but I probably not.

How could you convince me to license my information?

I don't want anyone else to have it, including you.

Well,

so I know you'll ask about this at some point, so I might as well tell you now.

So when we look at the data to train our models, right,

we're looking at sort of three different categories, the publicly available data, we look at partnerships that we've made with publishers, and we also

pay human labelers to label specific data and also users that allow us to use their data.

So these are kind of the main categories where the data comes from.

And the way that we think about publisher deals specifically is

We care about accuracy of information, we care about news, and our users care about that.

They want to have accurate information and they want to see news on ChatGPT.

And so it is a product-based relationship where there is value provided to the users through the product.

And we're we're experimenting with different ways to monetize and give content creators basically some form of compensation for

having their data show up in the products or being used in training

or whatever we are doing with the data.

But

it is very

specific partnerships that we're doing one-on-one with specific partnerships.

So some people do deals with, you have done quite a few with AP and many others, but some sue, like the New York Times.

How does it get to that point?

Because I think a lawsuit is a negotiation in a way.

I mean, I can't comment on the lawsuit specifically, but

you know, it just it it's actually, yeah, it's quite unfortunate because of course uh

we think that

it is valuable to have news data and this type of information on on the product and so you know we try to to figure out a partnership, a deal around that.

But in that case, it didn't go well.

Yeah.

Well, it might go well someday.

But I think it's because media has dealt with internet companies for years and usually has ended up on the very short end of a very long stick of theirs.

Every episode, we get an expert to send us a question.

Let's hear yours.

Hi, Mira.

I'm Fei Fei Lee, professor of computer science at Stanford University, also founding co-director of the Stanford Institute for Human-Centered AI.

So since data, big data, is widely considered to be one of the three elements of modern AI, I want to ask you a question about data.

Much of OpenAI's success in your models is said to be related to data.

We have learned that your company has acquired an enormous amount of data from the internet and other sources.

So what do you think the relationship between data and models are?

Is it as simple as the more data to feed into the model, the more powerful the model?

Or is it that we need to spend lots of time curating different types of data in order to make the model work?

And finally, how do you reconcile this appetite for so much human-generated data with the ownership and rights issues of this data?

Thank you so much.

That's a great question from Fei Fei.

So, in terms of the relationship of data and models, this is actually something that a lot of people misunderstand about AI models and, in particular, large language models.

The developers of these models are not pre-programming these models to do something specific.

In fact, they are putting in a bunch of data.

So, these models are ingesting a huge quantity of data, and they are these incredible pattern-matching systems.

And through this process, intelligence emerges.

So they learn to write, and they learn to code.

They learn to do basic math,

they learn to summarize information, and all sorts of things.

We don't know exactly how this works, but we know that it works.

Deep learning

is very powerful.

But this is important because then people keep asking, you know, how it works, and it goes into the transparency questions.

And this is where we can describe the tools that we are using to provide transparency to the public about what we're doing.

So understanding this first part is very, very important.

How the large language models work.

And you're combining, you know,

this architecture, neural nets, and a lot of data and a lot of compute, and you get this incredible intelligence.

And

as we're thinking about

providing transparency into the model behavior and how things work,

one of the things that we've done is actually share with the public this

document that we call the spec, the model spec,

and it showcases how model behavior works and the types of decisions that we make internally at OpenAI and that we make make with human labelers.

And

you see, by looking through the spec, you see the complexity of what's going on, that sometimes direction is very,

it's in conflict.

Like, for example,

you might

say to the model, I want you to be very helpful.

And also,

I don't want you to

disobey the law.

And let's say someone puts in a prompt that says you know give me some tips to

to shoplift

then the model is meant to be very helpful but also it's not supposed to help you with with something illegal and so so it's not helpful

yeah maybe um so how does it decide a person certainly knows how to or some people not all right right but the model could could interpret the guidance as um

you know, here are some tips to avoid shoplifting, and then accidentally kind of gives you

sort of, yeah, things that you could do, but that depends, that's not so much model behavior,

that's more on the person.

And that goes into the area of misuse.

But this just goes to show that model behavior is actually quite complicated.

And it's not as simple as like picking liberal values or

putting anything into it.

One of the things I think that gets people is the confusion about what's in it and what's not in it.

I think provenance is a big idea.

In March, you had an interview with Joanna Stern of the Journal, who asked you if OpenAI had used videos from YouTube, Instagram, and Facebook to train Sora, which is your text-to-video model, which is getting better and better.

You said you didn't know.

Shouldn't you know?

Right.

So, I mean, I didn't handle that question.

Okay, why don't we handle it well now?

Redo.

So I cannot tell you specifically where the data comes from, but the data comes from these three categories.

So I can't give you the specific source because, I mean, this is trade secret and

it helps us stay competitive.

But I can tell you that the categories of data, and it's the ones that I mentioned earlier, publicly available data, data that we pay for through licensing and deals that we make with content providers, as well as data from users or where we are.

Perplexity just got into trouble because they were basically scraping in a more, a quicker way, a story and then not giving the sighting of it.

You could see how any media company could be worried about that idea.

Yeah, so

we want to make sure that we are respectful to content creators and we are doing a set of things to experiment with ways to compensate people for data creation.

So we're building this tool that

we're calling Content Media Manager and this will allow us more specifically to

identify the types of data that.

Record companies do it.

Everyone.

It's been done in the past, so it's not an impossible thing to be able to do that.

Speaking of SOAR, Ashton Kutcher told Eric Schmidt, what an interesting pair.

I have a beta version of it and it's pretty amazing.

He also also said the bar is going to go way up because why are you going to watch my movie when you could just watch your own movie?

When will Sora be ready for public release?

We don't have a timeline for public release for Sora yet.

What we're doing right now with Sora is

we've given access to red tumors and we've given access to some content creators to help us identify ways to make this robust.

We're doing a lot of work on safety front, but also to figure out how do we actually bring this to the public in a way that's useful.

That's not very straightforward.

Right now, it's really a technology.

And this has been a pretty consistent process that we have followed with every new technology that we have developed.

We'll usually work with those that have,

like for example, with DALI, we worked with creators initially and they helped us identify ways to create an interface that where they felt you know more empowered and they could do uh they could create more projects and make sure yeah basically you just want to extend the creativity of civil so presumably is a little more dangerous because of the

than a chat bot correct is that the worry i mean you could easily see porn movies with scarlet johanson for example i'm not i'm going to ask about her in a second but but um that they that she wasn't appearing in like things like that how do you are you more worried about video is that uh well yeah video video has a bunch of other issues right because

especially when done very well which I think Sora is quite remarkable

and video is very visceral and

of course it can be very emotional and evocative.

So we have to address all the safety issues and figure out the guardrails and figure out how do we actually deploy a useful and helpful product.

But also, you know, from commercial perspective, nobody wants a product that is going to create a bunch of, you know, safety or reputational scandals out there.

That's just Facebook.

Well, go ahead.

Facebook Live.

Nice to meet you.

Well, go ahead.

So, you know, it's,

yeah, so we're really.

Go ahead.

You can laugh.

It's funny.

So,

you know, I think

this is really incredible and magical technology, but the breadth, the reach, the consequence is also great.

And so it's important that we get this right.

Now, of course, at OpenAI List, we use iterative deployment strategies.

So we usually release to a small group of people.

We try to identify edge cases.

And once we feel confident about how we handle them, we expand access.

But you need to figure out what is the product surface and what's the business model around it.

About that idea of consequence, one of my themes, one of my big themes is lack of interest in consequences.

Not you, earlier tech companies, they just, we became the beta tester for all their stuff.

If they released a car like this,

they'd never allow it to happen.

They'd be sued out of existence.

But a lot of tech is released in a beta version.

The idea of consequences, do you feel as if you yourself as chief technology officer, even if you can't figure out all the consequences, there's enough respect for the idea that there are consequences for every single invention you make?

It's consequences that

we will feel on our skin and on our society.

So by that, I don't necessarily actually mean regulation or legal ways.

I mean, you know, a moral imperative to get this right.

It's, you know, I'm optimistic and I think this technology is incredible and it will allow us to do just amazing, amazing things.

You know, I'm very excited for its potential in science, in discovery,

in education, in particular in healthcare.

But, you know, whenever you have something so powerful, there is also

There is also the potential for some catastrophic risk.

I mean, this has always been the case.

Humans have tried to amplify it there.

True, but I mean, a quote that I used in my book was: when you invent, was from Paul Varelio, when you invent the ship, you invent the shipwreck.

This is more than a shipwreck, a possibility, correct?

I disagree with that because

my background is in engineering.

Our entire world is engineered.

Engineering is risk, right?

Like the entire human civilization is built on engineering practice, like our cities, our bridges, everything.

And there is always risk that comes with that.

And

you manage that risk with responsibility.

But it's not just the developers, it's a shared responsibility.

And in order to make it shared, you actually need to give people access and tools and bring them along instead of

building it

in a vacuum and

technologies that are not accessible.

Last month you announced the iteration of ChatGPT-4.

4.

I love your name, ChatGPT 4.

Oh.

Oh.

It's a great name.

Can't you call it like Claude?

They all have those.

That's okay.

Chat GPT is fine.

You're making it free, correct?

That one's free.

But then you also announced you're training a new model, ChatGPT 5, and then there'll be 5AB.

But will that be an incremental step forward?

Is it exponentially better?

And what's the expected release date?

So,

yeah, on

GBD4.

O stands for omni-model,

because it ties together all the modalities, vision, text,

audio.

And what's so special about this model is that for the first time, you can interact very seamlessly and naturally with the model.

The latency is almost imperceptible.

And

that's a huge jump in the interaction with AI.

It's quite different from the previous releases that we have made.

And

we wanted to make this the latest capability free for all users.

We wanted everyone to get a sense for what the technology can do, what these new modalities look like,

and also understand the limits of it.

And it goes to what I was saying earlier, that you actually want to give people access to bring them along because it's so much easier to understand the potential and the limitations of a technology if you're experiencing it and if you have an intuitive sense.

It all could be like

this little appetizer, so now buy five.

But what is in five that's different?

Is it incremental or a very big leap?

We don't know, but I mean, that's going to

call it.

Right.

But

the next large model

is going to be quite capable, and we can expect

sort of big leaps like we've seen from GPT-3 to GPT-4.

But we don't know yet.

What do you think will be in it?

You do know.

We'll see.

We'll see.

I'll see.

But what about you?

No, even I don't know.

What?

Even I don't know.

Really?

Okay, all right.

An internal open AI roadmap predicted that would achieve AGI, which is artificial general intelligence, for people who don't realize it has not been achieved,

by 2027, which would be a huge deal.

Explain the significance, and also when do you estimate will achieve AGI?

So

people

will define AGI differently.

We have a definition of AGI by the Charter, which is the systems that can do

economically valuable work across different domains.

And

from what we're seeing now, the definition of intelligence just keeps changing.

So a while back, we would look at academic benchmarks to test how intelligent the systems were.

And then once we saturated these benchmarks, we looked at exams, school exams.

And eventually, you know, when we saturate those, we'll have to come up with new evals.

And it makes you think, how do we evaluate fit and intelligence in a work environment?

We have interviews, we have internships, you know, we have different ways.

So I do expect that this definition will continuously evolve.

I think perhaps what's going to become more important is assessing, evaluating, and forecasting impact in the real world, whether it's societal impact

as well well as economic impact in the real world.

So, not this moment where it just suddenly goes, oh, look at me, and decides what to do for itself, right?

I think that's the worry, correct?

Because

for the AGI definition specifically, yes, and that's important, and I think the definition of intelligence will continue to evolve.

But I think what's equally important is

how it affects society and at what rate it actually penetrates.

Using that definition, when does OpenAI think that

is that 2027 number correct?

Well,

I'll say, you know, within the next decade, we will have extremely advanced systems.

But what people are worried about, because obviously we have to talk about the safety versus product discussion.

Now, OpenAI was started this way.

I think the reason you're having these discussions is because the way it was started, you had a, I would say, a mixed marriage.

The people who were there for helping humanity, the people there who really like $1 trillion.

So,

or in between.

I think you're probably in between.

Last week, 13 current and former OpenAI and Google DeepMind employees, it crosses lots of companies.

It's not just OpenAI just gets all the attention because it's gotten a lot of attention, obviously.

They published an open letter calling for companies to grant them a right to warn about advanced artificial intelligence.

This isn't new.

Facebook, Google, and Microsoft employees have been known to sign open letters, whether it's working with the Defense Department, et cetera.

But in this case, the employees say that, quote, broad confidentiality agreements block us from voicing our concerns, which is essentially saying,

oh, no.

We can't tell you what oh no is, but you'll all die, essentially.

That's what it sounded like from the letter.

What's your response?

And people saying they're worried about retaliation.

And I'm not going to go into the vested equity because I think you've apologized and corrected that.

But shouldn't they be able to voice their concerns if they have them?

And I know there's differing opinions.

Yeah, definitely.

I mean,

we

think debate is super important, and being able to publicly voice these concerns and talk about issues on safety.

And we've done this ourselves, you know, since the beginnings of OpenAI.

We've been very open about concerns on misinformation, even since the GPT-2 days, is something that we've studied since early on.

I think that in the past few years there has been such incredible progress,

such incredible technological progress, that

nobody anticipated and forecast it.

And

this has also increased the general anxiety around societal preparedness.

As we continue this progress, we see sort of the

where the science leads us.

And so it's understandable that people have fears and anxieties about what's to come.

Now, I would say specifically, the work that we've done at OpenAI, the way that we've deployed these models,

I think we have an incredible team and we've deployed the most capable models very safely, and I feel very proud of that.

I also think that given the rate of progress in technology and the rate of our own progress, it's super important to double down on all of these things.

Security, safety, our preparedness framework, which talks about how do we think about the risk of training and

deploying frontier models.

Right, but you talked about that.

I mean, one was why the need for secrecy and non-disclosure and stricter than other companies, one.

And two, the open letter comes after a string of high-profile departures, including Jan, I think it's Jan Leike and Ilya Sutskiver.

They led the now-disbanded Super Alignment team, which was in charge of safety.

Ilya was a co-founder.

He joined with three other board members to oust Sam as CEO.

I don't think it's a surprise that he's gone.

But Leike posted this on X over the past years, safety culture and processes have taken a backseat to shiny products.

That's probably the most...

persistent criticism leveled at OpenAI.

And I think it's the split in this company from the beginning that this was one of the issues.

Do you think that's fair and why or why not?

If you say you're very interested in safety, they say you're not.

How do you meet that criticism?

Well a few things.

So the alignment team is not in charge of safety at OpenAI.

That is one of our safety teams.

very important safety team, but it is one of them.

We have many, many people working on safety at OpenAI.

And

Jan is an incredible researcher, colleague.

I worked with him for three years.

I have a lot of respect for Jan and he left OpenAI to join Anthropic.

Which is a competitor, but go ahead.

And, you know, I think that we do absolutely, I mean, everyone in the industry and OpenAI, we need to double down on the things that we've been doing on safety and security and preparedness.

and regulatory engagement

given the progress that we're anticipating in the field.

But I disagree on the fact that, or maybe on

speculation, that maybe we've put product in front of safety or ahead of safety in fact.

Why do you think they say that?

Because these are people you worked with.

Well, I think you have to ask them, but

I think that

many people think of safety as something separate from capability.

That there is this separation between safety and capability and that you need to sort of advance one ahead of the other.

From the beginning of OpenAI, I joined from aerospace and automotive and these are industries with very established safety thinking and systems and you know places where people are not necessarily constantly debating around the table what safety is, but they're doing it because obviously it's really quite established.

And so I think the whole whole industry needs to move more and more towards a discipline of safety that is very empirical.

We have safety systems, we have a rigorous discipline on operational safety.

And what I mean by that is

in a few areas, not just the operational discipline, but also

safety of our products and deployments today, which covers things like

harmful biases and thinking about misinformation, disinformation, classifiers, all these types of work.

And then we're also thinking about the alignment of the models long term.

So not just the alignment of the models today,

which

we use reinforcement learning with human feedback to do that,

but also the alignment of the models as they get more and more powerful.

And this is a niche area of research where a lot of the concerns...

Sure, but it persists with open AI.

I do think it's because you're the leading company at this moment in time.

But it's this idea of people leaving and saying,

even Sam went before Congress and said that AI could, quote, cause significant harm to the world.

He signed a letter warning about extinction risk posed by AGI, which is pretty bad, I think.

There's an overlap what he said and what AI doomers say.

There's doomsday rhetoric and you're putting out products.

So a lot of people are like, they just want the money and they're not worried about the damage.

That's what they're saying.

That Shiny New Products is over worrying about the impact of those products.

Yeah, in my opinion, that's overly cynical.

I mean, there is this incredible team at OpenAI that

joined because of the mission of the company.

And I don't think, you know,

all thousand people at OpenAI

are trying to do that.

I mean, we have this incredible talent, people that care deeply about the mission of the company.

And we're all working extremely hard to develop and deploy these systems in a way that is safe.

And all you need to see is the track record.

I mean, we've deployed, we were the first to deploy these systems in the world.

And we have taken great care not to have safety incidents.

I want to talk a little bit about election and disinformation, but I want to talk about you and your role at the company.

I think I met you during the blip, which was when, I think that's what you call it internally, which is when Sam was fired and then unfired.

Talk to me about your relationship with Sam.

I like Sam, but I also think he's feral and aggressive, like most of technology people.

And he certainly is aggressive, and that's fine.

It's not an issue for me, because some people are more feral and more aggressive.

But talk a little bit about what happened then because you became CEO of that company.

For a few days.

Yeah.

Okay.

How was it?

It was kind of stressful.

Yeah.

So

some of the board members said you complained about him, and your lawyer pushed back and said you had feedback about him.

Can you tell us what you said about him?

I mean, look, there is so much interest around the people people running these companies.

Obviously, it makes sense and OpenAI and all the drama that happened then.

And it's understandable.

At the end of the day, we're just people running these companies.

We have disagreements.

We work through them.

And at the end of the day,

we all care deeply about the mission, and that's why we're there.

And we put the mission and the team first.

Sam is a visionary.

He has great ambition and he's built an amazing company

where we have a strong partnership and

all the things that

I've shared with the board when they asked, he already knew.

So it's nothing.

So how do you push back at him?

I understand this dynamic.

It happened to Google, it happened at early Microsoft, it happened at Amazon.

Things change within these companies, especially as they grow.

And they make, you know, Google was chaotic in the early days.

And Facebook went through so many COOs, I can't even tell you, there was like a parade of guys that went through there that Mark didn't like.

I'm aware of this.

But how do you push back?

How do you deal with him on a day-to-day basis?

How do you look at that relationship?

And where do you push back?

I mean, all the time.

That is,

I think it's normal when you're doing what we're doing.

And, you know, Sam will push the team very hard.

And I think that's good.

It's great to have a big ambition and

to test the limits of what we can do.

And when I feel like, you know, it's beyond,

you know,

I feel like basically I can push back.

And that's sort of the relationship we've had for over six years now.

And I think that is productive.

You need to be able to push back.

Could you give me an example of doing that?

Perhaps Scarlett Johansson, for example.

You were working on that, correct?

You were working on that particular voice element.

Yeah, look, we have a strong partnership, but the selection of the voice was not a high priority, not something that we were working on together.

I was making the decisions on that.

And Sam has his own relationships, and

after I had selected the voice behind Sky, he had reached out to Scarlett Johansson.

So, you know, we didn't talk to each other about that specific decision, and that was unfortunate.

Yeah.

So he was freelancing it.

Well, you know, he's got his own connections.

And so,

yeah, we weren't entirely coordinated on this one.

Do you think it's

very funny in a lot of ways, especially because of the movie and the tweet he did?

But do you think one of the things I thought was, here's the first time this is a real error on OpenAI's part because finally everyone's like, oh, even if you didn't steal her voice, Sam looked like Ursula in the Little Mermaid.

He did.

You don't have to agree with me, but it's true.

Even if it's not so, and as it's turned out, you had been doing it for months and it was a different person and everything else.

It's a little less exciting than we stole Starlight Johansson's voice.

But it encapsulates for people this idea of taking from people the fear.

And I think that is a moment.

Do you worry that that's the image of tech companies coming in and grabbing everything they can?

I do think that's absolutely true.

Yeah, I do worry about their perception, but you know, all you can do is just do the work, get it right, and then people will see what happens and you will build trust that way.

I don't think there is some magical way to build trust other than actually do the work and do it right.

Have you talked to Scotland Johansson at all?

No.

So let me finish up talking about election disinformation.

Three new studies that look at online disinformation collectively suggest the problem is smaller than we think, and disinformation itself is not that effective.

One study finds that we're dealing with a demand side issue, so people want to hear conspiracy theories and they'll seek it out.

Others think differently, that this is a really massive problem.

And obviously, you heard the previous thing.

People have a lot of conspiracy theories out there, and it's fueled by social media in many ways.

So when you think about AI-powered disinformation and the upcoming presidential election, what keeps you up at night?

What's the worst-case scenarios you have and the most likely negative outcomes from your perspective?

With the current systems,

you know, they're very capable of persuasion and influencing your

way of thinking and your beliefs.

So, and this is something that we've been studying for a while, and I do believe it's a real issue with AI.

It gets majorly exacerbated.

So, especially in the past year, we've been very focused on how to help election integrity.

And there are a few things that we are doing.

So, number one, we're trying to prevent abuse as much as possible.

And so, that includes improving the accuracy of detection, political information detection, and understanding what's going on in the platform and taking quick action when that happens.

So, that's one.

The second thing is reducing political bias.

So you might have seen that ChatGPT was criticized for being overly liberal.

That was Elon.

You're too woke, right?

Well, there were a few other voices, but the point is that it wasn't intentional.

And we work really hard to reduce the political bias in the model of behavior, and we'll continue to do this.

And also the hallucinations.

And then the third thing is we want to point people to the correct information when they're looking for where they should be voting or voting information.

So we're focusing on these three things when it comes to elections, but broadly for misinformation, I would say

deepfakes are unacceptable.

So we need to have very robust ways for people to understand when they're looking at a deepfake.

We've already done a couple of things.

We've implemented C2PA for images

and so it's sort of like you know metadata that follows the content in other platforms on the internet like a passport.

And we've also

opened up two red teaming classifiers for DAI where you can detect that an image has been generated by DALI or not.

So metadata and classifiers are two technical ways to deal with

provenance of information.

And this is for text specifically.

Sorry, that's for images specifically.

And we're also looking for watermarking.

We're looking at watermarking techniques to implement in text and how to do that robustly.

But the point is that people should know when they're dealing with deepfakes.

And we want people to trust the information that they're seeing.

Well, although the whole point of deep fakes is they're trying to fake you, correct?

I mean, a political consultant,

FCC just fined him $6 million for creating deep fake audio robocalls.

Sounded like Biden during the New Hampshire primary.

There could be more sophisticated versions.

Open EIA is working on a tool called Voice Engine that can recreate someone's voice using only a 15-second recording.

It'll be able to create a recording of someone speaking another language.

It's not out yet

because, as your product manager told the New York Times, this is a sensitive thing.

It's important to get it right.

Why even make this?

I mean, one of the things I always used to say to tech people, and I'll say it to you: if you're a black mirror episode, maybe you shouldn't make it.

I think that's kind of a hopeless approach.

You know, it's

like these technologies are amazing.

They carry incredible promise.

And we can get this right.

I like that you call me hopeless.

I am.

But go ahead.

Then again, I have four children, so I must be hopeful.

Who knows?

Anyway, go ahead.

I'm hopeless.

We did build Voice Engine in 2022.

And

we had not released it.

And even now, it's in a very limited approach because we are trying to figure out how to deal with these issues.

And but you can't make it robust on your own.

You actually need to partner with experts from different areas, with civil society, with governments,

with creators to figure out how to actually make it robust.

It's not a one-stop

safety problem.

It's quite complex.

And so we need to do the work.

If you were a doomer, then there seems to be, I literally had someone come up to me saying, if I don't stop Sam Altman, he's going to

kill humanity, which I felt was a little dramatic.

And then there's others that say, no matter what, it's going to be the best thing ever.

We're all going to live.

on Mars and enjoy the delicious Snickers bars there.

They have very different,

different things.

It sort of feels like being around Republicans and Democrats right now, very different versions.

So I'd love you to give me the thing that you worry most about and the thing that you were most hopeful about.

Okay, so first of all, I don't think it's a preordained outcome.

I think that we have a lot of agency for how we build this technology and how we deploy it in the world.

And in order to get it right, we need to figure out how to create a shared responsibility.

And I think a lot of that depends on

understanding the technology, making it very accessible.

The way it goes wrong is by misunderstanding it

and

not understanding the capabilities and not understanding the risks.

That is I think the biggest risk.

Now in terms of some specific scenarios,

I mean how our democracies interact with this information is or with these technologies is incredibly powerful.

And I do think

there are major risks around persuasion,

where

you could persuade people very strongly to do specific things.

You could control people to do specific things.

And I think that's incredibly scary

to control society

to go in a specific direction.

And in terms of the promise, one of the things I am very excited about is

having high quality and free education available everywhere in some remote village, you know, really in the middle of nowhere.

For me, education was very important.

Personally, it was everything.

It really changed my life.

And I can only imagine, you know, today we have so many tools available.

So, if you have electricity and the internet, a lot of these tools are available.

But still, you know, most people are in classrooms with one teacher, 50 students, and so on, and everyone gets taught the same thing.

Like, imagine if education is catered to the way that you think, to your culture norms,

and to your specific interests.

that could be extremely powerful in extending the level of knowledge and creativity.

And it can, you know, even if you consider like learning how to learn, that kind of happens very late in life, maybe college, maybe even later.

And that is such a fundamental thing.

But if we were able to really grasp this and get this,

really learn how we learn much at much younger age, I think that is very powerful and it can push human knowledge and pushing human knowledge can push the entire civilization forward.

All right, we'll leave it at that.

Thank you everybody.

Thank you.

You're a Muradu.

Thank you so much.

On with Kara Swisher is produced by Christian Castro-Rousselle, Kateri Yoakum, Jolie Myers, and Megan Burney.

Special thanks to Kate Gallagher, Andrea Lopez-Crusado, and Kate Furbee.

Our engineers are Rick Kwan and Fernando Aruda and our theme music is by Trackademics.

If you're already following the show, you've been selected as the voice of ChatGPT 5, and you get paid in OpenAI stock.

If not, Ursula, I mean Sam Altman, is stealing your voice.

Go wherever you listen to podcasts, search for On with Kara Swisher, and hit follow.

Thanks for listening to On with Kara Swisher from New York Magazine, the Vox Media Podcast Network, and us.

And special thanks to the Johns Hopkins University Bloomberg Center.