Will AI Radically Change the World by 2027?

Transcript

Speaker 1 Pushkin.

Speaker 1 This is an iHeart podcast.

Speaker 2 In today's super competitive business environment, the edge goes to those who push harder, move faster, and level up every tool in their arsenal. T-Mobile knows all about that.

Speaker 2 They're now the best network, according to the experts at OoCla Speed Test, and they're using that network to launch Supermobile, the first and only business plan to combine intelligent performance, built-in security, and seamless satellite coverage.

Speaker 2 That's your business, Supercharged. Learn more at supermobile.com.
Seamless coverage with compatible devices in most outdoor areas in the U.S. where you can see the sky.

Speaker 2 Best business plan based on a combination of advanced network performance, coverage layers, and security features. Best network based on analysis by OOCLA of Speed Test Intelligence Data 1H 2025.

Speaker 1 When you own your own business, you own every decision. Now own the card that rewards you for it.

Speaker 1 Chase Sapphire Reserve for Business is a painful card that elevates your travel experience and offers premium benefits that will take your business to the next level.

Speaker 1 Sapphire Reserve for Business offers 8X points on all purchases through Chase Travel, 3X points on social media and search advertising, airport lounge access, and more.

Speaker 1 With over $2,500 in annual value, it's the card that gives back all you put in. Learn more at chase.com forward slash reserve business.
Chase for business. Make more of what's yours.

Speaker 1 Accounts subject to credit approval. Restrictions and limitations apply.
Cards are issued by JP Morgan Chase Bank NA, member FDIC.

Speaker 1 And now, Superhuman Shack.

Speaker 3 I keep telling them not to say that. I'm no superhuman.
Believe it or not, I struggle with moderate obstructive sleep apnea, or OSA.

Speaker 3 In adults with obesity, moderate to severe OSA is a condition where breathing is interrupted during sleep with loud snoring, choking, gasping for air, and even daytime fatigue.

Speaker 3 Let's just say it can sound a lot like this.

Speaker 3 Sound familiar? Learn more at don't sleep on OSA.com.

Speaker 1 This information is provided by Lilly, a medicine company.

Speaker 1 Hey, everybody, Nate here, jumping in before the show to say that we've been having a lot of fun answering your listener questions.

Speaker 1 So far, we've covered things like Pascal's wager, the hot hand fallacy, or the fallacy of the hot hand fallacy, actually, and the expected value of learning new languages.

Speaker 1 We want to keep doing this kind of thing, so send us all your questions about risk, decision-making, game theory, poker, you name it.

Speaker 1 Reach out to us on social media or email at riskybusiness at pushkin.fm.

Speaker 1 Even if you're not a premium subscriber, this is a great way to support the show so we can keep sharing it free of charge. We look forward to hearing from you.

Speaker 1 Welcome back to Risky Business, a show about making making better decisions. I'm Maria Konakova.
And I'm Nate Silver. Nate, today on the show is going to be a little bit doom-tastic.

Speaker 1 Yeah, I mean, I don't know if it's like worse than thinking about like the

Speaker 1 global economy going into a recession because of dumb fuck tariff policies.

Speaker 1 This is all about how we're all going to die in seven years instead. No, I'm just kidding.

Speaker 1 This is a very, very intelligent and well-written and thoughtful report called AI 2027 that we're going to spend the whole show on because I think it's such an interesting sub to talk about, but that, you know, includes some

Speaker 1 dystopian possibilities, I would say. It does indeed.
So let's get into it and hope that you guys are all still here to listen to us in seven years.

Speaker 1 The contrast is interesting between like all the

Speaker 1 chaos we're seeing with tariff policy in terms of starting a trade war with China and then other types of chaos.

Speaker 1 It's interesting to kind of look at this. I mean, I wouldn't call it a more optimistic future exactly,

Speaker 1 but like on a different trajectory of a future that's going to change very fast, according to these authors, with profound implications for, you know, everything, the human species.

Speaker 1 These researchers and authors are saying that everything is going to change profoundly. And

Speaker 1 even though there is some

Speaker 1 hedging here, here, this is kind of their base case scenario. And like, you know, base case number one and base case number two differ.

Speaker 1 There's like kind of a choose your own adventure at some point in this report. But they're both very different in the status quo, right?

Speaker 1 And the notion that you hear from AI researchers is that like everything becomes different if AIs become substantially more intelligent than human beings.

Speaker 1 People can debate and we will debate on this program what that means. But yeah,

Speaker 1 do you want to contextualize this, Mario? Do you want to tell people what the values are of this report? Absolutely. Absolutely.

Speaker 1 So the report is authored officially by five people, but I think unofficially there's also a sixth. We've got...

Speaker 1 Eli Leiflund, who's a super forecaster,

Speaker 1 and he was ranked first on RAND's forecasting initiative. So he is someone who is very good at kind of looking at the future, trying to predict what's going to happen.

Speaker 1 You have Jonas Vollmer, who's a VC at Macroscopic Ventures. Thomas Larson was the former executive director of the Center for AI Policy.
So that is a center that advises both sides of the aisle on

Speaker 1 how AI is going to go. And Romeo Dean, who is

Speaker 1 part of Harvard's AI safety student team. So someone who is still a student, still learning, but kind of the next generation of people looking at AI.

Speaker 1 And finally, we have Daniel Coctelo, who basically had written a report back in 2021. He was an AI researcher, and he looked at predictions for AI for 2026.

Speaker 1 And it turns out that his predictions were pretty spot on. And so OpenAI actually hired Daniel as a result of this report.
Previously, he's now left angry. Yes.
And then he left. Exactly.

Speaker 1 And importantly,

Speaker 1 there's also Scott Alexander. Exactly.
So I was saying he's the person who kind of is in the background. And

Speaker 1 you guys might know him as the author behind Astral Codex.

Speaker 1 And I know Scott. He's one of the kind of fathers of

Speaker 1 what you might call rationalism.

Speaker 1 I think Scott, when I interviewed him for my book, was happy enough with that term and accused me or co-opted me into also being a rationalist.

Speaker 1 These people are somewhat adjacent to the effective altruist, but not quite, right?

Speaker 1 They're just trying to apply a sort of thoughtful, rigorous, quantitative lens to big picture problems, including existential risk, of which most people in this community believe that AI is both an existential risk and also kind of an existential opportunity, right?

Speaker 1 That it could transform things. You talk to Sam Alp and he'll say, we're going to cure cancer and eliminate poverty and whatever else, right? And Scott's also an excellent writer.

Speaker 1 And so let me disclose something which is slightly important here. So I actually was approached by some of the authors of this report a couple of months ago.
I guess it was in February-ish,

Speaker 1 just to give feedback and chat with them. So, I'm working off the draft version, right? Which I do not believe they changed very much.
So, my notes pertain to an earlier draft.

Speaker 1 I did not have time this morning to go back and reread it.

Speaker 1 So, I have, I was not on the inside loop, so I did not get an earlier draft. And I've read this draft.
And basically, just to kind of big picture, sum it up,

Speaker 1 it outlines two scenarios, right? Two major scenarios for how AI might change the world as soon as 2030. Now, important note, like that,

Speaker 1 that date is kind of hedged. It might be sooner, it might be later.
There's kind of a, there's a confidence interval there.

Speaker 1 But the two different scenarios, one, basically, P doom 2030, humanity disappears and is taken over by AI.

Speaker 1 The positive report is in 2030, basically we get AIs that are aligned to our interests and we get kind of this AI utopia where AIs actually help make life much better for everyone and make the standard of living much higher.

Speaker 1 But the crucial turning point is before 2030. And the crucial kind of question at the center of this is

Speaker 1 will

Speaker 1 we be able to design AIs that are truly aligned to human interests rather than just appear to be aligned and kind of lying to us while actually following their own agenda.

Speaker 1 And how we handle that is kind of the linchpin.

Speaker 1 And it's actually interesting, Nate, that you started out with China because a lot of the policy choices and a lot of what they see as kind of the decision points that will affect the future of humanity actually hinge on the U.S.-China dynamic, how they compete with each other, and how that sometimes might basically clash against safety concerns because no one wants to be left behind.

Speaker 1 Can we manage that effectively? And can kind of that transition work in our favor as opposed to against us? I think that this is kind of one of the big questions here.

Speaker 1 And so it's funny that we're seeing all of this trade war right now as this report is coming out.

Speaker 1 Yeah, look, I think this exercise is

Speaker 1 partly just a forecasting exercise. I mean, you know, obviously there's this kind of like

Speaker 1 fork at the bottom where we learn to have an AI slowdown or we kind of are pressing fully on the accelerator. In some ways, scenarios are like not

Speaker 1 that different, right? Either one assumes

Speaker 1 remarkable rates of technological growth that I think even AI, I'm never quite sure who to call an optimist or a pessimist, right? Even AI believers.

Speaker 1 you know, might think is a little bit aggressive, right?

Speaker 1 But what they want to do is they want to have like a specific fleshed out scenario for how the world would look like it's kind of like a modal scenario and like i think they'd say that like

Speaker 1 we're not totally sure about either of these necessarily right and i don't think they'd be as like pedantic as to say if you do x y and z then we'll save the world and have utopia and if you don't then we'll all die right i think they probably say it's it's unclear and there's kind of like risk either way and we wanted to go through the scenario of like fleshing out like what the world might look like right i do think one thing that's important is that whatever decisions are made

Speaker 1 now

Speaker 1 could get locked in, right? That you, you pass certain points in overturn

Speaker 1 and it becomes very hard to decelerate like an arms race. This is, you know, what we found during the Cold War, for example.

Speaker 1 I mean, one of the big things I look at is like, do we force the AI to be transparent in its thinking? with humans, right?

Speaker 1 Like now there's been a movement toward the AI will actually explicate its thinking more. I'll ask it a query, OpenAI.

Speaker 1 The Chinese models do this too, right? And they'll say, I am thinking about X, Y, and Z, and I'm looking up P, D, and Q, and now I'm reconsidering this.

Speaker 1 It actually has this chain of thought process, right? Which is explicated in English.

Speaker 1 You know, one concern is that what if the AI just kind of communicates to one another in these implicit vectors that it's inferring from all the text it has.

Speaker 1 It's kind of unintelligible to human beings, right?

Speaker 1 It may be kind of quote unquote thinking in that way in the first place and then does us the favor of like translating back, so it goes from English to kind of

Speaker 1 this big bag of numbers, is one AI researcher called it, right?

Speaker 1 And then it translates it back into English or whatever language you want, really, in the end.

Speaker 1 What if it just cuts out that last step, right? Then we can't kind of like check what AI is doing, then it can behave deceptively more easily. So, you know, so that part seems to be important.

Speaker 1 I want to hear your

Speaker 1 first impressions before I kind of poison the well too much.

Speaker 1 Well, my first impressions is that the alignment problem is a very real one and an incredibly important one to solve.

Speaker 1 And what I got from this is that actually the problem that I've had with like these initial AI LLMs is the kernel of what they're seeing there, right?

Speaker 1 So, you and I have talked about this on the show in the past, and I've said, well, my problem is that when I'm a domain expert, right, I start seeing some inaccuracies inaccuracies and I start seeing like places where like it either just didn't do well or made shit up or whatever it is.

Speaker 1 Now, I think it's very clear that those problems are going to go away, right? That that is going to get much, much better.

Speaker 1 However, the kernel of it's showing me something, but that might just be, you know, I have no way of verifying if that's what's going on, what it's reading, like how it's, I don't want to say thinking about it, even though in the report they do use thinking, but it's

Speaker 1 too much. I think thinking is correct.
I think it's okay. We'll stick to that language.

Speaker 1 Okay. So, so how it's thinking about it,

Speaker 1 that those little problems and like the little

Speaker 1 glitches and the things that it might be doing where it starts actually glitching on purpose are not going to be visible to the human eye.

Speaker 1 And so, one of the main things that they say here is that as AI internal RD gets rapidly faster, So that means basically AI is researching AIs, right?

Speaker 1 And so internally they start developing new models.

Speaker 1 And as they kind of surpass human ability to monitor it, it becomes progressively more difficult to figure out, okay, is the AI actually doing what I want it to do? Is the output that it's giving me?

Speaker 1 its actual thought process and is it accurate or is it like trying to deceive me but it's actually kind of inserting certain things on purpose because it has different goals, right?

Speaker 1 Because it is actually secretly misaligned, but it's very good at persuading me that it's aligned.

Speaker 1 Because one of the things that actually came out of this report, and I was like, huh, you know, this is interesting, is if we get this

Speaker 1 remarkable improvement in AI, it will also remarkably improve at persuading us, right? So this part I know. This is one part I don't buy.

Speaker 1 Yeah, so this is, this is, but I'd never even thought about that. I was like, okay, fine.

Speaker 1 But one of the things that I do buy is that it's going to be very difficult for us to monitor it and to figure out, like, is it truly aligned with human wants, with human desires, with human goals?

Speaker 1 And the experts who are capable of doing that, I think, are actually going to dwindle, right, as AI starts proliferating in society. And so to me, that is something that is actually quite worrisome.

Speaker 1 And that is something that we really need to be paying attention to.

Speaker 1 Now, just to fast forward a little bit, in their doomsday scenario in 2031, when ai takes over it basically like suddenly releases some chemical agents right and humanity dies and the rest of the stragglers are taken care of by drones etc i don't even like it's a quick and painful

Speaker 1 quick and painless death i will say on the fastest

Speaker 1 let's hope we don't know what the chemical agents are might not be quick and painless some chemical agents are actually a very painful death mate so let's hope let's hope it's quick and painless uh quick and quick yes definite quick okay hopefully

Speaker 1 Some chemical agents are not quick.

Speaker 1 Let's hope it's quick and painless.

Speaker 1 But if they're actually capable of deception at that high level, then you technically don't even need them to do it.

Speaker 1 If we're trusting medicine and all sorts of things to the AIs, it's pretty easy for it to actually manipulate something and actually insert something into codes, et cetera, that will fuck up humanity in a way that

Speaker 1 we can't actually figure out at the moment.

Speaker 1 The way I think of it, and

Speaker 1 this is not from the paper, but this is just the way that my mind processed it, is like think about DNA, right? Like you have these remarkably complex, huge strands of data.

Speaker 1 And as we've found out, but it's taken forever, one tiny mutation can actually be fatal, right? But you can't spot that mutation.

Speaker 1 Sometimes that mutation isn't fatal immediately, but will only manifest at a certain point in time. That's the way that my mind tried to kind of try to conceptualize what this actually means.

Speaker 1 And so I think that, you know, that would be easy for a deceptive AI to do.

Speaker 1 And to me, like that that's kind of the big takeaway from this report is that we need to make sure that we are building AIs that will not deceive, right? That their

Speaker 1 capabilities, they explain them in an honest way, and that honesty and trust is actually prioritized over other things, even though it might slow down research, it might slow down other things, but that that kind of alignment step is absolutely crucial at the beginning, because otherwise humans are human, right?

Speaker 1 They're easily manipulated. And we often trust that computers are quote unquote rational because they're computers, but they're not.

Speaker 1 They have their own inputs, they have their own weights, they have their own values, and

Speaker 1 that could just lead us down a dark path. Yeah, so let me follow up with this pushback, I guess, right? Like, first of all, I don't know that humans are so easily persuaded.

Speaker 1 This is my big critique with like all the misinformation people who say, well, misinformation is the biggest problem that society faces.

Speaker 1 It's like people are actually pretty stubborn and they're kind of to sound pretentious. They're kind of Bayesian in how they formulate their beliefs, right? They have some notion of reality.

Speaker 1 They're looking at the credibility of

Speaker 1 the person who is telling them these remarks.

Speaker 1 If it's an unpersuasive source, it might make them less likely to believe they're balancing with other information with their lived experience, so-called, right?

Speaker 1 You know, part of the reason that I am skeptical of AIs being super persuasive is like,

Speaker 1 you know that it's an AI and you know it's trying to persuade you. You know what I mean?

Speaker 1 So like if you go and play poker against like a really chatty player, Phil Hemmoth or Scott Sieber or someone like that, right?

Speaker 1 You know on some level that the best play is just to totally ignore it, right? You know that they are trying to sweet talk you into doing exactly what they want you to do.

Speaker 1 And so the best play is disengage or literally you can randomize your moves if you have some notion of what the game theoretical optimal play might be, right?

Speaker 1 Or salesmen or politicians have reputations for being, oh, he's a little too smooth. Gavin Newsom, a little too fucking smooth, right? I don't find Gavin Newsome persuasive at all, right?

Speaker 1 He's a little too from the hair gel to the constantly shifting vibes.

Speaker 1 I mean, I don't really find Gavin Newsome persuasive at all, even though like in AI, I'd say, boy, Gavin Newsom's a good-looking guy. He's a little gravely-throated, but, you know, whatever.

Speaker 1 I mean, look, the big critique I have of this project, and by the way, I think this is an amazing project.

Speaker 1 In addition to like wonderful writing, if you view it on the web, not your phone, because they have these very cool like little infographic that updates everything from like the market value of, they don't call it open AI, they call it open brain, I guess is what they settle on for a substitute right now.

Speaker 1 Yeah, they call everything something else just to make sure that they're not um stepping on any toes so they have open brain and then uh they have deep scent from china i wonder which one that could be i wonder uh

Speaker 1 but it's beautifully presented and and written and like i appreciate their going out on

Speaker 1 a limb here you know i mean i think they have got it's been fairly well received i've gotten some pushback both from inside and i think outside the ai safety community right but they're putting their necks in the line here they will look if we if if things look pretty normal if the world looks pretty normal in 2032 or whatever, right, then they will look dumb for having published this.

Speaker 1 Well, and and they actually have that, right, as a scenario that you end up looking stupid if everything goes well, but that's okay. Now, can I push back on the persuasion thing a little bit?

Speaker 1 Um, just on two, two things.

Speaker 1 So, first of all, the poker example is not actually a particularly applicable one here because you know that you're playing poker and you know that someone is trying to get information and deceive you.

Speaker 1 The tricky thing, so this is kind of when I spent time with con artists, the best con artists aren't Gavin Newsom. Like they're not car salesmen.

Speaker 1 You have no idea they're trying to persuade you to do something. They are just like nice, affable people who are incredibly charismatic.

Speaker 1 And even in the poker community, by the way, like some of the biggest grifters who like it comes out later on were just like stealing money and doing all of these things are charming, right?

Speaker 1 They're not sleazy looking. Like they have no signs of, oh, I'm a salesman, I'm trying to sell you something.
The people who are actually good at persuasion, you do not realize you're being persuaded.

Speaker 1 And I think people are incredibly easy to kind of to subtly lead in a certain direction if you know how to do it. And I think AIs could could do that.

Speaker 1 And they might persuade you when you don't even think they're trying to persuade you. You might just ask, like, can you please summarize this research report? And the way that it frames it, right?

Speaker 1 The way that it summarizes it just subtly changes the way that you think of this issue.

Speaker 1 We see that in psych studies all the time, by the way, where you have different articles presented in slightly different order, slightly different ways.

Speaker 1 And people from the same political beliefs, you know, same starting point come away with different impressions of what kind of the right course of action is or what this is actually trying to tell you, because the way the information is presented actually influences how you think about it.

Speaker 1 It's very, very easy to do subtle manipulations like that.

Speaker 1 And if we're relying on AI on a large scale for a lot of our lives, I think that if it has like a quote unquote master plan, you know, the way that they present in this report, then persuasion in that sense is actually going to be,

Speaker 1 but you'll know you're being manipulated, right? That's the issue. No, you don't.
No, that's the thing you don't know you're being manipulated. You don't have to adapt because it's AI and the people.

Speaker 1 I don't know.

Speaker 1 I honestly, like Nate, I applaud your belief in humans' ability to adjust to this, but I don't know that they will because I've just seen enough people who are incredibly intelligent fall for cons and then be very unpersuadable that they have been conned, right?

Speaker 1 Instead doubling down and saying, no, I have not.

Speaker 1 So humans are stubborn, but they're also stubborn in saying, I have not been deceived, I have not been manipulated, when in fact they have, to protect their ego and to protect their view of themselves as people who are not capable of being manipulated or deceived.

Speaker 1 And I think that that is incredibly powerful. And I think that that's that's going to push against your optimism.
I hope you're right, but from what I know, I don't think you are.

Speaker 1 I'm not quite sure I call it optimism. So I guess maybe we do like slightly different views of human nature, but like there's not yet a substantial market for like AI driven art or writing.

Speaker 1 And I'm sure there will be one eventually, right? But like people understand that context matters, right?

Speaker 1 That you could, hey, I create a ripoff of Mona Lisa, but you can also buy a ripoff off the Mona Lisa on Canal Street for five bucks, right? And like it, you know.

Speaker 1 So it's the, it's it's the intentionality of the act and the context of the speaker. Now I sound like super woke, I guess, right?

Speaker 1 Where you're coming from in the middle of the big experience. I think that actually is how humans communicate.

Speaker 1 Like art that might be pointless, dribble coming from somebody can be something different coming from a Jackson Bollock or whatever.

Speaker 1 Absolutely. I think that that's a really important point, by the way.
I think it's a different point, but I think that that is a very important point. I think context does matter.

Speaker 1 We'll be right back after this message.

Speaker 2 They're now the best network, according to the experts at OOCLA Speed Test, and they're using that network to launch Supermobile, the first and only business plan to combine intelligent performance, built-in security, and seamless satellite coverage.

Speaker 2 With Supermobile, your performance, security, and coverage are supercharged. With a network that adapts in real time, your business stays operating at peak capacity even in times of high demand.

Speaker 2 With built-in security on the first nationwide 5G advanced network, you keep private data private for you, your team, your clients.

Speaker 2 And with seamless coverage from the world's largest satellite-to-mobile constellation, your whole team can text and stay updated even when they're off the grid. That's your business, supercharged.

Speaker 2 Learn more at supermobile.com. Seamless coverage with compatible devices in most outdoor areas in the U.S.
where you can see the sky.

Speaker 2 Best business plan based on a combination of advanced network performance, coverage layers, and security features. Best network based on analysis by OOCLA of Speed Test Intelligence Data 1H 2025.

Speaker 1 Being a small business owner isn't just a career, it's a calling. Chase for Business knows how much heart and effort go into building something of your own.

Speaker 1 That's why we make your business growth our priority. Our team takes the time to understand your mission, where you are now, and where you want to go.

Speaker 1 Our broad range of solutions is designed with you in mind so you can bring your ideas to life.

Speaker 1 From banking to payment acceptance to credit cards, you can conveniently manage all your business finances all in one place with our digital tools.

Speaker 1 Looking for tips and advice, our online resources are always available to give you the solutions you need to help your business thrive. See how your business can go farther with Chase for Business.

Speaker 1 Learn more at chase.com/slash business. Chase for Business, make more of what's yours.
The Chase mobile app is available for select mobile devices. Message and data rates may apply.

Speaker 4 This is Sophie Cunningham from Show Me Something. Do you know the symptoms of moderate to severe obstructive sleep apnea or OSA in adults with obesity?

Speaker 4 They may be happening to you without you knowing.

Speaker 4 If anyone has ever said you snored loudly or if you spend your days fighting off excessive tiredness, irritability, and concentration issues, it may be due to OSA.

Speaker 4 OSA is a serious condition where your airway partially or completely collapses during sleep, which may cause breathing interruptions and oxygen deprivation. Learn more at don'tsleep on OSA.com.

Speaker 4 This This information is provided by Lilly, a medicine company.

Speaker 1 I was buttering up this report before. My big critique of it is like, where are the human beings in this are put in another way, kind of like, where is the politics, right?

Speaker 1 They're trying not to use any remotely controversial real names, right? So you have open brain, for example. So where is President Trump?

Speaker 1 Let me just do a quick search to make sure the name Trump does not appear to know the name of the Trump. So, they do actually, I don't know if this existed.

Speaker 1 Maybe they took your criticism in this, but they do have like the vice president and the president. Like, they do put politicians in this version of the report.

Speaker 1 They don't have names, but they say the vice president in one of these scenarios, you know, handily wins the election of 2028. We have one vice president.

Speaker 1 They have general secretary, I think. I'm not sure if they, I mean, general secretary.
General Secretary resembles she.

Speaker 1 Yep. And the vice president kind of resembles J.D.
Vance, right? I don't think the president resembles Trump at all, right? It's kind of this demos

Speaker 1 character enough like. Yeah, they tried to sidestep that as well.

Speaker 1 If we think this is all happening in the next four years, then,

Speaker 1 you know, presidential politics matter quite a bit. I mean, I don't know.

Speaker 1 This is such a fucking, you know, I was jogging earlier on the East Side and I was listening to the Ezra Klein interview with Thomas Friedman. It's such a fucking yuppie fucking thing, right?

Speaker 1 It's okay. It's okay.
You're allowed to be a yuppie. Mr.
Friedman, like

Speaker 1 not a huge fan of necessarily, but like, you know, is well versed on like geopolitics and like China issues. He's like, yeah, China, he's just been back from China.

Speaker 1 He's like, yeah, China's kind of winning. You know what I mean? And like, I'm not sure.
How

Speaker 1 Trump's hawkishness on China, but like kind of

Speaker 1 imbecilically executed

Speaker 1 hawkishness on China. Like, I'm not sure how that figures in to this, right?

Speaker 1 If we're reducing U.S.-China trade, that probably does produce an AI slowdown, maybe more for us if we're like, if they're not exporting their like raw earth materials and so forth, but we're making it harder for them to get NVIDIA chips, so they probably have like lots of workarounds and things like that.

Speaker 1 Maybe, maybe Trump tariffs are good. I'd like to ask the authors this report because it means that we're going to have slower AI progress.
I'm not joking, right?

Speaker 1 On the other hand, it increases the hostility between the U.S. and China in the long run, right?

Speaker 1 I mean, that's that if we were sending all the terrorists tomorrow, I think we still permanently, or let's not say permanently, let's say at least for a decade or so, have injured U.S.

Speaker 1 standing in the world. And so I don't know how that figures in.
And I'm like, I'm also not sure, like,

Speaker 1 kind of quote, what the rational response might be. But one thing they tracked, let me make sure that they kept this into their report, right?

Speaker 1 so they actually have um

Speaker 1 their implied approval rating for

Speaker 1 uh how people feel about open brain which is their not very yeah it's very negative for open ai i think this actually is some feedback who that they took into account right they originally had it slightly less negative, but they have this being persistently negative and then getting more negative over time.

Speaker 1 It was a little softer in the in the previous version that

Speaker 1 I saw. So they did change that one thing

Speaker 1 at some stage. But like the fact that like AI scares people, it scares people for

Speaker 1 both good and bad reasons, but I think mostly for valid reasons, right?

Speaker 1 That the fear is fairly bipartisan that the biggest AI accelerators are now these kind of Republican techno-optimists who are not looking particularly wise given how it's going with the first 90 days, whatever we are, of the Trump administration, and the likelihood of like a substantial political backlash, right?

Speaker 1 Which could lead to dumb types of regulations. But like, you know, part of it too is like, okay,

Speaker 1 AI, they're saying, can do not just computer desk jobs, but like all types of things, right? And like

Speaker 1 humans kind of play this role initially as supervisors. And then literally within a couple of years, people start to say, you know what, am I really adding much

Speaker 1 value here, right? You kind of have like these legacy jobs and there is a lot of money. I think most of these scenarios imagine

Speaker 1 very fast economic growth, although maybe very lumpy, right? For some parts of the world and not others.

Speaker 1 But we're kind of just sitting around with a lot of idle time. It might be good for live poker, Maria, right?

Speaker 1 All of a sudden, all these smart people, all their open earth or open brain, excuse me, stock is now worth billions of dollars, right? And like,

Speaker 1 nothing to do because the AI is doing all their work, right? They have a lot of fucking time to play some fucking Texas holding, right?

Speaker 1 That is one way of thinking about it.

Speaker 1 Let's go back to

Speaker 1 your earlier point, which I actually think is an important one, because obviously they were trying to do, as all, you know, super forecasting tries to do, is you try to create a report that will work in multiple scenarios, right?

Speaker 1 You can't tie it too much to like the present moment, otherwise your forecasts are going to be quite biased.

Speaker 1 However, I do think that what you raise, kind of our current situation with China, et cetera, has very real implications given that this is kind of the central dynamic of this report that their predictions are based on.

Speaker 1 I think that it's incredibly valid to actually speculate, you know, how will, if at all, this affect the timeline of the predictions, the possible, the likelihood of the two scenarios.

Speaker 1 And I will also say that one of the things in the report is that all of these negotiations on like, will we slow down? Will we not? How aligned is it, this all takes place in secret, right?

Speaker 1 Like, we don't know.

Speaker 1 We, the humans, don't know that it's going on. We don't know what's happening behind the scenes.
And we don't know what the decision makers are kind of thinking.

Speaker 1 And so, for all we know, you know, President Trump is meeting with Sam Altman and trying to

Speaker 1 kind of do some of these things. And it's funny because we were kind of pushing for transparency in one way, way, but there's a lot of things here that are very much not transparent.

Speaker 1 Yeah, it's kind of the deep state, right? But also

Speaker 1 a lot of the negotiations are now AI versus AI, right?

Speaker 1 And

Speaker 1 look, I'm not sure that AIs will have that trust both with the external actor and internally. I'm skeptical of that, right? If that does happen,

Speaker 1 They kind of think this might be good because the AIs will probably behave in like a literally game theory

Speaker 1 optimal way, right? And And understand these things and make, I guess, like fewer mistakes than humans might.

Speaker 1 If they're properly aligned, like that's a crucial thing, because in the doomsday scenario, AI negotiates with AI, but they conspire to destroy humanity, right? So there are two scenarios.

Speaker 1 One, it's actually properly aligned, so AI negotiates with AI. Game theory

Speaker 1 works out, and we end up with, you know, democracy and wonderful things. But in the other one where they're misaligned, AI negotiates with AI to create a new AI, basically, and destroy humanity.

Speaker 1 So it can go one way or the other, depending on that alignment step, first of all.

Speaker 1 I mean, the utopia didn't seem that utopian to me, right? I'm not sure that. No, it didn't.
It actually seemed quite dystopian to me. Like, it seemed incredibly dystopian.
It's kind of like,

Speaker 1 but at least we're still alive.

Speaker 1 We'll have adventures. We'll probably live longer.
And again,

Speaker 1 lots of, lots of, lots of poker. The AI will be writing Silver Bulletin

Speaker 1 and hosting our podcast, right?

Speaker 1 Let me back up a little bit because I think we maybe take for granted that some of these premises are kind of controversial, right?

Speaker 1 So they have a break point, I think, in 2026. Well, it says,

Speaker 1 why aren't certain increases substantially beyond 2026, right? So that's kind of the breakpoint. It's like 27 is this inflection point.
I think I'm using that term correctly in this context.

Speaker 1 So I'm reading this report and

Speaker 1 up to 2026. I'm like, thumbs up.
Yeah, yeah, yeah, yeah. This seems like very smart and detailed about like,

Speaker 1 you know, how the economy is reacting and how politics are reacting and the race dynamic with China. Maybe there needs to be a little bit more Trump in there.

Speaker 1 I understand why politically they didn't want to get into that mess, right?

Speaker 1 But like,

Speaker 1 so there's kind of three different things here, right? One is a notion of what's sometimes called AGI or artificial general intelligence.

Speaker 1 And if you asked a hundred different researchers, you'd get a hundred different definitions of what AGI is.

Speaker 1 But, you know, I think it is basically like being able to do a large majority of things that a human being could do competently, assuming we're limiting it to kind of like desk job type tasks, right?

Speaker 1 Anything that can be done remotely is sometimes definition that is used or through remote work, right? Because clearly AIs are inferior to humans in like

Speaker 1 sorting and folding laundry or things like that, right? That requires a certain type of intelligence, right? If you use the kind of desk job definition, then like

Speaker 1 AI AI is already pretty close to AGI, right? I use large language models all the freaking time, and they're not perfect for everything.

Speaker 1 But like, you know, in terms of like being able to do the large majority of desk work

Speaker 1 at levels ranging from competent intern to super genius, like on average, it's probably pretty close to being generally intelligent by that definition, right?

Speaker 1 If you're the one using it, I just want to like once again point that out.

Speaker 1 Because one of the things that they say in the report is that as it gets more and more involved, what we're asking AI to do, it's like the human process to evaluate whether it's accurate and whether it's making mistakes will get longer and longer.

Speaker 1 And I think they say like for every like basically one day of work, it'll take several,

Speaker 1 it's like a two to one ratio at the beginning for how long it will take humans to verify the output, right?

Speaker 1 So you think like you think you save time by having AI do this, but if you want it to actually develop correctly, then you need a team and it takes them twice as long to verify that what the AI did is actually true and actually valid and actually aligned, et cetera, et cetera.

Speaker 1 Now, you're not asking it to do things that require that amount of time, but there do need to be little caveats to

Speaker 1 how we think about their usefulness and how you are able to evaluate the output versus other scenarios. When I use AI, the things that, it's best with our things that like,

Speaker 1 save me time, right?

Speaker 1 Where I feeded a bunch of different names for different college basketball teams. We We working in our NCA model.
I'm like, take these seven different naming conventions that are all different and

Speaker 1 create a cross-reference table of these, which is kind of like a hard task, right? You need to have a little context about basketball.

Speaker 1 And it did that very well, right? That's something I could have done. It might have taken an hour or two, but you know,

Speaker 1 instead I could do it in a few minutes. And it gets faster.
It's like, oh, I've learned from this, from you before, Nate. So now I can be faster doing this type of task in the future.

Speaker 1 I was at the poker tournament down in Florida last week and like

Speaker 1 you know, I asked open research, excuse me, God, is that like

Speaker 1 see exactly, right? It all becomes

Speaker 1 deep research. Deep research,

Speaker 1 deep research. I asked deep research.
Reading too many of these 2027 reports, your brain gets to pull a bunch of stock market data for me and then I'm playing a poker hand and like I make a

Speaker 1 really thin, sexy value bet with like fourth pair. No one knows that means, right? I bet a very weak hand because I thought the other guy would call with an even weaker hand.
And I was right.

Speaker 1 And I feel like I'm such a fucking stud here, value betting fourth pair. And while AI does the work for me, and then, of course, I like bust out of the tournament an hour later.

Speaker 1 And meanwhile, you know, deep research bungles this particular task, but in general, AI has been very reliable.

Speaker 1 But the point is that, like, there's like a inflection point where like I'm asking it to do things that like are just a faster version of what I could do myself.

Speaker 1 I wouldn't, at the moment, I ask AI, I'd be like, how I want you to design a new NCA model for me with these parameters because, like, I wouldn't know how to test it.

Speaker 1 Anyway, I'm being long-winded here. So, AGI, we're going to get AGI, or at least we're going to get someone calling something AGI soon.

Speaker 1 Artificial superintelligence, where it's doing things much better than human beings, I think this report takes for granted, or not takes for granted, it has lots of documentation about its assumptions, but it's saying, okay, this trajectory has been very robust so far, and people make all types of bullshit predictions.

Speaker 1 So the fact that these guys in particular have made accurate predictions of the past is certainly worth something, I think, right? But they're like, okay, you kind of follow the scaling law.

Speaker 1 And before too much longer, you know, AI starts to be more intelligent than human beings.

Speaker 1 You can debate what intelligent means if you want, but do superhuman types of things and or do them very fast, which I think might be different, right?

Speaker 1 And AI can do things very fast is, maybe it's certainly a component of intelligence, right? But I don't take for granted that like

Speaker 1 quote unquote AI can reliably extrapolate beyond the data set.

Speaker 1 I just think that like it's not an absurd leap of logic. It may even be like the base case or close to the base case, but like it's not assumable.
from first principles, I don't think.

Speaker 1 We've all seen lots of trend charts of the, you know, if you look at a chart of Japan's GDP in the 1980s, you might have said, okay, well, Japan's going to take over the world and be and people bought this.

Speaker 1 And now this economy hasn't grown for like 40 years, basically, right?

Speaker 1 And so like, we've all seen lots of curves that go up and then it's actually an esc or whatever the fuck you call it where like it begins to bend the other way at some point.

Speaker 1 And we can't tell until later, right? The other thing is like the.

Speaker 1 ability of AI to plan and manipulate the physical world.

Speaker 1 I mean, some of these things where they're talking about like, you know, brain uploading and Dyson swarms and nanobots, like, you know, there, I would literally

Speaker 1 wager money against this happening on the time scales that they're talking

Speaker 1 about, right? They double the time scale. Okay, then I might start to give some more probability.
And look, I'm willing to be wrong about that.

Speaker 1 I guess we're all be dead anyway, 50% likelihood in this scenario. But like,

Speaker 1 this scenario, 50% you do. Like, A, you know, the physical world requires requires sensory input and lots of parts to our brain that AI is not as effective at manipulating.

Speaker 1 It also requires like being given or commandeering resources somehow. By the way, this is like a little bit of a

Speaker 1 problem for the United States. I mean, we are behind China by like

Speaker 1 quite a bit in robotics and related. things right so like i don't know what happens if like we have the brainier smarter ais but like they're very good at manufacturing a machinery.

Speaker 1 So like what if we have the brains and they have the brawn, so to speak, right? And they have maybe a more

Speaker 1 authoritarian but functional infrastructure. So I don't know what happens then, right?

Speaker 1 Like but the ability of AIs to commandeer resources to control the physical world to me seems far-fetched on these timelines in part because of politics, right?

Speaker 1 I mean the fact that it takes so long to build a new building in the US or a new subway station or a new highway,

Speaker 1 and the fact that our politics is kind of sclerotic, right? And look, I mean, I don't want to sound like

Speaker 1 too pessimistic, but if you read the book I recommended last week, Fight, I mean, we basically did not have a fully competent president for the last two years.

Speaker 1 And I would argue that we don't have one for the next four years, right? So like all these kind of things that we have to plan for, like who's doing that fucking planning?

Speaker 1 Our government's kind of dysfunctional. And, you know, maybe that that means we just lose to China, right? Maybe that means we lose to China.
At least we'll have like nice cars, I guess.

Speaker 1 We'll be back right after this.

Speaker 2 They're now the best network, according to the experts at OOCLA Speed Test, and they're using that network to launch Supermobile, the first and only business plan to combine intelligent performance, built-in security, and seamless satellite coverage.

Speaker 2 With Super Mobile, your performance, security, and coverage are supercharged. With a network that adapts in real time, your business stays operating at peak capacity even in times of high demand.

Speaker 2 With built-in security on the first nationwide 5G advanced network, you keep private data private for you, your team, your clients.

Speaker 2 And with seamless coverage from the world's largest satellite-to-mobile constellation, your whole team can text and stay updated even when they're off the grid. That's your business, Supercharged.

Speaker 2 Learn more at supermobile.com. Seamless coverage with compatible devices in most outdoor areas in the US where you can see the sky.

Speaker 2 Best business plan based on a combination of advanced network performance, coverage layers, and security features. Best network based on analysis by UCLA of SpeedTest Intelligence Data 1H 2025.

Speaker 1 Being a small business owner isn't just a career, it's a calling. Chase for Business knows how much heart and effort go into building something of your own.

Speaker 1 That's why we make your business growth our priority. Our team takes the time to understand your mission, where you are now, and where you want to go.

Speaker 1 Our broad range of solutions is designed with you in mind so you you can bring your ideas to life.

Speaker 1 From banking to payment acceptance to credit cards, you can conveniently manage all your business finances all in one place with our digital tools.

Speaker 1 Looking for tips and advice, our online resources are always available to give you the solutions you need to help your business thrive. See how your business can go farther with Chase for Business.

Speaker 1 Learn more at chase.com/slash business. Chase for Business, make more of what's yours.
The Chase mobile app is available for select mobile devices. Message and data rates may apply.

Speaker 4 This is Sophie Cunningham from Show Me Something. Do you know the symptoms of moderate to severe obstructive sleep apnea or OSA in adults with obesity?

Speaker 4 They may be happening to you without you knowing.

Speaker 4 If anyone has ever said you snored loudly or if you spend your days fighting off excessive tiredness, irritability, and concentration issues, it may be due to OSA.

Speaker 4 OSA is a serious condition where your airway partially or completely collapses during sleep, which may cause breathing interruptions and oxygen deprivation. Learn more at don'tsleeponosa.com.

Speaker 4 This information is provided by Lilly, a medicine company.

Speaker 1 I think that your point about the growth trajectories not necessarily being reliable is a very valid one. The Japanic example is great.
You know, Malthusian population growth

Speaker 1 is another big one, right? We thought that population would explode and instead we're actually seeing population decline.

Speaker 1 So, you know, the world does change. The thing that I think they rely on is that the AIs are capable of designing just this.

Speaker 1 incredible technology much more quickly so that our building process and all of that gets sped up 100 fold from what it is right now.

Speaker 1 But it still, at least at this point, needs humans to implement it, right? And needs all of these different workers. And so, yeah, I think there are some assumptions built into here that

Speaker 1 I hope that that timeline isn't feasible. And I do think that there are things that are holding us back.
All the same, I think it's really, I think it's interesting.

Speaker 1 One of the reasons I like this report is that it forces you to think about these things, right? And try to game out some of these worst-case scenarios to try to prevent them,

Speaker 1 which I think is always an important thought exercise.

Speaker 1 I do want to go back to kind of their good scenario, which just, so bad scenario is, you know, we're all wiped out by, by a chemical warfare that the AIs release on us.

Speaker 1 Good scenario is that, you know, everyone gets a universal basic income and AI does everything and no one has to do anything and we can just live

Speaker 1 happy.

Speaker 1 Play poker. Yeah.
And that just, as you suggested, that seems actually like a very dystopian scenario where people can become much more easy to brainwash, control, et cetera, et cetera.

Speaker 1 It's like a dumbing town, right? Where we're not challenged to produce good art, to advance in any sort of way. It just, to me, it does not seem like a very meaningful outlook.

Speaker 1 The question is, there's by the way another, if you read AI 2027, which I highly recommend that you read it, there's also another post by a pseudonymous poster called L.

Speaker 1 Rudolph L, who wrote something called A History of the Future, 2025 to 2040,

Speaker 1 which is very detailed, but goes through kind of like what this looks like at more of a human level, how society evolves,

Speaker 1 how the economy evolves, how work evolves, right? And like very detailed, just like AI 2027 is, but kind of focuses on the parts of AI 27 that I think it kind of deliberately ignores.

Speaker 1 Maybe you can call them mild blind spots or whatever, right? But like, but that's interesting because that kind of thinks about like what types of jobs are there in the future.

Speaker 1 There are probably lots of lawyers, actually, right? Because the law law is very sluggish to change, especially in a constitutional system where there are lots of veto points, right?

Speaker 1 Probably high-end service sector.

Speaker 1 You go to a restaurant because everyone's or a lot of people are rich now, right? And you're flattered by the attractive young server and things like that.

Speaker 1 So it's kind of highly kind of like catered and curated experiences. I guess I have

Speaker 1 some

Speaker 1 faith in humanity's ability to fight back, quote unquote, against like

Speaker 1 two scenarios that it might not really like either one. You know what I mean? And like the scenario where like AI is producing like 10%

Speaker 1 GDP growth or whatever, right? I mean, it's great if you own stocks that are exposed to AI and tech companies, probably right, but it's also making that money on the backs of mass job displacement.

Speaker 1 And like, you know, it kind of must are confident in the long run that human beings find productive things to do. And,

Speaker 1 you know, mass unemployment has been predicted many times and never really occurred, right?

Speaker 1 But like, but it's not occurring this fast where they think the world ends in six years or whatever they're predicting, or we have utopia in six years.

Speaker 1 And just the ability of like human society to deal with that change at these time scales leads to like more chaos than I think they're more predicting.

Speaker 1 But I think also, and I told them this too, right? I think also leads to more constraints, right? That you hit bottlenecks. If you have

Speaker 1 five things you have to do, right?

Speaker 1 And you have the world's fastest computer, et cetera, et cetera, but there's like a power outage in your neighborhood, right? And

Speaker 1 that's a bottleneck, right? Maybe there are ways around it. You can go to Home Depot and buy a generator or, you know what I mean?

Speaker 1 But like, but the point is that like, you're often defined by like the slowest link and politics are sometimes the slowest link, but also by like, also, you know, I think the report

Speaker 1 maybe understates, and I think kind of in general, the AI safety community, like, maybe understates the ability of like human beings to cause harm to other human beings with AI, right?

Speaker 1 That concern kind of gets brushed off as like too pedestrian or like

Speaker 1 pedestrian. It was

Speaker 1 the exact word I was thinking of. I think that's a good, I mean, I think that's a great place to end it because

Speaker 1 yes, we do need to be concerned about all of these things about AI, but like that, that phrase I think is very crucial. Like do not underestimate the ability of humans to cause harm to other humans.

Speaker 1 And I think that that's,

Speaker 1 you know, it's not a very opt, it's not a very pleasant place to end, but I think it's a really important place to end. And I think that that's a very valid kind of way of reflecting on this name.

Speaker 1 Or to trust AIs

Speaker 1 too much, right? I generally think that concern is like

Speaker 1 somewhat misplaced, but like if we're handing over critical systems to AI, right?

Speaker 1 It can cause problems if it's very smart and deceives us and doesn't like us very much. It can also cause problems if it has

Speaker 1 hallucinations, bugs, and critical areas where it isn't as robust and hasn't really been tested yet that are outside of its domain.

Speaker 1 Or there could be espionage. Anyway, we will have plenty of time, although maybe only seven more years actually, to like explore

Speaker 1 these scenarios.

Speaker 1 Yes, and in seven years, we'll be like, welcome back to the final episode of Risky Business, because the prediction is we're all going to be dead tomorrow. But yeah, this was an interesting exercise.

Speaker 1 And I think

Speaker 1 my P doom has slightly gone up as a result of reading this, but I also remain optimistic that humans can do good as well as harm.

Speaker 1 Yeah, my interest in learning Chinese has increased as a result of recent developments at

Speaker 1 about my P doom. All right, yeah,

Speaker 1 let's do some language immersion. I'm with you.

Speaker 1 That's it for today. If you're a premium subscriber, we will be answering a question about whether MFAs can ever be plus EV right after the credits.
And if you're not a subscriber, it's not too late.

Speaker 1 For $6.99 a month, the price of a mid-beer, you get access to all these conversations and all premium content across the Pushkin network.

Speaker 1 Risky Business is hosted by me, Maria Konakova. And by me, Nate Silver.
The show is a co-production of Pushkin Industries and iHeartMedia. This episode was produced by Isabel Carter.

Speaker 1 Our associate producer is Sonia Gerwit. Sally Helm is our editor, and our executive producer is Jacob Goldstein.
Mixing by Sarah Bruguer. If you like the show, please rate and review us.

Speaker 1 You know, we like, we take a four or a five. We take the five.
Rate and review us to other people. Thank you for listening.

Speaker 1 And now, Superhuman Shaq.

Speaker 3 I keep telling them not to say that. I'm no superhuman.
Believe it or not, I struggle with moderate obstructive sleep apnea, or OSA.

Speaker 3 In adults with obesity, moderate to severe OSA is a condition where breathing is interrupted during sleep with loud snoring, choking, gasping for air, and even daytime fatigue.

Speaker 3 Let's just say it can sound a lot like this.

Speaker 3 Sound familiar? Learn more at don't sleep on OSA.com.

Speaker 1 This information is provided by Lilly, a medicine company. Honestly, honestly, honestly, no one wants to think about HIV, but there are things that everyone can do to help prevent it.

Speaker 1 Things like PrEP. PrEP stands for Pre-Exposure Prophylaxis, and it means routinely taking prescription medicine before you're exposed to HIV to help reduce your chances of getting it.

Speaker 1 PrEP can be about 99% effective when taken as prescribed. It doesn't protect against other STIs, though, so be sure to use condoms and other healthy sex practices.

Speaker 1 Ask a healthcare provider about all your prevention options and visit findoutaboutprep.com to learn more. Sponsored by Gilead.
Okay.

Speaker 1 Only 10 more presents to wrap. You're almost at the finish line.
But first.

Speaker 1 There,

Speaker 1 the last one.

Speaker 1 Enjoy a Coca-Cola for a pause that

Speaker 1 refreshes.

Speaker 1 This is an iHeart Podcast.