Why ChatGPT Sucks at Poker

Pushkin.

This is an iHeart podcast.

In today's super competitive business environment, the edge goes to those who push harder, move faster, and level up every tool in their arsenal.

T-Mobile knows all about that.

They're now the best network, according to the experts at OoCla Speed Test, and they're using that network to launch Supermobile, the first and only business plan to combine intelligent performance, built-in security, and seamless satellite coverage.

That's your business, Supercharged.

Learn more at supermobile.com.

Seamless coverage with compatible devices in most outdoor areas in the U.S.

where you can see the sky.

Best network based on analysis by OOCLA of SpeedTest Intelligence Data 1H 2025.

You've probably heard me say this.

Connection is one of the biggest keys to happiness.

And one of my favorite ways to build that, scruffy hospitality, inviting people over even when things aren't perfect.

Because just being together, laughing, chatting, cooking, makes you feel good.

That's why I love Bosch.

Bosch fridges with VitaFresh technology keep ingredients fresher longer, so you're always ready to whip up a meal and share a special moment.

Fresh foods show you care, and it shows the people you love that they matter.

Learn more, visit Bosch HomeUS.com.

LinkedIn makes it easy to post your job for free, share it with your network, and get qualified candidates that you can manage all in one place.

Here's how it works.

First, post your job.

LinkedIn's new feature can help you write job descriptions and then quickly get your job in front of the right people with deep candidate insights.

Second, either post your job for free or pay to promote it.

Promoted jobs get three times more qualified applicants.

Then, get qualified candidates.

At the end of the day, the most important thing to your small business is the quality of the candidates you attract.

And with LinkedIn, you can feel confident that you're getting the best.

Then, data.

Based on LinkedIn data, 72% of SMBs using LinkedIn say that LinkedIn helps them find high-quality candidates.

And last, share with your network.

You can let your network know you're hiring.

You can even add a hashtag hiring frame to your profile picture and get two times more qualified candidates.

Find out why more than 2.5 million small businesses use LinkedIn for hiring today.

Find your next great hire on LinkedIn.

Post your job for free at linkedin.com/slash gladwell-fake.

That's linkedin.com/slash gladwell-fake to post your job for free.

Terms and conditions apply.

Welcome back to Risky Business, a show about making better decisions.

I'm Maria Konakova.

And I'm Nate Silver.

Today on the show, we're going to be talking about how ChatGPT plays poker, i.e.

poorly.

If ChatGPT is getting ready for the World Series of poker like us, then it's got a lot of homework to do.

Yes, and

after we talk about that, we'll talk about Harvard 2.0, aka

the fact that foreign students are now being potentially barred from attending Harvard University.

Harvard has sued the administration.

Anyway, we'll be talking about what's going on with that and what the implications are for the future of the United States and kind of research, development, brainpower, and the U.S.

competitive edge.

So for people who don't subscribe to Silver Bulletin, which you absolutely should, Nate and I both had poker posts this last week.

Mine was about cheating, His was about chat GPT.

But his post this week was one of the funniest things I've read in a while about his attempts to get ChatGPT to simulate a hand of cash game poker.

And one of the reasons, I mean, I found this amusing because, I mean, first of all, it fused cards together for an image.

From the beginning to the end, it was pretty spot on in terms of being spot on wrong.

But Nate, you're also someone who is constantly writing about how good AI is at so many things.

And so it was funny to me to see it fall

so short of something that actually tests intelligence as opposed to being able to kind of pull together a lot of things and spit out an answer.

There are some reasons why I'm interested in testing ChatGPT on poker.

But they kind of fall into two big buckets, right?

One is it like poker played in the context of a real hand, or in this case, a real text-based simulation, I guess, of a hand.

It really does require quite a few skills, right?

There is the pure math part of it.

What is the equilibrium, the Nash equilibrium, the GTO strategy that you're solving for?

There's also making adjustments for how other players play.

There's the conversation you're having, the physical reads that you're getting.

There's stuff like knowing what the rules are, right?

right which might seem trivial but we've yeah

jpt also does not know it seems to me no it is fun look i mean i'm sure we've all had hands where we like misread a board or i you know i had a big hand in the main event at the when did i play back in florida back in april or whatever right where like a guy had like a 25 000 chip that was like almost the same color as the felt which is not good chip kit maintenance by the way seminal hard rock i i like the wpt but that chip kit it's got to be improved right?

But yeah, you're tired.

You make mistakes like that.

But, and you also have to have like a lot of short-term memory, which sounds trivial, but you want to remember how you arrived at this current spot in the hand, right?

You have to keep track of all these stack sizes, which again seems trivial, but like, it's a good test of general.

intelligence of a certain type, especially a live poker hand like I'm asking ChatGPT to simulate, right?

The other reason is that like, I don't think engineers in Silicon Valley are trying to optimize it for poker.

And the reason it's important is because like, look, there are benchmarks, like math Olympiad problems that these LLMs, large language models like ChatGPT, Claude, et cetera, compete on, right?

And they're like, well, we bragging that we can now, you know, beat all but the Nobel Prize mathematicians on X percentage of math problems and things like that.

And there are a couple of issues with that, right?

Like one is that like clearly if you train a transformer model, a machine learning model on a particular type of problem, like it can do fairly well or very well often, right?

Like poker is to a first approximation and it's an important approximation, right?

But if you have poker dedicated tools, I wouldn't consider a solver an AI.

That's a technical distinction that I think might not be that important for our listeners.

But like, you know, if you want to use computers to play very good poker, then they can play very, very good poker, right?

The question is, like, can you take a text-based model and have this organic property where intelligence emerges and converts from the data set towards super intelligence without training it on poker specifically?

And the answer, at least as of last week when I did this, is no, it fails miserably.

Yeah, and I think it's actually important.

One of the things that came to my mind when I was reading your piece was that poker has been a benchmark for AI development well before LLMs, right?

It has been kind of the gold standard that teams all over the world have been working on for decades because it is a much more complex game in many senses than chess, than even Go.

And it's a game where, you know, if computers can actually manage to outthink human players on a broader scale, that would mean something in terms of kind of broader intelligence capabilities.

And before LLMs were developed, that had not happened, right?

There had been computer programs, kind of AI algorithms that have been able to beat heads up one-on-one opponents in poker.

But when it came to full ring games, so for people who don't know poker, that means basically when you put it in a rich environment with, you know, six players, eight players, and you have a computer there it was still not quite there and one of the you know one of the reasons why poker is so interesting is it's not just math right especially when you have multiple players there's so many dynamics and there's actually a very thorny problems for AI which is that if you program something to follow an algorithm, right, to be GTO, game theory optimal, then it can be exploited by a human if the human figures out the the GTO strategy.

Now, if it were another human, then you'd adjust immediately, right?

The moment the human adjusts, the other human would adjust and kind of figure it out.

But if you're an AI, if you're a computer, and you build in that adjustment parameter, what ends up happening is that the model adjusts too much and too quickly.

And so that lacks kind of some of the nuance and flexibility that marks, that's kind of the hallmark of the great poker players, right?

Kind of that, that ability to intuit when it's time to slightly deviate.

And I mean, it would be terrifying, but amazing if a model could spontaneously, like an LLM could actually spontaneously figure that out.

And so far, I mean,

instead of doing that, Nate, you know, your model decided that the person who won the hand actually lost the hand, right?

It just got basic things completely wrong.

And you just, you know, obviously we've talked about Doom a lot on the show.

I think you and I are both worried about that.

But when I see this poker stuff, I'm like, oh man.

Yeah.

And I don't know how much to update.

But look, my, as I say in the post, I mean, like, my

appreciation and expectations for large language models have increased a lot over the past year.

I had a

pretty good experience working with them when I was building the NCA tournament model that we host Silver Bulletin and just helpful kind of Swiss army knives or like annoying data related tasks.

It's late at night.

You forget the stata command for something.

You're like, write, give me the right command and write five lines of code.

And like usually it works.

And it kind of now shows you like the chain of reasoning and like what it's thinking.

Right.

And, but, you know, people are saying that we're going to have like AI revolutionizing

all programming.

Or, you know, first of all, the big claim is we're going to have AGI meaning that a machine it's a little ambiguous when people say does this mean like chat GPT itself is AGI right well clearly not it's very far from human beings in some things right does that mean that a combination of transformer or machine learning based technologies, as we're zooming out the definition of AI, it's clearly AI still, right?

So that you have chat GPT says 70% of things well, and then you use specialized applications for 20%, and then there are 10% where you're not doing it very well.

Like that would still be a big achievement, very disruptive achievement.

I'm not sure it would kind of quite count as AGI.

Like one thing you could do is like some of these new models will say, okay,

I can detect that you asked for something that requires mathematical precision.

So let's take a simpler example, right?

Let's say I want to take the distance between pick two cities, Boise, Idaho, and Tampa, Florida, right?

That's pretty easy to calculate mathematically if you look up the lat-long coordinates for Boise and Tampa, and then there are various formulas that you can use, right?

So, a pure data set where it's just crunching text might not find that answer because there's not necessarily a lot of examples of like what's the distance between Boise and Tampa, right?

Whereas you would have that for like LA and New York or something.

However, you can set up kind of agentic, meaning agent AIs, where they're like, okay, well, this person asked a question that triggered a routine of mine where now I'm going to go and I know to search the internet for the coordinates, or maybe it's stored somewhere, right, of Boise and Tampa, and then the formula, I know the formula, and I can apply it, right?

So, like, you have the scaffolding, it's sometimes called, of different processes on top of one another.

And I guess that kind of, I guess that counts as, I mean, it's kind of how humans solve problems, right?

They're like, oh, and now I have to go look something up, right?

It's still, I think, subtly different.

So, one implication I think this has is artificial general intelligence versus super intelligence, ASI, as it's sometimes called, where there's like an explosion of intelligence just from kind of mining text or other data sources.

I have actually become a little bit more skeptical of that, or I think it's presumptive to assume it can find that.

I mean, like, let me give you one other example, right?

Like, one, I had

an idea for either a silver bulletin post or maybe even a risky business segment, right?

We're like, I asked ChatGPT, deep research, design a meal

with foods that have no clear precedent in other countries, right?

Huh.

And, and I thought, and the idea was that like, you know, at some point somebody invented pizza or spaghetti or whatever else, right?

Or, or something like that.

I mean, at some point, somebody invented bread and figured out that you can take flour and like bake shit with it it's amazing yes and essentially you have to be able to buy these ingredients at a trader joe's or a whole foods right um i'm just gonna like hire a professional chef and have a dinner party and and ask how these creations went right and it's like and all the recipes were like here's salmon with miso paste right really creative and things like that right and like it didn't have any ability to like extrapolate beyond the data set.

I'm sure these preparations are fine.

It was like they're a little chefy, meaning like things that like have one too many ingredients, right?

And like, and I don't, yeah, look.

So bad cheffy.

Yeah.

And meanwhile, I've been like, I've been interviewing candidates for like a sports assistant position at Silver Bulletin.

And one of the questions is, how are you using AI?

These are very bright, I'm going to say kids.

A lot of them are

recent college grads, you know.

early 20s to mid-30s, basically, right?

And very technically proficient people.

And asked them how they're using AGI, and their experience is like kind of similar to mine.

They're like, yeah, for certain tasks, it's very helpful.

For certain tasks, it's somewhat helpful.

For certain tasks, it's like not helpful at all.

I can get you in trouble, right?

Which again, contrasts with the experience of people at the AI labs who say, yeah, it's going to like displace all programming within a year or two.

I mean, I don't know.

You should weigh in here.

So I have an interesting kind of anecdote to add to this that's not poker related, but that talks about kind of some of this nuance and complexity and

the fact that, you know, that all of these LLMs still are wanting in certain major respects.

So I had the chance last week to speak to the CEO of one of the major AI companies.

I won't name which company it was,

but one of the big ones.

And

someone had asked him earlier, what was something that was surprising about the way that this AI functioned?

And he said, you know, one of the mistakes

that people make is kind of they trust it.

And what we know is that it's remarkably accurate 90% of the time, right?

And then 10% of the time, it isn't.

But the 10%

is getting harder and harder for humans to spot because it doesn't make errors in a way that's intuitive for people, right?

It makes errors in different ways.

It's like computer errors, AI errors.

That's not the way that the human mind makes errors.

He's like, but like, it's fine because we have really smart people who can figure out where it's making errors and can work with it and can kind of calibrate it and help.

And so my question, my follow-up question was, okay,

you know, what happens?

when the new generation that's being brought up with this AI comes up and they've been educated with it and they've been using it the whole time.

And they didn't necessarily get the same education that we got, right?

Because they aren't, the incentives are different and they're not getting the deep knowledge.

They're not actually able to figure out what are the programming errors because this person was talking that, you know, some of the biggest potential of AI is in like programming and biology and kind of those types of things.

And I said, well, what if that person doesn't have the neuroscience and the biology background or the programming background?

And what the CEO said was, this is a major problem.

And what we're hoping is that we can develop even more advanced AI to help and to help fix the problems that AI is causing because he was like, yeah, this is a big problem.

And we're not quite sure.

Like we're hoping for the best, but we don't know.

And the fact that this was our conversation didn't exactly inspire me to heights of thinking this is going to replace intelligence or become AGI or what did you call nate the other ASI?

Artificial superintelligence.

Yeah, artificial super intelligence.

So it didn't inspire me that that was going to happen.

And instead, it was like an uh-oh moment where like, what happens when the humans who are kind of helping push it along and make it better when...

that generation like when they age out and there are new people out there who who lack that sort of skill and expertise and who grew up with AI that is 90% accurate or even 95% accurate, but they can't, it's harder and harder to spot that 5% and 10%.

And that to me was actually like, it's a worrisome thing, and it's clearly worrisome to the people who are developing these technologies

as well.

And so

that I think is something that dovetails with what you were talking about here.

What happens, Nate, if all of a sudden all poker players are training with ChatGPT, right?

And

that's actually great for us, but not great for the future of poker.

And we'll be right back after this break.

I use large language models for tasks that I have strong domain knowledge over and could do myself, but they're a labor-saving device.

Often they save substantial labor, right?

Like

I use them to copy edit articles I post at the newsletter.

And I, you know, I'm a fairly proficient user of the English language and like for programming, right?

Like, you know, if you're like using AI to like design a website and some of the functionality breaks, right?

That might be okay.

You can patch it and fix it, right?

If I'm doing things involving modeling, I'm trying to.

project the value of basketball teams or football teams or basketball players or football players, right?

And there's a bug where all of a sudden, like one of the worst players in the league is rated as being very good, right?

I mean, and you see this in poker where like, um,

you know, AI is relying, you get punished in poker for sloppy application of imprecise application of heuristics.

Like one of the things like the AI did, so at first I had it simulate just one hand and it had like a backstory for each player.

It fucked that up, right?

Then like, okay, I'm gonna use deep research now and have it simulate like an orbit of eight hands.

And that was a little better.

It took more compute time, right?

Some of the hands were decent, but it, you know, it didn't know how to like read a board correctly and keep track of stacks.

But when it was

able to overcome those errors,

it also had poor strategy.

And it gave various excuses.

One thing it said is like, well, the quality of text-based content on poker on the internet really sucks, right?

It's aware of that.

It says it's worse than Go or chess.

Although when I've talked to people who know how AIs play chess, it's also a disaster.

Even something like Wordle, I heard from a user, you know, you think Wordle is a word game

and it does very well, but like it can't quite figure out the structure of the problem, right?

It kind of is learning a little bit more by a row.

And to be clear, like this is a very good strategy for like many types of problems.

And also, if you were to say, okay, here's a bunch of we bought data from PioSolver or GTO Wizard.

These are solvers, if you don't know, listeners,

or a database of high-stakes hands.

Like if you you traded on that and then had GPT say, okay, poker question, I'm going to call them a specially trained database, then it would do well.

And maybe if people are criticizing

on poker, then the labs will start to do that, right?

But it's still kind of like

slightly cheating from the standpoint of super intelligence.

Also, it says like, when you ask me to do a whole bunch of things at once, right?

So I told it, give me characters, give me eight characters who have like, they were all kind of like ethnic stereotypes, right?

It was.

They were hilarious.

They were very funny.

It was kind of not great way.

Yeah, in a not great way.

It was kind of at the one hand.

So half the players were women.

That was very feminist of it, right?

But then they're all ethnic stereotypes of different kinds, right?

And so it was kind of both woke and

unwoke, right?

And the opposite, yeah.

It was kind of good about like matching players' playing styles with

their actions, but like, but it couldn't keep track of stack size.

It's like, yeah, to keep track of stacks, I have to store a bunch of of stuff in memory.

And then when you ask me to do a whole bunch of things at once, right?

And like, you know, and even including like,

so in the hand I show on Silver Bulletin, it like, it misreads the board.

It doesn't recognize that a pair of nines has a higher two pair.

If you ask it that outright, then it thinks about it more and gets the question right.

Right.

But like it loses track of things in the context of these scaffolding situations where it has to keep track of a bunch of things at once.

And it doesn't know, it doesn't know what to prioritize, right?

Like, you know, you can make a lot of mistakes in poker.

If you don't know which hand beats which hand, that's a more elementary mistake than anything else.

Absolutely.

So there are a few things that stand out to me here.

One, I mean, it's a computer, right?

Like

we humans have working memory capacity problems.

It should be able to keep all these things.

Like, this is what it's good at.

But the problem that you're kind of, that you're illustrating is it doesn't know how to prioritize, right?

and that that is actually that is a major problem and also when we keep saying thinking but one of the first things so when I was just learning poker one of the first lessons I had with Phil Galfon who's one of the people who kind of I worked with and who taught me a lot

I remember very early on he said you know I can give you a bunch of charts and a bunch of outputs and you can memorize them and you'll be a very decent player very quickly.

Like, I know I can give you this shit.

You can memorize it.

You can spit it back at me.

You'll be fine.

He's like, but I don't want you to do that because that's going to make you a fine poker player and you'll do fine in the short run, but you'll never be a great poker player because you have no idea why you're doing it.

You're not actually thinking.

You don't understand.

You've just done a rote memorization, which is might make you more money in the short term, but in the long term is actually going to be detrimental to you because you're going to do stuff unthinkingly because you've memorized it.

He said, What I want you to do is instead think think through things and figure out, okay,

why, right, for every single play, why am I doing this?

Why would I play this hand and not this hand?

Why would I raise this type of hand and not this type of hand?

Why, why, why, why, why?

And that's something that to this day has stayed with me because it's such an important thing to remember when you're making a decision, right?

And this doesn't even have to be poker, it can be any sort of decision.

Why am I choosing this action?

Why is it better than all of the other actions, right?

And if you, even if you feed, you know, PIO solver and GTO wizard outputs to LLMs, they'll be doing some rote kind of memorization.

You know, at this point, they're not understanding the why.

And so

That will lead them still to make big mistakes, which happens when you just learn solver outputs, no matter how sophisticated those outputs might be and how correct they might be in one specific hand, in one specific spot.

If you don't understand the why, you're either going to overgeneralize it, right?

Or misapply it, like you're going to screw it up because you don't understand the underlying reasoning.

And there's the difference between looking like you're thinking and actually thinking.

And yeah, you know, as a human, you have other pitfalls.

And even if you're, you know, thinking through things, you can mess up.

So you mess up in different ways.

But I think that's a really important distinction.

And probably one of the reasons why AGI is not as close.

Like poker actually illustrates in a very practical way why the notion that you can just have a lot of data and all of a sudden have this, you know, flurry of insight might not be as,

I guess, as intuitive as one might think.

One thing ChatGPT told me when I asked it to like audit itself is like, well, poker is very difficult because it's adversarial.

You know, it understands that there's some game theory there, but it's adversarial and like there's no one.

I mean, there is like, I guess, some superstructure of like a solution for all poker hands if you get very zoomed out, right?

But like, but like subtle things when you have to calculate like an adversarial

equilibrium on the fly, basically.

And

it's

very difficult.

Yeah.

I mean, think about not even poker, but think about trying to do a game theory solution for multiple players, right?

It's hard enough trying to do a payoff matrix for two players when you're really trying to think through it and you're trying to think of all of the different payoffs and figuring out, you know, how to exactly do you weight them?

How exactly do you structure that?

Now, when it's three players, four players, I mean, it's incredibly difficult.

to do that accurately.

And so even if we just zoom out from poker, like this is just an incredible, a very tough problem,

even if you understand game theory.

Plus, I mean, I think that anyone who has used solvers and anyone who's talked about this understands anyone who doesn't even play poker, but has worked with algorithms, the common saying garbage in, garbage out, is absolutely accurate, right?

The solver works based on what ranges of hands you

as a human put in there, right?

And what responses you allow or don't allow, right?

And yes, it will come up with an equilibrium strategy.

But if you were wrong, right?

If there's a player who's playing a totally different range, if there's a player who's playing a totally different strategy, all of a sudden your solution means nothing.

And as a human, you can figure that out and you can kind of make adjustments from baseline, right?

If you understand what the baseline strategy is, you can adjust.

As an LLM, as an AI who is learning that, but doesn't kind of have that nuance and experience, at least at this point, I don't pretend to know what AI is going to be capable of in, you know, in five years and 10 years, but at least at this point,

they're not capable of

making those sorts of nuanced extrapolations and

figuring out, okay, were my inputs accurate, right?

Or were my inputs not quite accurate?

Yeah, look, one thing human beings are good at is that human beings are

relatively good estimators.

And Chat GPT can be sometimes.

If it's like, okay, take all this text on the web and kind of give me the average of that.

Like there are some applications where it's pretty good.

But like, you know, the other day I was getting

a drink with my partner and

a guy who looked like he was homeless comes in and like hands like the host slash bartender like

a note saying call 911 or something like that.

Right.

And you kind of have to make this judgment call about like, is this like a paranoid

schizophrenic or is someone actually in danger?

And like, I think it was pretty clear.

I mean, they're probably, you know, two two bartenders and eight customers.

Pretty clear to everybody, like, this guy, I think, is not like acutely in danger or anything.

But I'd never quite experienced a situation like that before.

And like, the fact that we can use like our general intelligence about like human behavior.

Yeah.

Like, I think everyone made the right decision that we're not going to call the police either to rat on him or to say he was in danger.

And like, you know, and that kind of thing is, is, is hard for

language models to do, you know.

No, I mean, I think in general, like this just this just illustrates a really good point which is that humans even not the smartest humans are much smarter and much more capable in very basic ways that we take for granted um than than a lot of kind of super intelligences right like it took forever for a robot to be able to pick up an egg, right?

It's something that we don't even think about.

But this was like a robotics problem that was absolutely unsolvable.

Like, how do you get a robot to pick up an egg without, without breaking it?

And we do it without just, without thinking about it.

And we make judgments all the time, just silly judgments that we don't think twice about, that are so easy for the human mind, but that are, that we don't even realize we're making, right?

Things about safety, things about just perceptions of the world.

So I think that, you know,

humans are much smarter than than we think in like in dumb ways, if that makes sense.

Like in ways that seem like they're not hard problems, but those are actually some of the hardest problems to solve.

I just think of there being four levels, right?

Like one, where AI performs better than any human.

Two, where AI performs better than like

all but expert level humans.

Three, where AI gives kind of a passable substitute performance, but like, you know, not quite professional, high-grade level.

And four, where it's just inept, right?

To the point of being comically inept.

And like, you know, my heuristic is that like within a few years, we might have roughly an even divide between those four.

And I'm counting, by the way, tasks that involve manipulating the physical environment, which I think AI will mostly be pretty bad at, right?

I don't know how much I should extrapolate from the poker example.

It was just so incongruent with these predictions of like imminent AGI,

right?

Where a year ago I would have said, okay,

yeah, of course it's sucking at poker, right?

It's not really trained on this and it's a hard problem and ha ha ha, right?

But like, if you want to have these complicated, like structured, nested tasks, like I'm much less worried now in periods of, oh, let's say five years

of

AI, like being able to build like a statistical model to like forecast elections or the NFL or whatever else, because that like just requires like a superstructure of lots of little tasks, all of which are fuckuppable.

And if you get the superstructure wrong too, then you're kind of just drawing dead, to use a poker term, right?

If you have to chain together 20 steps and at any step, there's a 90% chance you get it right.

Well, 0.9 to the 20th power means you're almost for sure.

You're going to fuck something up.

Yeah, fuck-uppable is a really great word to describe this.

Nate, to kind of sum this up, you did ask ChatGPT why it was bad at poker.

What did it tell you?

Yeah, no, I mean, it said various things.

It said, look, you stressed me out by

having me have to make up these players and things like that.

You stressed it out, Nate.

You stressed it out.

It said the data that it trained on on the internet is bad, which seems realistic.

Yeah, it's a data problem.

Yeah, it's not me.

It's you.

It said, it gave you some complicated information about how

it doesn't realize how important stack sizes are because it's just a bunch of tokens.

Like all the input you give it is generated into tokens, right?

And like the word the is not very important.

The fact that Maria has $52,000 in chips is very important, right?

And it doesn't know how to distinguish those from its transformer architecture.

So like, look, it is good at like, it is good at verbal reasoning.

Like, again, I, I, you know, we are talking about, we've done other segments where we're like amazed by AI progress.

Like it's very good at verbal reasoning.

or at least faking that, but in ways, if it's faking, it's doing a pretty good job, right?

Like it's very good at verbal stuff for the most part, right?

And AIs trained on poker are very good at mimicking software solutions, right?

And like, and so like, and machine learning can get you a lot of ways when you have good data, but like when you don't have the data in the training set, then it's not seeming to extrapolate very well.

And then these complex tasks that require more and more compute.

Like I have not been impressed by deep research, which is where it goes away and says, I'm going to perform several tasks for you that require deeper thought, right?

And then it takes 15 minutes.

You come back and you're like, oh, you fucked this up, right?

And like, anyway, so I think it has affected my priors a little bit.

Yeah, that's that's really interesting.

And I love, by the way, Nate, that, you know, even though that this is an AI, which is supposed to kind of be better than humans in so many ways, it used such human excuses for why it fucked up.

You know, you stressed me out.

You gave me that data.

It's not my fault.

This is hard.

And on that note,

let's take a break and talk a little bit about Harvard and the other type of intelligence, human intelligence.

In today's super competitive business environment, the edge goes to those who push harder, move faster, and level up every tool in their arsenal.

T-Mobile knows all about that.

They're now the best network, according to the experts at OOCLA Speed Test, and they're using that network to launch supermobile the first and only business plan to combine intelligent performance built-in security and seamless satellite coverage with supermobile your performance security and coverage are supercharged with a network that adapts in real time your business stays operating at peak capacity even in times of high demand With built-in security on the first nationwide 5G advanced network, you keep private data private for you, your team, your clients.

And with seamless coverage from the world's largest satellite-to-mobile constellation, your whole team can text and stay updated even when they're off the grid.

That's your business, supercharged.

Learn more at supermobile.com.

Seamless coverage with compatible devices in most outdoor areas in the U.S.

where you can see the sky.

Best network based on analysis by OOCHLA of Speed Test Intelligence Data 1H 2025.

Life's messy.

We're talking spills, stains, pets, and kids.

But with Anibay, you never have to stress about messes again.

At washablesofas.com, discover Anibay Sofas, the only fully machine-washable sofas inside and out, starting at just $699.

Made with liquid and stain-resistant fabrics.

That means fewer stains and more peace of mind.

Designed for real life, our sofas feature changeable fabric covers, allowing you to refresh your style anytime.

Need flexibility?

Our modular design lets you rearrange your sofa effortlessly.

Perfect for cozy apartments or spacious homes.

Plus, they're earth-friendly and built to last.

That's why over 200,000 happy customers have made the switch.

Upgrade your space today.

Visit washable sofas.com now and bring home a sofa made for life.

That's washable sofas.com.

Offers are subject to change and certain restrictions may apply.

Top reasons: Your career wants you to move to Ohio.

So many amazing growth opportunities, high-paying jobs in technology, advanced manufacturing, engineering, life sciences, and more.

You'll soar to new heights, just like the Wright brothers, John Glenn, even Neil Armstrong.

Their careers all took off in Ohio, and yours can too.

A job that can take you further and a place you can't wait to come home to.

Have it all in the heart of it all.

Launch your search at callohiohome.com.

Nate, we've talked on the show a few times now about kind of the crusade that Donald Trump seems to have against higher education, specifically the Ivy League, and even more specifically Harvard University.

Last time we spoke was when

basically the administration had threatened to pull funding if Harvard didn't comply with a certain set of demands.

Harvard said, fuck you, and sued, and

yeah, and tried to go that route, and the funding was frozen.

And then the administration last week, so we're taping this on Wednesday, the 28th of May.

So last week, the administration decided that it wanted to punish Harvard even more because, you know, they didn't like the fact that Harvard wasn't complying, that it was being defiant, that it was taking them to court.

And so said, hey, Harvard, you can no longer have foreign students.

Anyone who has visas effective immediately, those are going to be revoked.

So Harvard's class of 2025 that's graduating next week is going to be your last basically foreign student class.

And we're not going to be granting visas anymore to those students.

Harvard has sued again.

So now we have a second set of lawsuits.

The initial court cases have frozen that.

But then after that happened, in the last few days, the administration said, Marco Rubio actually said,

that we are going to freeze interviews for all foreign student visas, not just Harvard.

So they not only went after Harvard, but doubled down and said, you know what, you can sue us and you can freeze this.

But if we don't grant the interviews, last laugh is with us.

Yeah, were they likely to win the lawsuit or are they likely to?

Yeah, I think so.

I think that Harvard is likely to win the lawsuit.

Yes.

However,

I'm not a lawyer, from the legal analysis that I have seen, they are likely to win the lawsuit.

But Nate, let's remember that that requires, first of all, a lot of times, right?

Because how many times can we appeal this?

And ultimately, well, if it goes to the Supreme Court, it ends up being these days less of a legal issue and more of a political issue.

Look, I don't think the Supreme Court is likely to be very sympathetic to Trump on this type of issue, in part because like,

you know, clearly they are somewhat transparent about like they are not doing it for like some security concern and they're doing it because they don't like some of the speech that Harvard is making, but it's almost certainly First Amendment protection.

Yeah, exactly.

And I actually think that the, so I don't know if you read Steven Pinker's op-ed in the New York Times.

I don't remember what the op-ed was called, but he termed this Harvard derangement syndrome, right?

Kind of playing off of Trump derangement syndrome.

And full disclosure, Steve Pinker was my undergrad advisor.

So someone I know well, someone I'm, you know, I'm still close with.

So I'm very sympathetic, but I thought it was an excellent way of framing what's going on.

And he is someone who has attacked Harvard.

He's a tenured professor at Harvard, but he's attacked Harvard many, many times for not standing up for free speech, for not protecting conservative viewpoints, for being a little bit too woke, right?

He has been one of the main people who said, hey, like we have an issue here, but this his op-ed was this ain't the way, right?

This is not the way to solve this problem.

And you're actually attacking the students who are some of my most critical thinking, least woke people who are on campus.

But yeah, look, I mean, I kind of want to zoom out and say, like,

what is the equilibrium here?

Like, what is the Trump administration trying to achieve?

And what is Harvard trying to achieve?

Like, I support, maybe some listeners will get pissed off, right?

You know, I think clearly some of these colleges are

not complying with like the spirit and maybe the letter of the Supreme Court's affirmative action decisions, right?

And so like I wouldn't mind if like the Trump administration's like aggressively enforcing the law on those or things like diversity statements, which are enforced political speech, although Harvard's a private organization, so it's a little bit more complicated, right?

But like I, you know, I don't, it's such a blunt tool to like

target foreign students.

And even if you hate Harvard, you know, as a patriotic American, I want the best and brightest from all around the world to come here and contribute to our economy and pay our taxes and everything else, right?

And like, I mean, it's just like purely destroying the value of these very bright students from helping our economy.

Yes, it's destroying our intellectual capital.

So by the way, when we're taping this right now, I'm actually in Hong Kong and I just found out today that several universities in Hong Kong have offered blanket admissions to all students who can show that they were admitted to Harvard and can no longer go because they're a foreign student.

They said, you don't even have to apply.

You can just come here.

And these are some of the best universities in Hong Kong.

And that's super smart, right?

Like universities all over the world should be doing stuff like that right now because what the administration is doing is kind of bankrupting the future of kind of intellectual development in the country by saying, oh, foreign students, you know, we've, we've just frozen all visa applications so to me like that's super smart right like that is the we've we talked about this several months ago nate right what can china do to capitalize on the moment it's doing it it's being like come here like you don't even have to apply you don't have to do anything just show us the letter that showed you were accepted and welcome come yeah i went to london school of economics for my junior year and if i were them i'd be doing cartwheels and they always got a lot of foreign students right absolutely i mean i think everyone should be doing this right now.

And I just do not see, I mean, a lot of these things at this point, it just derangement syndrome seems right because there does not seem to be any strategic value to this.

There's not any kind of any value in terms of trying to get at the types of problems that the Trump administration says it wants to get at because it's not even addressing that, right?

Like at the originally, anti-Semitism was the pretext for this.

And this is just like so far divorced from it.

Everyone is having their visas revoked or not granted or their interviews not even granted.

So it's just, you know, the pretense is now crumbling.

We always knew it was just a pretense.

But at this point, it's just not accomplishing anything other than to further kind of create this absolute.

chasm in higher education and future brainpower, future research ability, future creativity of the United States, basically mortgaging its future, right?

Saying that we were going to give up one of the biggest, if not the biggest edge that the United States has had historically,

which is kind of this brain power, creativity

place where people can

develop and find support for

their ideas.

And now

that is

not possible.

And if this actually stands, right, if foreign students are not allowed into U.S.

universities, because right now, like I said, this interview process being paused for everyone, not just Harvard.

I mean, that is going to be just detrimental in so many respects to what happens here.

Yeah,

so Harvard is not a very sympathetic case for reasons that are partly their fault, right?

But like, you know, so far the Trump admin has been pretty smart about kind of which schools they pick on.

I mean, I don't know what their end game is here or what Harvard's is, really.

You know, J.D.

Vance had a tweet

where he was like, he's oddly transparent sometimes about his thinking.

And he's like, well, look, just like, look, these colleges aren't serious about complying with the law, citing like the students for fair admissions, I think it's called, which just was the affirmative action decision, which I do think they've kind of not been serious about complying with in some cases, case-by-case basis.

And so like, unless we kind of show real muscle, then what can we do?

We have to fight them because...

they're in the wrong and so we're overreaching it is a subtext of that, right?

Which is like

maybe not crazy and they have more leader because harvard isn't that sympathetic people got perceptions of higher education that's gone way down so it's like harder to marshal public opinion on the side of this right and like and like

you know i saw some harvard professors on twitter over the weekend be like this is the greatest threat to american i'm like please don't say that because it's it's

It's maybe more of a threat to American competitiveness.

I'm not sure it's a threat to democracy per se, right?

No, I think there are bigger threats to democracy.

Right.

But like, but what's Harvard's end game?

Like, I think, like, if they could go back

and tone down some of the things they've done for the past 10 years, and I think they maybe would, but like, but now, I mean, if they back down, then they look weak and that also might hurt their recruitment.

Oh, not that they'd have real problems.

But like, what, if you're the president of Harvard, what's the name, Alan Garber, is that right?

Yep.

If you're Alan Garber, Maria, what do you do right now?

And you know Harvard?

Yeah, yeah.

I mean, you're in a horrible position.

I think you're doing what you can, which is trying to stand up to the Trump administration.

So Alan Garber is trying to actually walk a very fine line.

He's trying to stand up to Trump and he's, you know, spearheading these lawsuits.

And he is

leading committees into the claims of anti-Semitism and kind of and discrimination on campus, the anti-awakeness.

So he is actually trying to address all of these things

in different ways.

And I have no idea, you know, how successful that will end up being.

But it is, it's a, it's a tough, you know, he's been put in a really tough spot because obviously he hasn't been president for long.

The last president departed on less than stellar circumstances, as did a number of presidents of Ivy League institutions.

And, you know, you're

picking up the presidency at a very difficult point in time where, yeah, you have to acknowledge that Harvard has made mistakes, right?

Not just Harvard.

A lot of universities, I think, were trying to be.

I don't even want to use the word woke because I don't think that's accurate here, but were becoming

places where people were afraid of voicing their minds.

And I wrote, I think almost a decade ago, I wrote a piece for The New Yorker about basically

the problems in academia that come from not having enough conservatives, right?

That it actually has trickle-down effects in research and the types of things that are being researched and the findings that are allowed to be voiced, and that it can be incredibly problematic.

And so I think you need to acknowledge that, and those are things that need to be rectified.

At the same time, you are currently fighting this battle against, you know, Donald Trump, who's trying to destroy higher education in the United States.

And that is an overriding priority, which doesn't dismiss all of the things that have gone wrong and all of the things that are wrong with academia.

But it's just, it's kind of this fine balance where you're like, yeah, I know I'm not sympathetic because I've fucked up and I've done all of these things.

And yet right now I'm one of the people best positioned to fight this.

And so let's just try to figure out how to prioritize this and how to not spread ourselves too thin so that we can actually protect higher education and then start to remedy the problems that we have internally.

Because I do think those are problems that need to be remedied, right?

Like you can't just be like, well, now we're all united against Trump.

So let's forget we ever did anything wrong.

I think that all of these things, you remember, Nate, like...

less than, you know, like six months ago, I was railing against Harvard because I was really pissed at the way they were reacting to a lot of things.

Now I think, now I'm supporting it because they're being attacked.

It's the classic psychology thing, right?

Where you can actually unite a lot of people who criticize you when you attack, right?

All of a sudden, these various factions, when they feel these

life-threatening attacks from the outside, all of a sudden these factions start uniting.

And I think that's what's happening at Harvard right now, as it should be.

But we still have to realize that, you know, there are all of these different issues.

And it's a really, I mean,

it's a shit show.

That's a technical term, Nate.

Yeah, look, I'm not afraid to use the term woke.

I mean, you know, critical race theory and intersectionality, all these things kind of come from an extremely academic context, right?

And look, the thing I'm probably most concerned about

is the fact that I think the quality of academic research is often has lots of problems, but like political bias in predictable directions is among those problems.

I'm trying to think of a politically expedient solution that the Trump administration might like and that people would find tolerable on the Harvard side, right?

We'll collect data on the political affiliation of our students and faculty.

We will make

concerted efforts to,

I'm not going to try to frame this right, concerted efforts to like hire conservative scholars, right?

Maybe you have like,

you know, if you have an ethnic studies department, maybe you have a conservative studies department.

No, I mean, that's, isn't that, you know, exactly what

people have been trying to reverse by kind of with the affirmative action rulings, right, that you have quotas for certain types of people.

So I think that the better way to do it is basically having blind hiring, just like you have blind admissions.

You are willing, you just don't, you don't ask any of that and you don't disqualify people because they have conservative leanings, right?

And that it's not even, it's not a thing because you don't really care.

Like, I don't care if my,

you know, neuroscience professor, well, maybe not neuroscience as insofar as I study psychology, but if my astrophysics professor is, you know,

what their political views are, as long as they correctly, you know, as long as they're one of the best astrophysicists in the world.

So I don't think that that's, you know, that what you're proposing is affirmative action for politics.

I'm just saying, like, it's slightly stupid, but like maybe J.D.

Vance would like it.

But, but you're saying like, do something that he would like.

Columbia tried to do everything and had all of their funding frozen anyway.

Right.

So like this is the problem that like actually doing, trying to appease is not the correct play here.

I think that's actually the completely wrong game theoretical way to respond here

because it's not working.

And that's a meta thing across everything that the White House does.

You see it playing out on tariffs too, where on the one hand,

they reverse course and undercut themselves.

all the time, right?

You know, and they're sloppy enough where they probably, maybe if they're doing this right, they could have had more likelihood of success in the courts than they have, right?

On the other hand, they don't necessarily hold up their quid pro quo unless it's kind of very transactional and explicit, right?

So they, they're not even like

presenting clear enough information to make themselves easy to bargain with, even if it were some theoretical

optimal solution or or win-win.

Yeah.

So, I mean, so I guess, you know, we're, we're just in a very sticky spot and we'll have to see how it plays out.

I think Harvard is doing its best.

And, you know, I wish that I could give it, you know, a playbook, Harvard, This Is What You Should Be Doing, but I can't.

So

let's just

hold out hope.

I mean, there are real changes that like

Harvard should provide.

And

I think Steven Pinker has advocated for a lot of them.

Like, as I said, I'm biased, but I think that

Harvard should listen to Steve Pinker.

That's my advice.

I think he's one of the smartest people I know who actually knows a lot about this and has very rational views and can advise on this.

It is almost 11 p.m.

for me in Hong Kong, so I think we're going to wrap it for this week.

The World Series of Poker has already started.

You and I are not going to be out there for another, it's another basically two weeks, I think, for us, week and a half.

So let's wish all of our risky business listeners who are already in Vegas for the World Series of Poker good good luck.

We hope that you all crush it at the tables, and we're looking forward to joining you soon.

Let us know what you think of the show.

Reach out to us at riskybusiness at pushkin.fm.

And by the way, if you're a Pushkin Plus subscriber, we have some bonus content for you.

That's coming up right after the credits.

And if you're not subscribing yet, consider signing up.

For just $6.99 a month, what a nice price.

You get access to all that premium content and ad-free listing across Pushkin's entire network of shows.

Risky Business is hosted by me, Maria Konakova, and by me, Nate Silver.

The show is a co-production of Pushkin Industries and iHeart Media.

This episode was produced by Isabel Carter.

Our associate producer is Sonia Gerwit.

Sally Helm is our editor, and our executive producer is Jacob Boldstein.

Mixing by Sarah Bruguer.

Thanks so much for tuning in.

This is an iHeart podcast.