AI Isn't Making Engineers 10x As Productive with Colton Voege

This is an iHeart podcast.

Live in the Bay Area long enough, and you know that this region is made up of many communities, each with its own people, stories, and local realities.

I'm Erica Cruz-Guevara, host of KQED's podcast, The Bay.

I sit down with reporters and the people who know this place best to connect the dots on why these stories matter to all of us.

Listen to The Bay, new episodes every Monday, Wednesday, and Friday, wherever you get your podcasts.

Every business has an ambition.

PayPal Open is the platform designed to help you grow into yours with business loans so you can expand and access to hundreds of millions of PayPal customers worldwide.

And your customers can pay all the ways they want with PayPal, Venmo, Pay Later, and all major cards so you can focus on scaling up.

When it's time to get growing, there's one platform for all business, PayPal Open.

Grow today at paypalopen.com.

Loan subject to approval in available locations.

Let's be real.

Life happens.

Kids spill, pets shed, and accidents are inevitable.

Find a sofa that can keep up at washable sofas.com.

Starting at just $699,

our sofas are fully machine washable inside and out.

So you can say goodbye to stains and hello to worry-free living.

Made with liquid and stain-resistant fabrics.

They're kid-proof, pet-friendly, and built for everyday life.

Plus, changeable fabric covers let you refresh your sofa whenever you want.

Neat flexibility?

Our modular design lets you rearrange your sofa anytime to fit your space, whether it's a growing family room or a cozy apartment.

Plus, they're earth-friendly and trusted by over 200,000 happy customers.

It's time to upgrade to a stress-free, mess-proof sofa.

Visit washablefas.com today and save.

That's washable sofas.com.

Offers are subject to change and certain restrictions may apply.

There's more to San Francisco with the Chronicle.

More to experience and to explore.

Knowing San Francisco is our passion.

Discover more at sfchronicle.com.

Coolzone Media.

Hello and welcome to Better Offline.

I'm your host, Ed Zitch on as ever.

Buy our merchandise, Subscribe to the newsletter.

It's all in the episode notes.

And today I'm joined by programmer Colton Voge, who wrote an excellent piece about a month ago called No, AI is Not Making Engineers 10Xs Powerful.

Colt, thanks for joining me.

Thank you for having me.

So tell me a little bit about your day-to-day work.

What do you do for a living?

Yeah, so I'm a software engineer and I work specifically in what's called web application development.

So that's kind of like building rich applications that work in the web browser.

So think of something like Amazon or Google Drive or something where you can do a lot of interactive features within the application within a browser.

Right.

So kind of like the fundament of most cloud software.

Yeah, I would say that

if not the majority of engineers work in web application development, then probably the plurality do right now.

And it's also

a large amount of how people interact with software is now in this way, though.

Yeah, exactly.

Yeah.

And

that's why most developers perhaps work in this field now.

And you'd think something like that would be perfect for AI coding environmentally.

It is surely.

It is definitely the most applicable place for AI coding of all of them.

There's sort of a thing where people are like, you know, oh, you know, LLMs don't really work really well for my language, and they'll be talking about something like Rust, which is more of like a

high-performance language rather than a web language.

What is a high-performance language in this case?

Just something that's designed to work

on like weak hardware or at like extremely high

speeds.

So like, for example, like video games aren't built in Rust.

They're usually built in a language called C ⁇ , but they're built in a high-performance language because

you know you're you're trying to max out as much as you can do.

You're taking advantage of the high performance compute you have.

Yeah.

Whereas with a website like you know like Amazon or something like that,

you're really not pushing the limit in terms of interactivity.

And so more modest languages are fine.

Got it.

So you laid it out really nicely in the piece, so you're going to have to repeat a few things, I imagine.

But run me through why AI coding tools can't can't actually make you a 10x engineer or a 10 times better engineer.

Yeah, so basically

AI is really good at, AI is almost shockingly good at generating code, and it's almost shockingly good at generating code that

runs.

And

it can answer a lot of questions that are like difficult,

or at least, you know, annoying is probably the better word.

It's really good at dealing with things that are annoying.

It's really good at, you know, so in coding, we have this concept called boilerplate, and it's just code that you have to kind of rewrite a lot.

And as an engineer, ideally, you don't have to write a lot of boilerplate.

Ideally, you automate things and abstract things, so

you don't need to do that very often.

But it's sort of like a requirement of the job.

And it's really good at writing that because, you know, it's just, it's kind of like a low-intent, high-volume code,

a quantity over quality type thing.

So it's really good at stuff like that.

The problem is that generating code really isn't the hard part of being a software engineer.

It's one of the things that really matters.

And it's like, obviously, if you go to college to learn how to code, what you're going to spend most of the time doing is typing code.

But it's really just like one thing that you're doing.

Like in reality,

you're doing a lot more thinking about like, okay, how does this work with the systems I already have?

How do I avoid creating what's called tech debt?

So tech debt is basically like an easy way of thinking about it is when you write code that makes writing future code harder.

So an example would be, you know, you want to write like a piece of logic that let's say computes sales tax.

If you're, you know, making an online store, you'd want to compute sales tax based on where they are.

You only want to write that once.

You don't want to have two different places that

handle sales tax because then if Illinois changes their sales tax rate, you have to change two places instead of one.

Doesn't the very nature of code also mean that it naturally creates tech debt?

You can try to mitigate as much as you can, but there's not even an idea that all code is tech debt.

What LLMs, so kind of what I mentioned before, LLMs are really good at producing boilerplate, but if you get to the point where boilerplate is really easy to produce and you're not constantly thinking like, okay, how do I avoid doing this in the future?

You start, yeah, exactly.

You start generating more and more code.

And just like generating code isn't, it's kind of like writing in terms of like, you know, writing a book, like a good writer writes fewer words rather than more.

Yeah, brevity is the soul of wit.

No, it's because what I've been realizing is that there is this delta between people that actually write software and people that are excited about AI coding.

Because I had Carl Brown from the Internet of Bugs on, and it was this thing of, yeah, being a software engineer is not just writing 100 lines of code and then giving the thumbs up to your boss.

You have to interact with various different parts of the organization.

And early on in the piece as well, you had this bit called the math.

And I'll link to this obviously in the episode notes, where it's any software engineer who's worked on actual code in an actual company knows that you have these steps and where it isn't just like I code and then I give the thumbs up and I'm done for the day.

You have to go through a reviewer.

You have to wait for them to get back to you.

You have to, you refer to context switching as well, where you have to change different windows and do different things.

There's a shit ton of just intermediary work that is nothing to do with actually writing code, right?

Yeah, absolutely.

I mean, it's

it really is more of, you know, there's parts of coding that is more science, and there's parts of coding that are more art and more, you know, in college, it's really common if you're getting a computer science degree to take communications classes because you just have to interact with a lot of people.

So the standard way that a

thing gets done in a software project is you have somebody called a product manager and

that person's job is to think about what the product is as it exists now and what it should be in the future and like what features we should build.

Like basically more than anything, it's you know what should we build next?

And then you have designers who are going to decide how that thing should look.

And engineers have to be this meeting influence because they're the only ones in the process who actually know

what it's like, how hard it is going to be to build something.

So a frequent interaction is like, you know, a product manager is like, oh, we should add this, and an engineer has to kind of step in and be like, no, that's borderline impossible, or you know, that would take six months.

Right.

And you want

coding LLMs don't fix that problem.

They don't, do they actually make it easier to develop products?

I, I,

there are uses for LLMs in coding, and I quite like that.

And I'm not, I'm not even denying that.

I'm just like, how, like, what do what do those uses end end up actually creating?

Basically, you know, what I found is that I don't really like using LLMs for most product work, even though they're good at, you know, say like, oh, add a button here that does this.

They can be good at that, especially if you have a code base that

works really well with LLMs.

So some programming languages, some tools, some, we call them libraries encoding.

You know, they're basically like shared software.

Some of those things work really well with LLMs because the LLMs are just trained on the internet.

So sites like Stack Overflow and sites like Elite Code and stuff like that.

And so if they have a large body of this code to work within their training data, like these tools and these languages, they tend to be better at writing them.

And so, you know, I work primarily in JavaScript, as do most web application engineers.

And LLMs are quite good at JavaScript.

But I still don't like to use it that much because it tends to just not

understand context very well.

That does not mean in practicality.

So it's kind of just like understanding the existing resources that you've already built in your code base.

So this is like the avoiding writing the same sales tax thing twice.

They tend to default to rewriting things.

And they tend to struggle to

reuse the same style.

So an important thing in coding is maintaining consistent styles of your code.

So there's a whole theory and practice of actual,

like how your code should look, like as like how the texture, like how you should avoid,

what things you should avoid, because there's bad patterns, there's good patterns, and there's everything in between.

I think it's very similar to like, you know, when image generation AI kind of came onto the scene.

You had a lot of people being like, all right, well, graphic designers are done.

You know, they're out of a job.

They don't need to do anything.

Because I can generate a logo now, and it looks pretty good.

Like, I can generate a logo for my nail slot or whatever.

And look at it.

It looks great.

But as soon as you try to generate a second logo that looks like the first one and, you know, maybe has a slight modification or something like that,

it's really bad at it.

It's really bad at using the context.

And you're like, okay, now I need a graphic designer.

And as soon as you need,

you know, it's a good thing.

You need to firing one anyway.

Yeah, exactly.

And so, and, and, you know, that's, it's very similar to engineers.

You know, they a lot of what you're doing is sort of working around context and avoiding redoing things and avoiding moving away from your existing styles.

And what is the

critical nature of styles?

Is that because

is that so that people looking at your stuff in the future can say, okay, this is what they were going for.

This is so that everything doesn't break?

Yes, exactly.

It's about consistency.

It's about

so there are certain, you know, for example, JavaScript, which is the main language I work in, is almost infamous for basically allowing you as the developer to do all sorts of buckwild stuff that you should like never do coding-wise.

Like a lot of old patterns, a lot of recycled stuff, a lot of,

it just lets you, it's an extremely varied language in the things it supports.

And so what you want to do when you're writing in JavaScript is only use the good parts.

You want to strategically avoid doing a bunch of bad things.

And some of those bad things, there's tools that will automatically detect if you're doing them.

Live in the Bay Area long enough and you know that this region is made up of many communities, each with its own people, stories, and local realities.

I'm Erica Cruz-Guevara, host of KQED's podcast, The Bay.

I sit down with reporters and the people who know this place best to connect the dots on why these stories matter to all of us.

Listen to The Bay, new episodes every Monday, Wednesday, and Friday, wherever you get your podcasts.

Tired of spills and stains on your sofa?

WashableSofas.com has your back, featuring the Anibay Collection, the only designer sofa that's machine washable inside and out, where designer quality meets budget-friendly prices.

That's right, sofas started just $699.

Enjoy a no-risk experience with pet-friendly, stain-resistant, and changeable slip covers made with performance fabrics.

Experience cloud-like comfort with high-resilience foam that's hypoallergenic and never needs fluffing.

The sturdy steel frame ensures longevity, and the modular pieces can be rearranged anytime.

Check out washable sofas.com and get up to 60% off your Anibay sofa, backed by a 30-day satisfaction guarantee.

If you're not absolutely in love, send it back for a full refund.

No return shipping or restocking fees.

Every penny back.

Upgrade now at washable sofas.com.

Offers are subject to change and certain restrictions may apply.

So I've shopped with Quince before they were an advertiser and after they became one.

And then again, before I had to record this ad, I really like them.

My green overshirt in particular looks great.

I use it like a jacket.

It's breathable and comfortable and hangs on my body nicely.

I get a lot of compliments and i liked it so much i got it in all the different colors along with one of their corduroy ones which i think i pull off and really that's the only person that matters i also really love their linen shirts too they're comfortable they're breathable and they look nice get a lot of compliments there too i have a few of them love their rust coloured ones as well and in general i really like quints the shirts fit nicely and the rest of their clothes do too they ship quickly they look good they're high quality and they partner directly with ethical factories and skip the middleman so you get top tier fabrics and craftsmanship at half the price of similar brands and I'm probably going to buy more from them very very soon keep it classic and cool this fall with long-lasting staples from quince go to quince.com slash better for free shipping on your order and 365 day returns that's q u i n c e dot com slash better

free shipping and 365 day returns quince.com slash better

in business they say you can have better cheaper or faster but you only get to pick two what if you could have have all three at the same time?

That's exactly what Kohir, Thomson Reuters, and specialized bikes have since they upgraded to the next generation of the cloud.

Oracle Cloud Infrastructure.

OCI is the blazing fast platform for your infrastructure, database, application development, and AI needs, where you can run any workload in a high availability, consistently high performance environment and spend less than you would with other clouds.

How is it faster?

OCI's block storage gives you more operations per second.

Cheaper?

OCI costs up to 50% less for computing, 70% less for storage, and 80% less for networking.

Better?

In test after test, OCI customers report lower latency and higher bandwidth versus other clouds.

This is the cloud built for AI and all your biggest workloads.

Right now with zero commitment, try OCI for free.

Head to oracle.com/slash strategic.

That's oracle.com/slash strategic.

What are these bad things?

Is it just that Java?

What's so special about Java that it allows you to do that?

Yeah, so JavaScript, which is technically different from Java.

Yeah, no problem.

Basically, the reason JavaScript has all these bad parts is because it was created in 10 days as a toy language at Mosaic, which was the precursor to Firefox, you know, the first browser that really gained traction and

the internet.

Oh, of course.

I'm old enough to have used Mosaic somehow.

Yeah, so JavaScript was created in like 10 days almost like just as a thing to test, like just something to try.

Like, okay, what if we put a coding language into the browser that people could just ship with their websites?

And

in response to that, other companies said, okay, well, we need to support JavaScript as well.

And so when Internet Explorer came out, they shipped their own version of JavaScript, which was slightly different.

And so you have this engineer that was kind of built on like these like very hacked together principles that was then rebuilt with slightly different principles.

And then what you have now is 30 years later, sort of the conglomeration of all those things into one language.

And it's improved dramatically in that time.

But the nature of coding languages is there's a lot of backwards compatibility, like a video game, where, you know, you...

you know, you want to be able to build a computer that can run a game from 1991.

So you want the browser to be able to still run code largely that was written in 1995 or whatever.

So a lot of that stuff still exists.

And what they've done is introduce new patterns that can be better.

And so

there's a sort of an equal amount of like avoiding that.

And then also just

consistency.

So there are times where there's two good ways to write ways to do something, but you always want to do it only in one way so that any time someone reads it, they see that pattern and they say, okay, I know exactly what this does.

Right.

And this fundamentally feels like something large language models are bad at because they don't know anything and they don't create anything unique either.

They just repeat.

Yeah, exactly.

I mean, they are statistical inference models.

And so they are very good at generating what they think should come next based on probabilities.

And you can...

When you're training a large language model, you can try to push it in one way, push it in another way to say like, no, don't do that, don't do that.

But like trying to do that on the broad spectrum of all things code is basically impossible.

And so, you're going to get things that default to the way that they were done on the internet.

And so much, like, like I said, JavaScript is 30 years old.

It's extremely,

it's gone through a lot of dark times in terms of patterns.

And so, on the internet, there are just fast swaths of terrible JavaScript.

And

so, it's just these models are trained on bad code as well as good code.

Yeah, exactly.

And,

I'm sure they're trying very hard when they're training the models to filter some of this stuff out, but like trying to do a broad filter on hundreds of thousands, millions even of words.

How long do you say again?

Like decades?

I mean, JavaScript's been around for 30 years, as have most programming languages that are in use right now.

Right, but this particular one seems particularly chaotic in how it's sprawled.

Yes.

JavaScript is a particularly thorny language, yeah.

So what you're describing, I'm not trying to put words in your mouth, is that this stuff doesn't do the stuff that everyone's saying it does.

Like, it's not replacing engineers.

It doesn't even seem like it could replace engineers.

Like, it's just not, it doesn't do.

In fact, maybe it's fair to say that soft, it doesn't do software engineering.

That's almost the perfect way to put it.

It does coding, but it doesn't do software engineering.

And software engineering is kind of this broader practice of like everything that comes together around coding.

So, you know, some people really integrate LLMs deeply into their everyday work and they do similar work to what I do.

So, you know, there are people who are primarily having like LLMs write their code and they can be good engineers,

but they're

intervening pretty constantly from what I understand and you know they're having to sort of redirect it make sure it stays on patterns and all that stuff and

there's just kind of, you know, this is the reason that I don't really use LLMs that much is there's just a constant tension between

the

lack of context they have and, you know, what you want them to do and constantly reviewing the code and making sure it's up to standards when I know I could just write it and, you know, it'll take about as much time as it would take prompting and reprompting.

It's just, I get better code that way, and that's, that's what I care about most.

So do you think it's kind of a mirage almost, the productivity benefits?

I think it would vary a lot.

I think, yes, broadly.

I think that there are productivity benefits, but like these like huge outsize, like, oh my gosh, I'm galaxy-brained right now.

I'm doing so much.

I think that that is largely a mirage based on like

extrapolation of like small wins.

I talk about this a little bit in my article that, you know, there are one thing that I use LLMs for a lot, and not a lot, sometimes,

but that they are really good at is, you know, sometimes you're writing code and you're like, I need to write a thing that I will only run once and I will throw in the trash, or I need to write a thing and it uses this tool that like I don't have the time to learn.

But like I'd really like to have this tool.

Like I'd really like to just like use this once.

And so you can vibe code something and not really understand what the code does.

You can run it once and not validate the output and make sure it works fine.

And you know, I've saved in that time, you know, it might have taken me five hours to like learn how to use this tool properly and like learn how to use it with good standards.

It could even have taken me more.

And instead, I spent 20 minutes.

You know,

wouldn't that be dangerous because you don't know how it works?

Exactly.

Exactly why I like to only use it for these like one-time, like, low.

You know, so for example, I wrote some code the other week, and

I basically refactored some existing code.

So I adjusted how it worked a little bit.

And I realized there was a way that

I could break some existing code with it, but I didn't have an easy way of checking across the entire code base.

So we have tests in our code base that would catch issues, but there's always some code that isn't quite up to par with tests.

And so I had this idea that

a really simple language parser could go through and make sure that this code was right.

You know, something that is only like 30 lines of code, but language parsing is really complicated and the tools to do it are, you know, there are a lot of them and they're very well supported and because language parsing is a huge deal.

But it would it would take me a lot of time to learn that.

So I just I just vibe coded a little tool.

I said, hey, find me every case where I use this function and I call it like this in this entire code base.

Or actually I said, write a script that will find me that.

And I looked at the code.

I was like, yeah, that looks right.

That looks right.

That looks right.

I ran it.

It said there was no issues in my code base.

So I intentionally created an issue just to make sure it worked.

And it worked.

It showed it.

And so then I got rid of the intentional issue.

I was like, okay, this is probably good.

And I pushed my code and it turned out to be good.

But that's, you know.

That also seems low stakes.

Exactly.

And it's what I would say wouldn't scale.

So if I

really needed to start doing like language parsing constantly, like I was doing it daily at my job, I simply would have to learn.

Like you said, you know, how do you know that there aren't any issues?

I would have to learn the tools.

Right.

So the time that I saved was by avoiding learning for this one thing.

But eventually, like, if you're going to make something your full-time job, you have to learn it because you can't fully trust the output.

Because otherwise, it isn't your job.

Like,

you were just kind of mimicking.

Yeah, and, you know, you have to, you know, the LLMs make mistakes in occasionally extremely catastrophic ways.

There's a thing called slop squatting.

Have you heard of slop squatting?

No, but please tell me.

I love this term so much already.

Yeah, so basically

you might have heard of like domain squatting.

So this is where like...

I think I know what this is, and I'm very excited to hear more.

Yeah, so like, you know, this was a thing where like in 1991, you're like, okay, I think the internet's going to be big, so I'm going to grab nike.com.

Right.

I'm just going to hold it.

So that's squatting a domain.

So slop squatting is basically where you

the LLMs, you know, they're statistical inference machines.

They don't actually understand

what they're doing.

And so, you know, sometimes they will import

software.

They will.

Oh, is this when it looks on GitHub for something that doesn't exist?

Yes, and

it will just add code to your project that's like import this thing and it will install it and it'll say like okay, this is the library that you want to use and it's it's not right it's it's either misspelled or it's you know for example you would be looking for a library called left dash pad and it would import something called left pad with no dash

and so what people realized is that they can

you know when the LLMs, they're statistical machines and so they frequently they do the same thing a lot.

They make the same mistakes a lot so you could grab that library left pad with no dash

and

inject and put code in there that works that does the thing that the library is supposed to do but also retrieves you know all of the uh secrets that are in your environment or like looks for your crypto wallets or like uh production database passwords and stuff like that and if you're someone that can't read code or can't read that kind of code you would have no idea this is happening You'd have no idea.

And even if you're somebody who does read code really well, if you

look at that, you said it's imported left out of, you're like, that sounds right.

That looks right.

Like, unless you are really familiar with this library, and even if you are familiar, you might just gloss right over it.

So it's really, really, really dangerous.

And like, this is the type of thing that could,

you know,

Like maybe if like if the LLM is making you even twice as productive, you know, that doesn't mean much much if, you know, there's a chance you could destroy your entire company with a catastrophic security breach, you know?

Yeah.

Leak all your database to this hacker.

And so

not good.

Yeah, exactly.

And is that becoming more prevalent?

I haven't heard much of it like happening in the wild, but it's just one of those things that is bound to happen because, again, these are just...

just statistical models.

They don't have the ability to really reason about the actual nature of of the things that they're doing they can try you know they can make a sub-LLM call and ask the other LLM like does this look right

but then you're you know burning more and more tokens and also at some point you are trusting the statistical model to measure a statistical model's ability to do its job yes exactly and it you know it kind of devolves um anybody who's who's used an LLM for coding knows that the deeper you go into like a single prompt, like the more back and forth, the larger the the context window,

the more garbage it gets.

And so like as you have things like working off of other LLM input, which is effectively what you're doing in a large context window, you know, it's just

the LLM is sort of reprocessing the text that it generated and that you've added to it.

It steadily gets worse the later in the context window.

So basically all of these sort of mitigations are, you know,

They've made surprising progress with the way that like, you know, things like this don't happen happen as like just raw hallucinations of like i think this library exists they happen a lot less now than they used to but they're they're just always

they're always gonna risk and they're also always gonna be there yeah like it's not it's not really something you could unless we invent new maths

yeah i mean at least with the way we approach ai right now which is is is based purely on language as tokens and and and it can't really fundamentally like understand things outside of word probabilities.

So, where is this pressure coming from?

Because it feels like it's everywhere.

And you've got people like Paul Graham who are wanking on about, oh, I met a guy who writes 10,000 lines of code, I think.

He said, What

are we just finding out how many people don't know how coding works?

I think a lot of that, yes.

Like I said, it's exactly the same way as people, you know, when the first image generation models were like, oh, great, we don't need graphic designers anymore.

And then they, or, you know, oh, great, we don't need customer support chat anymore.

Because they fundamentally don't understand what those roles do.

You know, they're.

They think graphic designer and they think image generator.

But a graphic designer is a human being that's dealing with different stakeholders, dealing with people saying, like, no, no, no, the logo can't have that, or, you know, yes, the logo must get this.

Yeah, yeah, exactly.

And they're dealing with, you know, different requirements.

And then they're needing to make variations, which is something that LLMs are not always very good at.

Or I should just say, generative AI is not always very good at.

Yeah.

And so

strange.

Part of it

is people who, you know, like I said, you know, you get these brief bursts where you're like, oh my God, it just saved so much time.

And you extrapolate that.

Like, so some of it, I think, is engineers who are just like, actual engineers who know how to code, who see these things, and they think, like, I did this today,

and like, I must have been so much more productive as a result.

You know, I saw this one thing happen, but they don't really actually measure in depth what they produced, and was it more than what they would have produced.

There have been some studies to measure that, and they haven't looked particularly good for LLMs.

You know, if you actually compare people

using AI versus not using AI,

the results don't always look particularly great for AI.

And one thing that's very common out of that is that people overestimate their performance.

And I think that might be a problem across all, like a lot of people don't.

I also think my grander theory with all of this is a lot of people don't know what work is.

Like a lot of these investors and managers and such, and even people in the media, don't seem to actually know what jobs are and how the jobs work.

And they think that things like coding is just, like I said earlier, yeah, just walk into work, I'll write 10,000 lines of code, and I walk home.

But now I can write 20 billion lines of code because that's all my job is.

Yeah, absolutely.

I mean,

this is something I talk about in my article: there's always this like degree of separation.

And the people who are talking the most about AI coding aren't really coders, and they're not really providing like detailed reproduction steps.

You know, I know engineers who love using AI like every single day and they use it for like all of their projects.

And I know really good ones who do that.

And if I asked them like, hey, how could I like be more likely, be more like you?

How could I

be a better coder?

One of the last things they'll say is probably, you know, start using AI more.

It's really just a tool to accomplish part of their job.

And so, so yeah, I think there's a lot of, you know,

there's probably you know plenty of genuine people who just like

you know they've never written a line of code in their life.

They pull up lovable they say generate me an app that does this and like they legitimately are like oh my gosh it actually worked like I I can code

And they just

they just you know naturally you know they don't realize it they don't realize that like there is so much more to this actual practice and you know

they're they're they're just not in tune with the way that coding actually works.

Yeah, it's a little bit sad as well because it really feels like a lot of this is just

the point you've made about like the image generators.

It's just this immediate moment of, wow, look what this could do.

Imagine what it could do next.

And then you look at what it could do next and it can't.

Like it looks like it can generate code, but it can't actually generate software.

Like it can't, it doesn't seem to do the steps that make software functional and scale because there's these tendrils

of from software into the infrastructure to make sure it can be shown in different places or to make sure that it actually functions on a day-to-day basis.

Yeah.

And it's like, yeah.

It's a pretty simple curve.

Like, you know, you start out and it generates so much code that's like pretty correct really fast.

If you start a completely bare project from

just like a lovable prompt or something like that.

But as you go on, you know, it's like the curve kind of flattens and it goes down and it becomes less and less productive.

And I think eventually, usually, like pure vibe-coded projects hit a pretty big wall because you're just introducing so much code and it's not consistent, it's not

using shared tools.

And so you eventually just end up with this, you know, they call it spaghetti code.

It's, you know, code that is so interwoven and difficult to understand that like you can't actually see what's going on with it.

Yeah, it feels like context is the whole problem.

Just because even not to say like LLM generated writing is dog shit.

And I think it's worse than code because code code code is functional in a way that I don't think writing has to be.

Writing conveys meaning, but good writing is usually more than just...

I am entering this writing into someone's brain so something happens in the way that code is.

But with writing, they share the same problem, which is great writing has contextual awareness.

It builds, it connects, there is an argument or there is an evocation from it.

In the case with software, it appears to be if you don't know every reason that everything was done and fully understand the reasons that were previous and the reasons that are happening right now, you kind of will fuck something up naturally.

Even if you do know how to code, if you just don't read any of the notes,

if it's not clear why things were done, things will break anyway, right?

Yeah, exactly.

And great writing, you know, it knows when to, you know, a great writer knows, you know, when they need to explain something and when they don't need to explain something.

You know, they know, you know, I'm writing a trade publication, like I don't need to explain how friction works.

Or I'm writing a, you know, a public press release, okay, I do need to explain how friction works, or, you know, whatever.

So it's, it's very,

you know,

these are the things that the LLMs are like, yeah, exactly.

Just like very not good at.

You know, they can generate stuff that looks good, but it's just the more you try to build on top of it the more it'll it'll end up restating like you know

LLMs are really good at writing like the classic school level five paragraphed essay but everybody who actually writes anything at all knows that like the five paragraphed essays that you wrote in high school are like terrible and like nobody wants to read something structured as like you know premise three argumentative paragraphs conclusion like that's it's really bad like unconvincing writing.

And it's the same with you know LMs are really good at making toys and like quick things that like are fun or maybe a little bit useful in certain situations,

but really bad at writing things that you know

like a book level type things.

Like you know, AIs are horrible at writing books because they restate things and they lose track of what they're talking about.

And the more stuff it creates, right?

The more it looks over, the more confused it gets.

Exactly.

It's really, really similar.

I think that coding has really just followed the same trajectory as all of these other things where we're like, oh, we don't need copywriters anymore.

Oh, we don't need

designers anymore.

We don't need graphic designers.

We don't need this, that, and the other.

We don't need lawyers anymore.

We can have an LLM beer lawyer.

And you just very quickly realize that

these jobs aren't

dumb factory

producing, pulling a lever type jobs.

They're about interaction.

And that's not an insult to factory labor.

That is very hard work, but it's not like a repetitive action that is always the same thing.

No, not at all.

And like a factory worker, you know,

is doing a lot more than just pulling levers,

of course.

But it's not like just hitting a button, but I think people condense coding to this thing.

Yeah, it is.

It's very similar to like robotics in factories, too, where, you know, the promise is like, oh, you know, we'll just have a robot do this thing that a human does.

But the human is doing a lot more than just pulling the lever.

They're like observing the process.

They're making sure that things are not getting broken or getting gummied up and stuff like that.

So

there's just limits to what machines can do when they're not actually intelligent.

And that's just what it comes down to.

Do you buy that any of these companies like Google are writing 30% of their code with AI?

The thing about those numbers is that they are super easy, maybe not to game, but to first of all, I don't know anyone who's actually measuring this in a really

effective way.

Because the thing about your code editing editor is that it's

a lot of people, I don't like to use the AI auto-completion, but some people do.

There's pieces where it'll, you know, you'll start writing some Boilerplate and it'll pop up what it thinks you mean to write, and you'll just hit tab, the tab key on your keyboard, and it'll just finish the line of code that you were writing.

And maybe that would be written in AI.

Yeah, yeah, exactly.

And it might even

very often what it produces is wrong, but it's like 75% right.

So you're like, oh, I can just like use these keystrokes.

I can save these keystrokes and I can just fix what it did wrong.

Oh, God,

it's like saying autocorrect wrote parts of your book.

Yeah, it really is.

And,

you know, there are times where, like,

an AI does fully write the code for a feature.

So people use, you know,

things where they can prompt an LLM and it will

with like what they want the feature to be or what they want the bug fix to be.

And it will go and will write all the code and it will make the sort of, we call it a merge request, but that's the request for new code

that then gets reviewed.

It will go all that way.

But the the thing is, you know, it might have written the code, but somebody took time away from, you know,

their normal job, which would just be writing code, to write a really good prompt to make sure it didn't screw it up and then reinteract with it.

And so, you know, did it write the code?

Yes, but did it do the task?

Not really, because it needed somebody else to do some support work to make it even possible for it to do it.

And that person could have, that person needed to be technical and needed to be able to say, like, oh, you need to look in this part of the code base.

And so you end up just getting

the same type of like actual work from the actual specialist who knows how to code, the same amount of work.

They're just doing something slightly different.

Cole, it's been such a pleasure having you here.

Where can people find you?

For sure.

My blog is colson.dev, C-O-L-T-O-N.dev.

I don't post that often because

I work full-time and I just post when something really gets to me, but

I might say some things here and there.

And you have your excellent blog that I brought you on for.

Yeah, that's my, it's my most recent one.

You can feel free to check it out.

I'm sure the link to that will be in the description.

But yeah, it was great being here.

Thanks so much.

Thank you for listening to Better Offline.

The editor and composer of the Better Offline theme song is Matt Osowski.

You can check out more of his music and audio projects at mattasowski.com.

M-A-T-T-O-S-O-W-S-K-I dot com.

You can email me at easy at betteroffline.com or visit betteroffline.com to find more podcast links and of course my newsletter.

I also really recommend you go to chat.where's your ed.at to visit the Discord and go to r slash betteroffline to check out our Reddit.

Thank you so much for listening.

Better Offline is a production of CoolZone Media.

For more from CoolZone Media, visit our website, coolzonemedia.com, or check us out on the iHeartRadio app, Apple Podcasts, or wherever you get your podcasts.

This is an iHeart podcast.