The Botched Software Update That Cost $600 Million

Hey everyone, it's Jess.

We're off for the holiday, but we wanted to share a story about how one company's disastrous software update led to widespread customer anger, layoffs, and a complete leadership overhaul and ended up costing $600 million.

Here it is.

Updating your software.

It's one of our modern, common chores.

Mostly, it's annoying, inconvenient, but we do it because it's supposed to make sure our stuff works better.

So when a software update somehow makes things worse, people get mad.

Like back in 2014, when an iPhone update caused a bunch of people's phones to crash.

The latest software update called iOS 8.0.1 meant to fix software bugs, reportedly crashing some users' phones instead.

Or in 2016, when an update to the Nest thermostat left people angry and cold.

Their internet-connected thermostats have been malfunctioning ever since they got a software upgrade last month.

Or last year, when a CrowdStrike software update caused major travel delays.

It was a faulty software update by cybersecurity company CrowdStrike that caused disruptions across multiple industries.

In the best case scenarios, companies act fast and fix the problems, and we can all move on.

But our colleague Ben Cohen recently wrote about a software update that has plagued a company for months now.

It was so buggy that it turned into one of the most disastrous software updates in the recent history of consumer technology, which I know sounds like a bit of an exaggeration, but it's kind of not.

The company with the software update from hell is Sonos.

It makes high-tech speakers that are controlled through its app.

And when Sonos updated that app last spring, a lot of users suddenly suddenly ran into all kinds of issues.

Many couldn't do basic things like connect to their devices.

Recently, they had an app update.

Oh my god, I can't get anything to play on it.

Sometimes I have to spend 20 minutes trying to figure out what the heck is even going on with my devices.

But everybody's mad about the app.

The app, the new release, has been a disaster.

Don't buy Sonos products.

Don't do it.

Sonos has apologized and spent months trying to fix the problem.

But customers are still upset, and the issue has hit the company's reputation, led to layoffs in a leadership overhaul, and cost it hundreds of millions of dollars.

Welcome to The Journal, our show about money, business, and power.

I'm Jessica Mendoza.

Coming up on the show, Sonos and the terrible, horrible, no good, very bad update.

This episode is brought to you by Credit Karma.

A good podcast helps you connect the dots.

It pulls in interviews, backstories, and multiple POVs to help you see the full picture.

It's what the journal aims for.

Credit Karma does the same thing, but for your money.

Just link your accounts and the app can show you ways to save, pay down debt, and build smarter spending habits.

Get the full picture of your money in one place.

Download Intuit Credit Karma to get started.

This episode is brought to you by Contentful.

Marketers, you know that feeling when your creative clicks?

When that social post sends engagement through the roof?

When your outside-of-the-box campaign hits ROI positive?

When a personalized homepage turns prospects into customers?

It's utter marketing bliss.

Contentful helps you create tailored omni-channel experiences without working overtime.

No stress, no limits, only possibilities.

Get the feels at contentful.com.

Do you listen to music at home?

And what do you listen to?

I do listen to music at home.

And if I had my choice of what we would listen to, it would probably be a lot of Taylor Swift.

But I don't have a choice because I have a three and a half year old daughter.

So what I've been listening to a lot of lately is the soundtrack to the movie Cars 2.

Oh my God.

When Ben and his daughter listen to those bangers from the car soundtrack, their device of choice is not a Sonos.

I am not a Sonos household.

However, I have learned that basically everyone in my life belongs to a Sonos household.

Tell me about Sonos.

What is the company known for?

Sonos is known as a premium home audio equipment company, and it really revolutionized home audio by creating this ecosystem of smart audio products that work seamlessly with each other.

And in fact, when I talk to Sonos users, The company that often comes to mind is Apple.

Like if you're an Apple user, if you have an iPhone, probably you have a MacBook or you have an iPad and you want to be able to control all of it within that same ecosystem.

The idea is you put your Sonos devices on the same Wi-Fi network and they can talk to each other.

And you can control all your devices with a single app on your phone.

And people who own Sonos products don't just own one Sonos product.

The average Sonos household owns three products.

So maybe that's a speaker with a soundbar under the TV or a portable speaker that they can bring on the road or headphones.

I mean, there are a lot of people who own like lots of Sonos products because they need speakers in each room of the house.

And when your three and a half year old daughter is listening to Cars 2 in her bedroom, you might want to listen to Taylor Swift in another room.

Or you might want to listen to Taylor Swift in all the rooms, and then you can also do that.

Right.

Sonos has been around since 2002, and it won over hardcore audio lovers with its emphasis on quality sound.

On its YouTube channel, there are videos about how Sonos users can really perfect their TV sound.

With a quick tapping the app, you can make dialogue even clearer and what the best Sonos speakers for gaming are.

Try Beam to unlock the 3D effect of Dolby Atmos.

Next, Parasub to Fuel.

Sonos products range from a couple hundred bucks to thousands of dollars for some of their sets.

And although the company is a lot smaller than other tech companies that also sell speakers, like Amazon, Sonos was able to carve out a place for itself in the luxury audio space.

By the end of March last year, the company was worth over $2 billion.

Then, in April, the company announced that it was upgrading its software.

The old app was sometimes hard to manage, and the company wanted an update to make it easier for them to release new products.

In a statement, then CEO Patrick Spence said, quote, after thorough development and testing, we are confident this redesigned app is easier, faster, and better.

The new app was released globally on May 7th, 2024 as a software update.

But lots of customers had problems with it almost immediately.

So what was it exactly that happened?

Basically, everybody noticed right away, like the very first day, in part because it was pretty hard not to notice.

Sonos users couldn't use basic features of their speakers.

They couldn't access their own audio systems.

It was almost as if these speakers had become like sleekly designed bricks.

Like very expensive bricks.

Very, very expensive bricks.

And what actually happened with the app kind of depends on the user.

Some found that it was like missing essential features of the old app, like the ability to edit playlists on the fly or set alarms for when they should wake up in the morning.

Some people found entire libraries of music were just suddenly inaccessible to them.

Speakers that vanished from their audio systems in the middle of a song, that basic promise of being able to control music in a room suddenly wasn't being fulfilled.

And for most Sonos users, regardless of the experience they were having, the product basically just became worse overnight.

The tech problem was complicated.

Part of it was that over the years, Sonos had continued to rely on a lot of obsolete code.

They'd done a lot of updates, though never a complete overhaul.

And they ran into issues when they tried to bring their software up to date to match their hardware ambitions.

Sonos says they looked closely at whether or not to revert back to the old app, deciding eventually it wasn't viable.

But they also struggled to fix the new one.

And then there was the PR problem.

At first, the company defended the update, according to a statement published by a tech news outlet.

The chief product officer at the time defended it as courageous to do this, like because, you know, they were releasing this new app and it would have been easy to just keep going the way they were going, but they felt that this was like a necessary change that they had to make for the future of the company.

Soon after the messy rollout, Sono started releasing additional software updates to try and fix the bugs.

And in July, Spence, the CEO, published a letter of apology.

But customers were still mad.

A lot of them still couldn't use their devices the way they wanted.

In October, more than four months after the app rolled out, Spence released another statement, this time a video.

For more than 20 years, we have been obsessed with delivering an audio experience that is easy, reliable, and sounds amazing.

The video is more than three minutes long, and it's titled, Recommitting to Quality and Customer Experience.

Recently, we rolled out a new app that fell short of this standard.

It's been painful for our customers and gut-wrenching for all of us at the company.

Ben says that for a lot of customers, the response was too little, too late.

How has all of this

impacted Sonos, the company?

In a very, very big way.

So the company has said that it has cost at least $100 million in revenue and the company had to delay two product launches last year as it was dealing with the fallout of this botched app release and the bungled response to it.

So that's $100 million in revenue.

And the company's market cap has plummeted by around $600 million since the app came out.

Sonos released an app that was supposed to be their most extensive app redesign ever.

And it kind of turned into their most expensive app redesign ever.

After the break, we asked Sonos directly about the Saga.

Are you a forward thinker?

Then you need an HR and finance platform that thinks like you do.

Workday is the AI platform that helps propel your organization, organization, your workforce, and your industry into the future.

Workday, moving business forever forward.

This episode is brought to you by FirstNet.

During September 11th, first responders struggled to communicate and respond because the networks and systems they relied on were overloaded or destroyed.

Today, they rely on FirstNet.

FirstNet was built with and for first responders, ensuring they have reliable communication wherever and whenever they need it.

FirstNet covers more first responders than any other network.

That's why it matters for every American.

FirstNet, built with ATT.

Learn more at firstnet.com/slash public safety first.

Do you remember what you were doing the day of the app rollout?

I don't offhand.

No, May 7th, 2024.

Not I remember the weeks afterwards with my phone blowing up.

That's Eddie Lazarus.

He's the chief legal and strategy officer at Sonos.

Were you hearing from a lot of different people, people you knew?

Of course.

A lot of us know a lot of people who have Sonos, and some of those people were having problems.

I actually do know that I was in Europe at the time.

I think I may even have been on a bike trip, but I got off my bike and started getting back into it.

Eddie has been with Sonos for about six years, and he was in charge of investigating what happened in the aftermath of the app rollout.

You were the one, Eddie, who did the internal audit.

Why did it go so badly?

If I could sum up in just a few sentences, it would be, we tried to do too much

too fast, and we did not apprehend what would happen when we released all this new work into the wild.

We discovered that with all the code that we had changed and all the work we had done, when the system got out into the wild, where you have people with a lot of old equipment and new equipment and mixed old and new equipment and good routers and bad routers and all kinds of other complexities, the performance was just not even close to the standard that we expect of ourselves or that our customers expect of us.

And we disappointed our customers

and we disappointed ourselves.

But I guess that's sort of

the question that I think has been on a lot of people's minds and was especially immediately after was why?

Why do so much all at once?

This was something that came up a lot on Reddit and on community forums.

You know, why did the company decide to roll out such big changes all at one time rather than do it slowly for small groups of users at a time?

We'll never do that way again.

And in retrospect, of course, it was a mistake.

But we thought we had a new modern app that was ready to go, and we wanted to give the benefits of that app to our customers as quickly as we could.

And

frankly, we should have acted with more humility, and that's one of the absolute number one lessons learned.

And as I said, we won't be doing that again.

Was there a sense that the team or the company underestimated the complexity of of what this would take?

100%.

Eddie says, one reason that Sonos went big with their new app was that they were about to launch new products that would benefit from updated software.

A big one was Sonos' first pair of headphones.

How was that product launch tied to the timing of the release of the redesigned app?

The headphones came out shortly thereafter, and we wanted the new app out into the marketplace in advance of the headphones.

Now, we delayed the launch a couple of times while we continued to work on the app to get it to a place that we felt it was ready to go.

But again, we misjudged.

And those headphones are a fantastic product.

And the app unfortunately has been, up until recently, a bit of a cloud over it.

The new product wasn't enough to boost Sonos.

And the company's total units sold, so across all its products, were down by 14% in the back half of last year compared to the year before.

Talk to me about what Sonos did to fix the problems with the new app over the course of the weeks and months that followed.

Well, we've had 22 new software releases since then, and we had a dedicated team and still do that does nothing but think about these performance and reliability issues and how to fix them.

Now we're over 90% of the features that we're missing.

It's just been, it's been the number one priority of the company.

So, just to be clear, is the new app fixed?

Well, when you say fixed, not every single feature that was in the old app is in the new app.

Almost all of them are.

On many of the performance metrics, we're actually doing much better than the old app did.

But there are a few cases involving a few of the older products where we're still having performance issues.

There are a few other issues out there.

So, again,

we have our eye on not only getting to that parity bar, but to go well beyond it.

Sonos has tried to be transparent with its efforts to fix the app, holding Q ⁇ A's on Reddit and the company's community forum, and sharing a Trello or project management board with the public.

But the damage to the company's reputation has been hard to shake.

I really think there's only one way to fix reputational damage like that, and that is to show that we're doing the right things by our customers every day.

And we're going to make our software and our experience better than anyone else in the field.

And as people see that we're delivering on that, I certainly hope they'll give us another try.

So we just have to, we have to win them back through action.

Internally, the company has also struggled.

They've had two sets of layoffs since the app update, losing around 300 employees.

Then at the beginning of this year, the CEO Patrick Spence stepped down.

Here's our colleague Ben Cohen again.

They replaced the CEO Patrick Spence with a guy named Tom Conrad, who was already on Sonos' board, but he's someone with extensive experience in product design, software, and music platforms, having been the chief technology officer of Pandora for 10 years.

He's also something of a Sonos geek.

He like really, really cares about the product.

In fact, his first day on the job, he wrote this letter to employees in which he said that he has a Sonos home system.

He watches TV with his Sonos soundbar.

And when his daughter was born, he brought another kind of Sonos portable speaker into the delivery room of the hospital.

This is a guy who is such a fan of the company's products that he even has a tattoo of Sonos headphones.

So like, he is permanently inked with his devotion to Sonos, even before he was the CEO of Sonos.

And is that passion enough to turn things around?

Like, what does Tom Conrad need to do to get confidence back in the brand?

Well, I think that's like the million-dollar question for Sonos right now, right?

What lessons can be learned here?

The first is that these apps that run our lives, they demand constant improvements and they can't have any disruptions.

The best updates are the ones that you don't notice.

And technology is at its best when it just works, right?

When it has that magic, as Steve Jobs used to say.

In this case, it just kind of stopped working.

Right.

And I think Sonos has sort of rethought its software development process throughout this time.

Like part of the issue here is that lots and lots and lots of people using Sono's products felt these changes at the same time.

And if you had rolled them out in smaller chunks, you could have contained the damage and fixed these issues before they became issues for everyone.

This episode was originally published in March.

The journal is a co-production of Spotify and the Wall Street Journal.

Thanks for listening.

We'll be back tomorrow with a new episode.

At the University of Maryland, brilliant minds unite across disciplines to ignite AI innovation.

We apply visionary technology to track wildfires, spot hidden illness, protect our food supply, and make AI's benefits accessible to all, while preparing tomorrow's leaders to explore AI's potential and put people first.

We're shaping the future of AI for discovery, for community, for the public good.