The Botched Software Update That Cost $600 Million

Updating your software. It's one of our modern, common chores.
Mostly, it's annoying, inconvenient, but we do it because it's supposed to make sure our stuff works better. So when a software update somehow makes things worse, people get mad.
Like back in 2014, when an iPhone update caused a bunch of people's phones to crash. The latest software update called iOS 8.0.1 meant to fix software bugs, reportedly crashing some users' phones instead.
Or in 2016, when an update to the Nest thermostat left people angry and cold. and cold.
Theirconnected thermostats have been malfunctioning ever since they got a software upgrade last month. Or last year, when a CrowdStrike software update caused major travel delays.
It was a faulty software update by cybersecurity company CrowdStrike that caused disruptions across multiple industries. In the best-case scenarios, companies act fast and fix the problems, and we can all move on.
But our colleague Ben Cohen recently wrote about a software update that has plagued a company for months now. It was so buggy that it turned into one of the most disastrous software updates in the recent history of consumer technology,

which I know sounds like a bit of an exaggeration,

but it's kind of not.

The company with the software update from hell is Sonos.

It makes high-tech speakers that are controlled through its app.

And when Sonos updated that app last spring,

a lot of users suddenly ran into all kinds of issues. Many couldn't do basic things like connect to their devices.
Recently, they've had an app update. Oh my God, I can't get anything to play on it.
Sometimes I have to spend 20 minutes trying to figure out what the heck is even going on with my devices. But everybody's mad about the app.
The app, the new release has been a disaster.

Don't buy Sonos products. Don't do it.
Sonos has apologized and spent months trying to fix the problem. But customers are still upset.
And the issue has hit the company's reputation, led to layoffs and a leadership overhaul, and cost it hundreds of millions of dollars.

Welcome to The Journal, our show about money, business, and power.

I'm Jessica Mendoza.

It's Thursday, March 6th.

Coming up on the show, Sonos and the terrible, horrible, no good, very bad update. This episode is brought to you by the Toyota Grand Highlander.
Life's journey brings constant change filled with exciting surprises and grand challenges. And with its spaciousness, power, and style, the Toyota Grand Highlander is more than a vehicle.
It's your partner in embracing all life throws at you.

From the spacious adult-sized third row to seating for up to eight passengers,

the Toyota Grand Highlander is up for every grand challenge. Learn more at toyota.com

slash grandhighlander. Toyota, let's go places.
Since you're new to H&R Block, we'll look at your returns from the last three years for any money your last guy might have missed. For free.
I could get money back from last year? You could. We'll find any mistakes.
Could have really used that two years ago when I dated that mistake for five months. Don't leave money on the table.
Switch to H&R Block and get a free second look review. Second look is included at no additional cost with the purchase of tax preparation.
Results vary. All tax situations are different.
Fees apply if you have us file an amended return. Do you listen to music at home? And what do you listen to? I do listen to music at home.
and if I had my choice of what we would listen to, it would probably be a lot of Taylor Swift. But I don't have a choice because I have a three and a half year old daughter.
So what I've been listening to a lot of lately is the soundtrack to the movie Cars 2. Oh my God.
When Ben and his daughter listen to those bangers from the Cars soundtrack, their device of choice is not a Sonos. I am not a Sonos household.
However, I have learned that basically everyone in my life belongs to a Sonos household. Tell me about Sonos.
What is the company known for? Sonos is known as a premium home audio equipment company, and it really revolutionized home audio by creating this ecosystem of smart audio products that work seamlessly with each other. And in fact, when I talk to Sonos users, the company that often comes to mind is Apple.
Like, if you're an Apple user, if you have an iPhone, probably you have a MacBook or you have an iPad, and you want to be able to control all of it within that same ecosystem. The idea is you put your Sonos devices on the same Wi-Fi network, and they can talk to each other.
And you can control all your devices with a single app on your phone. And people who own Sonos products don't just own one Sonos product.
The average Sonos household owns three products. So maybe that's a speaker with a soundbar under the TV or a portable speaker that they can bring on the road or headphones.
I mean, there are a lot of people who own like lots of Sonos products because they need speakers in each room of the house. And when your three and a half year old daughter is listening to Cars 2 in her bedroom, you might want to listen to Taylor Swift in another room.
Or you might want to listen to Taylor Swift in all the rooms and then you can also do that. Right.
Sonos has been around since 2002 and it won over hardcore audio lovers with its emphasis on quality sound. On its YouTube channel, there are videos about how Sonos users can really perfect their TV sounds.
With a quick tap in the app, you can make dialogue even clearer. And what the best Sonos speakers for gaming are.
Try Bean to unlock the 3D effect of Dolby Atmos. Sonos products range from a couple hundred bucks to thousands of dollars for some of their sets.
And although the company is a lot smaller than other tech companies that also sell speakers, like Amazon, Sonos was able to carve out a place for itself in the luxury audio space. By the end of March last year, the company was worth over $2 billion.
Then in April, the company announced that it was upgrading its software. The old app was sometimes hard to manage, and the company wanted an update to make it easier for them to release new products.
In a statement, then-CEO Patrick Spence said, quote, After thorough development and testing, we are confident this redesigned app is easier, faster, and better. The new app was released globally on May 7, 2024, as a software update.
But lots of customers had problems with it almost immediately. So what was it exactly that happened? Basically, everybody noticed right away, like the very first day, in part because it was pretty hard not to notice.
Sonos users couldn't use basic features of their speakers. They couldn't access their own audio systems.
It was almost as if these speakers had become like sleekly designed bricks. Like very expensive bricks? Very, very expensive bricks.
And what actually happened with the app kind of depends on the user. Some found that it was like missing essential features of the old app, like the ability to edit playlists on the fly or set alarms for when they should wake up in the morning.
Some people found entire libraries of music were just suddenly inaccessible to them. Speakers that vanished from their audio systems in the middle of a song, that basic promise of being able to control music in a room suddenly wasn't being fulfilled.
And for most Sonos users, regardless of the experience they were having, the product basically just became worse overnight. The tech problem was complicated.
Part of it was that over the years, Sonos had continued to rely on a lot of obsolete code. They'd done a lot of updates, though never a complete overhaul.
And they ran into issues when they tried to bring their software up to date to match their hardware ambitions. Sonos says they looked closely at whether or not to revert back to the old app,

deciding eventually it wasn't viable. But they also struggled to fix the new one.
And then there was the PR problem. At first, the company defended the update, according to a statement published by a tech news outlet.
The chief product officer at the time defended it as courageous to do this, like, because, you know, they were releasing this new app and it would have been easy to just keep going the way they were going, but they felt that this was like a necessary change that they had to make for the future of the company. Soon after the messy rollout, Sonos started releasing additional software updates to try and fix the bugs.
And in July, Spence, the CEO, published a letter of apology. But customers were still mad.
A lot of them still couldn't use their devices the way they wanted. In October, more than four months after the app rolled out, Spence released another statement, this time a video.
For more than 20 years, we have been obsessed with delivering an audio experience that is easy, reliable, and sounds amazing. The video is more than three minutes long, and it's titled, Recommitting to Quality and Customer Experience.
Recently, we rolled out a new app that fell short of this standard. It's been painful for our customers and gut-wrenching for all of us at the company.
Ben says that for a lot of customers, the response was too little, too late. How has all of this impacted Sonos, the company? In a very, very big way.
So the company has said that it has cost at least $100 million in revenue. And the company had to delay two product launches last year as it was dealing with the fallout of this botched

app release and the bungled response to it. So that's $100 million in revenue.

And the company's market cap has plummeted by around $600 million since the app came out.

Sonos released an app that was supposed to be their most extensive app redesign ever.

And it kind of turned into their most expensive app redesign ever. After the break, we asked Sonos directly about the saga.
Your snacking routine can get a little dull. Time for an Oikos Remix or Light & Fit Remix.
Like a crunchy storm of sea salt, praline pretzels, dark chocolate, and butter toffee, showering down into a smooth, creamy yogurt. Enjoy six Remix varieties, three Epic Complete Protein Oikos Remix options, or three craveable light and fit remix options see remixyogurt.com it's time to rewrite the vacation rules with royal caribbean your family can look forward to just about anything break the thrill barrier roaring down the tallest water slide in north america jump into breathtaking jungles and jaw-dropping coves

vibe off the charts on our private island perfect day at coco kay and end the day knowing things are just getting started because this isn't just any vacation this is all the vacations come seek the royal caribbean chips registryamas. Do you remember what you were doing the day of the app rollout? I don't offhand.
No, May 7th, 2024, not, uh... I remember the weeks afterwards with my phone blowing up.
That's Eddie Lazarus. He's the chief legal and strategy officer at Sonos.
Were you hearing from a lot of different people? People you knew? Of course. A lot of us know a lot of people who have Sonos, and some of those people were having problems.
I actually do know that I was in Europe at the time. I think I may even have been on a bike trip.
But I got off my bike and started getting back into it. Eddie has been with Sonos for about six years, and he was in charge of investigating what happened in the aftermath of the app rollout.
You were the one, Eddie, who did the internal audit. Why did it go so badly? If I could sum up in just a few sentences, it would be, we tried to do too much too fast, and we did not apprehend what would happen when we released all this new work into the wild.
We discovered that with all the code that we had changed and all the work we had done, when the system got out into the wild, where you have people with a lot of old equipment and new equipment and mixed old and new equipment and good routers and bad routers and all kinds of other complexities, the performance was just not even close to the standard that we expected ourselves or that our customers expected of us. And we disappointed our customers and we disappointed ourselves.
But I guess that's sort of the question that I think has been on a lot of people's minds and was, especially immediately after, was why? Why do so much all at once? This was something that came up a lot on Reddit and on community forums. You know, why did the company decide to roll out such big changes all at one time rather than do it slowly for small groups of users at a time? We'll never do that way again.
And in retrospect, of course, it was a mistake. But we thought we had a new modern app that was ready to go.
And we wanted to give the benefits of that app to our customers as quickly as we could. And frankly, we should have acted with more humility.
And that's one of the absolute number one lessons learned. And as I said, we won't be doing that again.
Was there a sense that the team or the company underestimated the complexity of what this would take? 100%. Eddie says one reason that Sonos went big with their new app was that they were about to launch new products that would benefit from updated software.
A big one was Sonos' first pair of headphones. How was that product launch tied to the timing of the release of the redesigned app? The headphones came out shortly thereafter, and we wanted the new app out into the marketplace in advance of the headphones.
Now, we delayed the launch a couple of times while we continued to work on the app to get it to a place that we felt it was ready to go.

But again, we misjudged.

And those headphones are a fantastic product.

And the app, unfortunately, has been, up until recently, a bit of a cloud over it.

The new product wasn't enough to boost Sonos.

And the company's total units sold, so across all its products,

were down by 14% in the back half of last year,

compared to the year before.

Talk to me about what Sonos did to fix the problems with the new app over the course of the weeks and months that followed.

Well, we've had 22 new software releases since then, and we had a dedicated team, and still do, that does nothing but think about these performance and reliability issues and how to fix them. Now we're over 90% of the features that we're missing.
It's just been the number one priority of the company. So just to be clear, is the new app fixed? Well, when you say fixed, not every single feature that was in the old app is in the new app.

Almost all of them are.

On many of the performance metrics, we're actually doing much better than the old app did.

But there are a few cases involving a few of the older products where we're still having performance issues.

There are a few other issues out there.

So, again, we have our eye on not only getting to that parity bar, but to go well beyond it. Sonos has tried to be transparent with its efforts to fix the app, holding Q&As on Reddit and the company's community forum, and sharing a Trello or project management board with the public.
But the damage to the company's reputation has been hard to shake. I really think there's only one way to fix reputational damage like that, and that is to show that we're doing the right things by our customers every day.
And we're going to make our software and our experience better than anyone else in the field. And as people see that we're delivering on that, I certainly hope they'll give us another try.
So we just have to win them back through action. Internally, the company has also struggled.
They've had two sets of layoffs since the app update, losing around 300 employees. Then, at the beginning of this year, the CEO, Patrick Spence, stepped down.
Here's our colleague, Ben Cohen, again. They replaced the CEO, Patrick Spence, with a guy named Tom Conrad, who was already on Sonos' board, but he's someone with

extensive experience in product design, software, and music platforms, having been the chief

technology officer of Pandora for 10 years. He's also something of a Sonos geek.
He really,

really cares about the product. In fact, his first day on the job, he wrote this letter to

employees in which he said that he has a Sonos home system.

He watches TV with his Sonos soundbar.

And when his daughter was born,

he brought another kind of Sonos portable speaker

into the delivery room of the hospital.

This is a guy who is such a fan of the company's products

that he even has a tattoo of Sonos headphones.

So like he is permanently inked with his devotion to Sonos, even before he was the CEO of Sonos. And is that passion enough to turn things around? Like what does Tom Conrad need to do to get confidence back in the brand? Well, I think that's like the million dollar question for Sonos right now, right? What lessons can be learned here? The first is that these apps that run our lives, they demand constant improvements, and they can't have any disruptions.
The best updates are the ones that you don't notice, and technology is at its best when it just works, right? When it has that magic, as Steve Jobs used to say. In this case, it just kind of stopped working.

Right.

And I think Sonos has sort of rethought

its software development process throughout this time.

Like, part of the issue here is that lots and lots

and lots of people using Sonos products

felt these changes at the same time.

And if you had rolled them out in smaller chunks,

you could have contained the damage

and fixed these issues before they became issues for everyone.

Before we go, we've got exciting news.

We're planning a live event for the show.

And we want to know what you want to see.

Take our survey by following the link in our show notes and leave your email address to be the first to know when it's happening. That's all for today, Thursday, March 6th.
The Journal is a co-production

of Spotify and The Wall Street Journal. Additional reporting in this episode by Dan Gallagher.

Thanks for listening. See you tomorrow.