Making Sense: How sound becomes hearing

Speaker 1 Most AI coding tools generate sloppy code that doesn't understand your setup. Warp is different.
Warp understands your machine, stack, and code base.

Speaker 1 It's built through the entire software lifecycle, from prompt to production.

Speaker 1 With the powers of a terminal and the interactivity of an IDE, Warp gives you a tight feedback loop with agents so you can prompt, review, edit, and ship production-ready code.

Speaker 1 Trusted by over 600,000 developers, including 56% of the Fortune 500. Try Warp free or unlock Pro for just $5 at warp.dev slash top code.

Speaker 1 Support for this show comes from OnePassword. If you're an IT or security pro, managing devices, identities, and applications can feel overwhelming and risky.

Speaker 1 Trellica by OnePassword helps conquer SaaS sprawl and shadow IT by discovering every app your team uses, managed or not. Take the first step to better security for your team.

Speaker 1 Learn more at onepassword.com slash podcast offer. That's onepassword.com slash podcast offer.
All lowercase.

Speaker 9 For a lot of people, figuring out what you're meant to do with your life is a long, winding process.

Speaker 12 But for some lucky ones, a career path becomes clear in an instant.

Speaker 15 Well, I've always been very interested in music. I spent all my time playing the piano and composing and so on.

Speaker 16 For Diana Deutsch, that moment happened back in the 50s, but it didn't go exactly how she imagined it.

Speaker 15 My music teacher performed at the BBC third program in the mornings. She was playing piano in a trio and I was asked to be a page turner.

Speaker 12 Essentially she'd be turning the pages of the sheet music so her teacher wouldn't have to stop playing.

Speaker 15 So I went up to BBC House and I was all of 16 at the time and very excited about doing this.

Speaker 18 Diana had always dreamed of being a musician, so even just turning pages on the BBC felt like the big time.

Speaker 15 What happened was I turned the first page, no problem. I turned the second page, no problem.
When it came to the third page, unfortunately, my hand jerked and all the pages flew down onto the floor.

Speaker 15 The poor lady had to, while playing the piano with one hand, pick up the pieces with the other.

Speaker 15 Yeah, it was a terrible experience.

Speaker 20 Diana came face to face with her dream and she knew with complete clarity that it wasn't for her.

Speaker 15 It certainly made me realize that being a performing musician was probably not a good idea for me.

Speaker 9 Instead of aiming for a career as a performer, Diana got into researching the psychology of music, particularly how different people perceive sounds.

Speaker 14 And she was one of the first people to study this by generating synthesized tones using enormous mainframe computers.

Speaker 9 One day in 1973, she was experimenting with playing two sequences at the same time.

Speaker 15 And I had no idea what would happen, but I thought it would be interesting to try.

Speaker 28 You can actually hear exactly what Diana heard back then, but only if you're listening on headphones.

Speaker 25 So if you have a pair around, now would be a good time to put them in.

Speaker 15 I started off with a high tone alternating with a low tone in one ear.

Speaker 15 And at the same time, a low tone alternating with a high tone in the other ear.

Speaker 12 High low on one side, low high on the other.

Speaker 15 And what I heard seemed incredible.

Speaker 15 I heard a single high tone in my right ear that alternated with a single low tone in the left ear.

Speaker 22 Both ears were getting high-low sequences, but she wasn't hearing them in both ears.

Speaker 2 She only heard high tones on the right and low tones on the left.

Speaker 15 Just as a kind of knee-jerk reaction, I switched the headphones around

Speaker 15 and it made no difference to what I perceived. The high tones remained in my right ear and the low tones remained in my left ear.

Speaker 19 If you have headphones on, flip them around.

Speaker 13 There's probably no difference.

Speaker 15 I went out into the corridor and pulled in as many people as I could. And by the end of that afternoon, I must have tested, oh, I don't remember how many, but probably dozens of people.

Speaker 15 And most of them heard exactly what I heard.

Speaker 8 Diana literally couldn't believe it.

Speaker 15 I was beside myself. It seemed to me that, you know, I'd entered another universe or I'd gone crazy or something.
It just seemed that the world had just turned upside down.

Speaker 17 I'm Noam Hassenfeld, and this is Making Sense, a new series from Unexplainable about the weird, perplexing, enormous unknowns of our senses.

Speaker 32 We're starting by trying to make some sense of sound.

Speaker 33 What are we actually hearing when we're hearing?

Speaker 34 How much of it is the real world, and how much is constructed in our brain?

Speaker 35 All knowledge must come through the senses. All that we perceive, and all of the awareness of our daily existence.

Speaker 8 Light, double rainbow, oh my god, sound, listen to me, listen to me, Touch.

Speaker 30 Squishes. Odors.
You.

Speaker 8 And tastes.

Speaker 36 What are your thoughts concerning the human senses?

Speaker 35 As meat and wine are nourishment to the body,

Speaker 35 the senses provide nutriment to the soul.

Speaker 30 All that we perceive, see, all the awareness. Hearing, all knowledge must come through the sentences.
I have an incredible sense of touch.

Speaker 30 All that we perceive, tasting, all the awareness. Smelling, all knowledge must come through the sentences.
Doesn't make sense. Sound a weave off.

Speaker 22 Before we get to all the unknowns, let's start with what we do know about sound.

Speaker 37 Sound is rapid changes in air pressure that happen when something is vibrating.

Speaker 12 Matthew Wynn, audiologist, University of Minnesota.

Speaker 37 So you can think of it in the same way that you think of a wave in a pond. None of the water particles move very far.
They just sort of bob up and down, but they set a whole wave into motion.

Speaker 37 And it's like a domino effect moving through space.

Speaker 17 This pressure wave travels through the air.

Speaker 37 And then, you know, a whole chain of events will set into motion in your ear.

Speaker 12 The wave passes through the ear canal.

Speaker 37 The eardrum vibrates back and forth.

Speaker 8 And a few little bones amplify that vibration, sending it deeper toward the cochlea, this spiral-shaped organ in the inner ear that's covered with thousands of hair cells.

Speaker 37 The cochlea is where the sensory cells are that pick up the sound and turn it into something the brain can use.

Speaker 16 Pressure waves become electrical impulses, which are eventually interpreted as sound.

Speaker 37 So this sounds like a long, complicated process, but it's extremely fast. I mean, there's no sense that's faster than hearing.
Your ear can do this whole process thousands of times per second.

Speaker 17 All of that, the pressure waves, the ear vibrations, the transformation to electrical impulses, that's the simple part, the part we know.

Speaker 8 The complicated part is pretty much going to take up the rest of this episode.

Speaker 40 Because there's a difference between the pressure waves that enter our ears and what we actually end up hearing.

Speaker 37 If we actually perceived every different sound that came in, we would be utterly confused.

Speaker 23 Take Matthew's voice, for example.

Speaker 37 Even in the room that I'm in right now, I'm just in a room in my house, there are echoes all around me because anytime you have a flat surface on a table, a wall, a computer screen, anything, the sound will in fact reflect off of it.

Speaker 39 All of these echoes bouncing around should theoretically make sounds really hard to locate in space.

Speaker 37 And so if we hear that and then hear another echo coming from the wall on my right, and then I hear an echo coming off the ceiling and then my table,

Speaker 37 how would I know which direction the sound is coming from?

Speaker 4 It's coming from all directions.

Speaker 32 But our brain has an answer.

Speaker 37 Thankfully, our brain knows

Speaker 37 sounds only come from one direction, and that's the only way the world makes sense.

Speaker 12 In order to function in the real world, our brain makes a guess.

Speaker 37 It perceives that first wave of sound coming in, and then every subsequent reflection of that sound, it's like saying, Okay, I can suppress you, which is why a lot of people aren't even aware that there are echoes because our brain is so good

Speaker 37 at suppressing them.

Speaker 12 Our brain essentially edits our auditory experience.

Speaker 37 The way I like to phrase it is that the brain is being nudged in a direction rather than just straight out reading the world.

Speaker 12 Which is exactly what Diana stumbled across that day in the 70s when she was flipping her headphones back and forth.

Speaker 15 It just seemed that the world had just turned upside down.

Speaker 5 These days, auditory illusions aren't as unheard of as they used to be, but Diana's a big reason why.

Speaker 8 She's She's now a psychology professor at UC San Diego, and she's been using computer-generated sounds to study the brain's editor for decades.

Speaker 2 With that first illusion she discovered, Diana thinks two parts of your brain are disagreeing, the parts that determine pitch and location.

Speaker 16 That's why you hear a high tone on one side and a low tone on the other, even though they're really on both sides.

Speaker 40 And after finding that first illusion, Diana couldn't stop thinking about it.

Speaker 15 Of course, I didn't sleep much that night. This can't be the only illusion that does this kind of thing.

Speaker 8 Diana started wondering whether she could design other illusions to learn more about the brain's internal machinery.

Speaker 15 In the same way as, you know, if a piece of equipment, such as a car, breaks down, you can find out a lot about the way the car works just by fixing what went wrong.

Speaker 8 So she started brainstorming.

Speaker 15 I was sort of half asleep and I was imagining notes jumping around in space and

Speaker 15 by the next morning they had sort of crystallized into what I named the scale illusion.

Speaker 17 The scale illusion.

Speaker 28 Just like before, this illusion consists of two tone sequences, one in each ear.

Speaker 15 So there's one channel alone.

Speaker 8 Some high notes, some low notes.

Speaker 15 And then the other channel alone.

Speaker 29 Some more high notes, some more low notes.

Speaker 15 And then you hear them together again.

Speaker 16 If you're listening on headphones, you're probably hearing all the high notes on one side and all the low notes on the other, even though those notes are actually jumping from left to right.

Speaker 9 That's your brain editing the sounds.

Speaker 27 It's separating them to reflect the way the world usually is.

Speaker 15 In the real world, one would assume that sounds that are in a higher pitched range are coming from one source and sounds in a lower pitched range are coming from another source.

Speaker 28 So that's what the brain assumes is happening here.

Speaker 15 The brain reorganizes the sounds in space in accordance with this interpretation.

Speaker 40 Just like removing echoes, this kind of brain editing would normally help you make sense of the world.

Speaker 8 But Diana's illusion is explicitly designed to fool the brain into making a wrong guess.

Speaker 40 And not everyone's brain makes the same guess.

Speaker 15 Left-handers as a group are likely to be hearing something different from right-handers as a group.

Speaker 11 Right-handers tend to hear high tones on the right side, but for left-handers, it's more complicated.

Speaker 19 They're likelier than other people to hear high tones on the left or in even weirder ways.

Speaker 12 All of this reorganization, the way the brain edits our hearing to help us navigate the real world, it's sometimes called top-down processing.

Speaker 15 Top-down processing occurs when the brain uses expectation, experience, and also various principles of perceptual organization to influence what is perceived.

Speaker 8 Instead of bottom-up processing, which is sensing the world and then having that travel up to the brain, top-down processing means that our brain is influencing how we hear.

Speaker 15 To some extent, our brain is hearing what we are expecting to hear.

Speaker 8 In a sense, a lot of what we perceive isn't actually us hearing sound waves hit our eardrum.

Speaker 9 It's a prediction of what those waves should be.

Speaker 11 To illustrate this, Diana uses something called the mysterious melody.

Speaker 15 This is a well-known tune, but the notes are presented in different octaves.

Speaker 17 For all the non-music folks out there, an octave is basically a standard range of musical notes.

Speaker 9 In this illusion, the notes stay the same, but which range they're played in changes. So instead of playing Do Ray Me in the same range with all the notes next to each other,

Speaker 26 you could play Do Ray Mi with the notes jumping into a different range.

Speaker 16 So Diana takes a well-known tune, doesn't change the melody, just changes the range.

Speaker 15 And the question is: can people recognize this melody?

Speaker 15 And in fact, people can't recognize the melody.

Speaker 14 Now, listen to a simplified version of the same sequence.

Speaker 15 In this case, all the notes are in the same octave.

Speaker 44 Same range.

Speaker 16 You know what it is.

Speaker 15 Yeah, indeed, it's Yankee Doodle.

Speaker 23 And a lot of times when people go back and listen to the scrambled version, they can hear Yankee Doodle in there.

Speaker 28 When you have a frame of reference for what you're hearing, when you have an expectation, it actually changes what you're hearing.

Speaker 38 Illusions like this tend to circulate around the internet every once in a while, like this one where depending on which word you're thinking of, you might be able to hear either laurel or Yanni.

Speaker 30 Laurel.

Speaker 30 Laurel.

Speaker 48 Remember last year when that Laurel versus Yanni thing, everybody's going nuts over? Well, there's a kiddie version of it making the rounds right now.

Speaker 14 This is from Jimmy Kimmel's show.

Speaker 39 And he starts by pulling up a clip from Sesame Street, of all places.

Speaker 30 I move it to follow you. Move the camera.

Speaker 49 Yes, yes, that sounds like an excellent idea. All right.

Speaker 48 And pay attention to this because tell me if you hear Growburgers say one of two things. That sounds like an excellent idea, or that's that's an effing excellent idea.
Are you ready?

Speaker 49 Yes, yes, that sounds like an excellent idea.

Speaker 4 Kim, what did you hear?

Speaker 7 It's a finger.

Speaker 4 You heard that. Yes, I did.

Speaker 48 It's the first time I heard it, I didn't hear a curse word at all. And then the next 12 times I watched it, the F-word was all I heard.

Speaker 20 But just in case you want one more go at it, here's Grover maybe making a lot of parents upset.

Speaker 49 Yes, yes, that sounds like an excellent idea.

Speaker 28 This type of misperception is true to an extent with all our senses.

Speaker 40 We've all seen visual illusions, or you might remember the debate around the dress.

Speaker 13 But Diana eventually found that the various ways our brain edits the world, they're not just due to hard-coded differences, like whether you're right or left-handed.

Speaker 36 Brain editing can vary from person to person based on life experience.

Speaker 25 To prove this, she asked listeners to determine whether a pattern is going up or going down.

Speaker 28 For people who know a bit of music theory, this interval is a tritone, which is exactly half of an octave.

Speaker 9 So to get from note to note, you travel the same distance whether you're going up or down.

Speaker 23 If you don't know that much about music, all you need to know is that this is a particularly ambiguous pattern.

Speaker 34 But Diana does something really interesting in her experiment here.

Speaker 25 She plays the melody in a bunch of registers at the same time.

Speaker 9 So you might have an extra hard time figuring out if it's rising or falling.

Speaker 15 And And sure enough, you get huge differences from one individual to the other. And this is something that really does surprise people.

Speaker 29 I hear it going up.

Speaker 23 And Diana found that other people hear it going up, but some people hear it going down.

Speaker 25 What's truly mind-boggling is that Diana's found that the difference in how two people perceive this pattern, it might come down to where you grew up.

Speaker 14 Believe it or not, when Diana compared two groups, people from southern England and people people from California, she found that the English people tended to hear this pattern as rising,

Speaker 25 whereas the Californians heard that same pattern as falling.

Speaker 21 Diana's hypothesis is that based on where you grow up, you tend to hear different pitches as low or high.

Speaker 15 It has to do with the pitch range of the speech to which you have been most frequently exposed, particularly in childhood.

Speaker 36 So if you hear that first pattern, which goes from the notes D to G sharp as falling,

Speaker 23 you probably hear this second pattern, which goes the exact same distance from the notes A to D sharp as rising, or vice versa.

Speaker 14 But ultimately, the mechanics of all this are still pretty much a mystery. Scientists don't really know how all this brain editing happens.

Speaker 15 I mean, we know that the brain does that, but we don't really know how.

Speaker 19 In a sense, it's almost like we're all listening to a play performed in our heads just for us.

Speaker 38 There's a script, the entire world of pressure waves bouncing around, but how we actually hear it all is up to the performers.

Speaker 46 In so many ways, our brain dictates how we hear the world.

Speaker 17 But even though we don't know exactly how our brain does this, there are times when harnessing that brain magic starts to become a lot more important.

Speaker 30 It was like my hearing was pouring out of my head like water out of a cracked jar.

Speaker 17 Coming up after the break, one man's quest to hear his favorite piece of music again.

Speaker 8 That's next.

Speaker 52 Support for this show comes from Robinhood. Wouldn't it be great to manage your portfolio on one platform?

Speaker 52 With Robinhood, not only can you trade individual stocks and ETFs, you can also seamlessly buy and sell crypto at low costs. Trade all in one place.
Get started now on Robinhood.

Speaker 52 Trading crypto involves significant risk. Crypto trading is offered through an account with Robinhood Crypto LLC.

Speaker 52 Robinhood Crypto is licensed to engage in virtual currency business activity by the New York State Department of Financial Services.

Speaker 52 Crypto held through Robinhood Crypto is not FDIC insured or SIPIC protected. Investing involves risk, including loss of principal.

Speaker 52 Securities trading is offered through an account with Robinhood Financial LLC, member SIPIC, a registered broker dealer.

Speaker 3 As a founder, you're moving fast towards product market fit, your next round, or your first big enterprise deal.

Speaker 3 But with AI accelerating how quickly startups build and ship, security expectations are also coming in faster, and those expectations are higher than ever.

Speaker 3 Getting security and compliance right can unlock growth or stall it if you wait too long.

Speaker 3 Vanta is a trust management platform that helps businesses automate security and compliance across more than 35 frameworks like SOC2, ISO 27001, HIPAA, and more.

Speaker 3 With deep integrations and automated workflows built for fast-moving teams, Vanta gets you audit ready fast and keeps you secure with continuous monitoring as your models, infrastructure, and customers evolve.

Speaker 3 That's why fast-growing startups like Langchain, Ryder, and Cursor have all trusted Vanta to build a scalable compliance foundation from the start.

Speaker 3 Go to Vanta.com slash Fox to save $1,000 today through the Vanta for Startups program and join over 10,000 ambitious companies already scaling with Vanta.

Speaker 3 That's vanta.com slash box to save $1,000 for a limited time.

Speaker 3 Move the camera! Yes! Yes, that's a an excellent idea!

Speaker 29 Unexplainable, we're back.

Speaker 23 And we've been talking about the mysterious way our brain filters, edits, edits, and even reconstructs the world that we hear.

Speaker 16 For some people, this kind of brain magic can be interesting to highlight as a party trick, but for others, it can be way more important.

Speaker 30 Okay, testing one, two, three, testing.

Speaker 11 This is Mike Chorist.

Speaker 30 So it's like you take the word chorus just to add a T at the end.

Speaker 12 Mike's a science writer who was born with severe hearing loss, but he was able to use hearing aids.

Speaker 19 And starting from when he was 15, he became obsessed with Bolero, the famous piece by Maurice Revelle.

Speaker 30 It was this riotous melange

Speaker 30 with such a fascinating drumbeat underneath it all. It really thrilled me and fascinated me.

Speaker 5 He particularly loved the way the melody would gradually evolve over the course of the piece.

Speaker 30 Each repetition is on a higher level, it's louder, the resonance is deeper, and it's only reached a climax. So it's a very auditorily overwhelming piece of music.

Speaker 7 He would listen to Bolero over and over and over.

Speaker 30 It was kind of my piece of music that I would come to again and again and again to test out new hearing aids. So it's always been an auditory touchstone for me.

Speaker 12 And then one day in 2001, the limited hearing he still had started disappearing.

Speaker 30 I was standing outside a rent a car,

Speaker 30 and I suddenly thought that my batteries had died. My hearing aid batteries.

Speaker 5 Suddenly, the traffic on a nearby highway started sounding different.

Speaker 30 It was just that sound that you associate with cars going by. You know, vroom, vroom, vroom.

Speaker 30 But all of a sudden, it sounded more like,

Speaker 30 whoo,

Speaker 30 who.

Speaker 30 As if somebody had dumped a whole whole bunch of cotton onto the highway.

Speaker 12 Pretty soon, Mike found out he was quickly losing what was left of his hearing.

Speaker 30 It was like my hearing was pouring out of my head like water out of a cracked jar.

Speaker 30 So after about four hours after that initial realization, I was essentially completely deaf. It was just such a shocking experience.

Speaker 23 But Mike was eligible to receive a cochlear implant.

Speaker 21 It's a surgically implanted device that can offer a form of hearing in some deaf people.

Speaker 44 Many people in the deaf community prefer to communicate using sign language or lip reading rather than using a cochlear implant.

Speaker 45 But for some people, especially people who've lost their hearing later in life and want to continue using their native spoken language, cochlear implants can be helpful tools.

Speaker 37 The cochlea is this tiny spiral-shaped organ inside your head. And a cochlear implant is a string of electrodes that's carefully inserted inside that spiral organ.

Speaker 11 This is is Matthew again, the audiologist who actually works with cochlear implant users to help them understand their experience.

Speaker 37 There's this external part that looks like a hearing aid, but is not a hearing aid.

Speaker 37 It's a microphone and a computer that analyzes the sound and sends instructions to those electrodes that are inside the ear.

Speaker 16 The implant essentially bypasses a lot of the ear.

Speaker 23 It directly activates the cochlea, which then passes an electric signal onto the brain.

Speaker 14 But cochlear implants don't just reproduce normal hearing.

Speaker 44 Mike says that reducing sound to digital ones and zeros and beaming them directly into your brain, it can sound strange.

Speaker 30 It was shocking. It's not at all what I expected.

Speaker 9 When Mike's implant was turned on, the first thing he did was listen to his own voice.

Speaker 30 And my voice sounded really weirdly high-pitched. I almost sounded like,

Speaker 30 you know, it was that kind of sound.

Speaker 30 It was like listening to a demented mouse.

Speaker 11 Matthew actually gave me a program he uses as an audiologist to simulate various types of cochlear implant sounds.

Speaker 5 So here's a general idea of what it might have sounded like to Mike.

Speaker 30 It was very upsetting. I thought the world would sound pretty much like I heard with hearing aids, just fuzzier.
I was completely unprepared for the huge difference in pitches.

Speaker 16 Because of the way the implants are designed, they tend to make everything seem a bit high-pitched.

Speaker 30 So when you send a signal to any part of the cochlear implant, the brain will interpret that as a high-pitched sound, even if it's a low pitch.

Speaker 36 Which is why everything can sound all mousy.

Speaker 30 But the interesting thing is, within just a day or two, I started to hear low pitches again.

Speaker 30 And part of that was my brain adapting to it. My brain was saying, okay, this is my voice.
I know it's supposed to be a low pitch.

Speaker 30 However, right now I'm hearing it as a high pitch. Never mind that.
Because I know it's a low pitch, I'm going to interpret it as a low pitch.

Speaker 38 Essentially, Mike's brain was editing the world for him.

Speaker 30 So very quickly, my brain started figuring out: okay, the world sounds really weird, but I'm going to try to fit that into my preconception into what the world is supposed to sound like.

Speaker 12 He was taking command of his own top-down processing.

Speaker 30 So within hours, I stopped sounding like making most of myself.

Speaker 2 And then Mike started training.

Speaker 30 I got the audio books of the Winnie the Pooh books. And I remember the first time I put the tape into the cassette player and played Winnie the Pooh and some B's.
I think that's the one.

Speaker 30 I couldn't make it out at all. It was just complete gibberish.

Speaker 13 But he also had the physical book. So he read along with the tape.

Speaker 30 So I was able to start matching up the weird input that I was getting with the words on the page that told me what that input meant.

Speaker 53 What about a story?

Speaker 8 Said Christopher Robin.

Speaker 53 Could you very sweetly tell Winnie the Pooh one?

Speaker 30 This is what the S sounds like. This is what the phoneme poo sounds like.

Speaker 41 Winnie the Pooh.

Speaker 30 So it is a process of remapping.

Speaker 9 According to Matthew, this process of brain remapping is a pretty normal experience for cochlear implant users.

Speaker 37 Any good audiologist would say to someone, if they're thinking about a cochlear implant, that when you first get it and it first is activated, you probably won't understand much at all.

Speaker 37 But over the first six months, maybe the first year, your brain learns to reorganize how it associates sound with meaning.

Speaker 12 Training's more accessible these days.

Speaker 19 It's certainly not as DIY as it was for Mike 20 years ago.

Speaker 51 But this kind of improvement can still be hard to believe.

Speaker 37 A lot of the people that I've worked with will say, now when I listen to my spouse, it sounds like her voice, which baffles all of us who work in this field, because if you look at how the ear is being activated, there's no explanation.

Speaker 37 I mean, not to be too on the nose, but it's unexplainable, right? So there's no way that that could possibly be true. And yet a lot of people say it.

Speaker 12 Tweaking settings on the implant does make it work better, but that doesn't account for most of this incredible improvement.

Speaker 37 A lot of the success of the cochlear implant is really a testament to how strong the brain is working, rather than a reflection of the high quality of the sound input.

Speaker 38 Our brains have an almost uncanny ability to predict language and fill in gaps, even when we hear something muffled or distorted.

Speaker 45 But while cochlear implants work pretty well for speech, They don't work nearly as well for music.

Speaker 25 Music is just a much more complicated kind of sound.

Speaker 23 You need to distinguish melodies and harmonies and textures and most fundamentally, pitches.

Speaker 16 And an implant only has a small number of electrodes.

Speaker 37 You have to simplify all the frequencies and you can think of it as like pixelating the sound.

Speaker 50 Making this even harder, because the cochlea is filled with fluid, it's hard to use electrical pulses to stimulate the exact part that codes for the right frequency.

Speaker 24 Instead, the pulses kind of spread out around the part that codes for that frequency.

Speaker 37 Let me make an analogy. Suppose you're playing a note on the piano.
You can be really careful and hit the exact key you want, or you can be kind of crude and put your whole hand down on the piano.

Speaker 37 Like you're going to be in the right ballpark of the note, but you're not going to hit the exact note very clearly. So a cochlear implant is more like putting your whole hand down on the note.

Speaker 37 It's not a very precise frequency you're hearing.

Speaker 38 When you take all of this into account, translating music with a cochlear implant can seem almost impossible.

Speaker 37 The current design of cochlear implants isn't set up really for music. It's set up to understand speech.

Speaker 30 But I'm wanting my bolaro back.

Speaker 9 Even though Mike's brain had learned how to edit those high-pitched tinny sounds to understand speech, music still wasn't the same.

Speaker 30 It just sounded awful. awful.
I'm like, oh my god, you know.

Speaker 30 It was really shocking because I was like, even if it gets twice as good as this, it's still going to be awful.

Speaker 30 Even if it gets three times as this, it's still going to be awful. It was really bad.

Speaker 34 Mike upgraded the hardware of his cochlear implant.

Speaker 42 He upgraded the software.

Speaker 23 He even volunteered as a guinea pig for some tests on new equipment.

Speaker 30 So I would put on a side of headphones. I would hear the set of beeps and boops.

Speaker 30 I'm like, okay, which song is that?

Speaker 30 I was like, I don't know.

Speaker 30 It's like, could anybody know?

Speaker 30 And for me, this was a very deeply frustrating kind of experiment because I know Twinkle, Twinkle, Null Star. I was like, that doesn't sound like Twinkle, Twinkle, Null Star to me.

Speaker 30 How could this sound like Twinkle, Twinkle, Null Star to anybody else?

Speaker 17 Researchers I spoke to told me that some cochlear implant users just don't enjoy music that much.

Speaker 6 It's certainly harder to get used to than speech.

Speaker 32 And because patients are often told to focus more on improving listening to speech, music can get left by the wayside.

Speaker 32 But appreciating music through an implant can sometimes be presented as an insurmountable obstacle.

Speaker 12 You can see this in the movie The Sound of Metal, where a musician gets a cochlear implant after losing his hearing.

Speaker 32 and then goes to this performance, listening to the the song you're hearing right now.

Speaker 12 In this scene, the movie shows what other people at the performance hear, and then it gradually shifts perspectives to highlight what the main character hears through his cochlear implant.

Speaker 47 The performance is so upsetting for the main character that he ultimately takes his processor off.

Speaker 47 He essentially decides not to use his implant anymore.

Speaker 40 You can find a lot of simulations online like this.

Speaker 12 So I asked Mike if these kind of simulations, or even ones like the simulations I created of a distorted voice or a distorted bolero for this episode, if they seem like accurate representations of what music sounds like through an implant.

Speaker 30 I think you have to be extremely careful when listening to these simulations because

Speaker 30 basically what those simulations are telling you is

Speaker 30 this is what

Speaker 30 the software is giving to the user. Okay.

Speaker 30 That's not the same thing as what the user hears. These are two very different things.

Speaker 30 You know, when I listen to these simulations, and I have listened to them, it does sound a lot like what I heard on day one.

Speaker 30 It does not sound like what I hear in year 20.

Speaker 54 For Mike, this was a combination of training himself with careful listening, but also tweaking the settings of the implant.

Speaker 54 Because with a lot of practice and effort and time, the experience of listening to music can improve.

Speaker 30 Yeah, I would listen to music over and over again,

Speaker 30 and I would try tweaking different settings. And I would go to my audiologist and I would say,

Speaker 30 these pictures sound really fuzzy to me. Can you do something about that?

Speaker 30 And so she would tweak how much electricity went to different electrodes. And so this was an iterative process that went on and is still going on.

Speaker 31 After years of upgrades, tweaks, training, Mike's noticed some real improvement, but not for all music.

Speaker 30 Most of the piece of music that I enjoy is music that I heard with hearing aids. It's familiar to me.

Speaker 16 Mike does listen to some new music, but preferring familiar music, it's a pattern that Matthew notices with his patients too.

Speaker 37 And I think it's a testament to the brain filling in those gaps, conjuring the memory of what the sound quality should be.

Speaker 37 The implant sort of gives you just enough that the brain can put together the whole puzzle.

Speaker 31 And of course, Mike is listening to Bolero again.

Speaker 30 Well, it sounds good. I really enjoy it.
But there are things that I know that I'm missing.

Speaker 30 I know that I'm still not getting some of that intensity and the purity where the music is reaching for a crescendo in each of its iterations. So I know I'm missing that.

Speaker 31 In a sense, Bolero is so familiar, it's almost like language for Mike.

Speaker 30 Bolero sounds really good to me because I know exactly what it's supposed to sound like.

Speaker 31 This new Bolero is certainly different from the version he remembers, but Mike loves the new version.

Speaker 30 Even though the input I'm getting of Bolero is incomplete and I can hear that it's incomplete, it is still a source of pleasure to me.

Speaker 41 Ultimately, we don't really know exactly how our brain is able to do this.

Speaker 5 It can almost feel like magic, how it filters out echoes, how it shifts high tones to one ear and low tones to the other, how it can take a tinny, noisy input and rebuild a new version of bolero.

Speaker 15 We do this very complex calculation, but I don't think that we really know exactly how it's done.

Speaker 47 Psychologist Diana Deutsch again.

Speaker 15 There are an awful lot of things about our hearing that we don't understand,

Speaker 15 and what we hear is often quite different from what in point of fact is being presented.

Speaker 12 But we do know that the brain is constantly editing, shaping and building the world that we hear.

Speaker 41 Our brain, our life experience, our familiarity with a piece of music, it all shapes how we hear and what we hear, which raises a pretty fundamental question.

Speaker 15 When an orchestra performs a symphony, what is the real music?

Speaker 16 Is it in the mind of the composer?

Speaker 15 Or is it in the mind of the conductor who has worked long hours to shape the orchestral performance?

Speaker 12 Is it in the mind of someone in the audience who's never heard it before and doesn't know what to expect?

Speaker 15 And the answer is surely that there's no one real version of the music, but many. And each one is shaped by the knowledge and expectations that listeners bring to their experiences.

Speaker 12 The idea that to a very real extent our brains conjure different individual realities inside our heads.

Speaker 16 On the one hand, it's a clear reminder to be humble and not just for hearing.

Speaker 6 No matter how certain we are, what we perceive isn't unfiltered reality.

Speaker 39 So it's worth questioning ourselves at our most stubborn moments.

Speaker 7 At the same time though, how cool are brains?

Speaker 7 I know they're this perfect reminder of our own subjectivity and humility, but I also just can't get over the fact that our brain puts on this fireworks show every day.

Speaker 47 and that a lot of people using a cochlear implant can tap into this almost magic ability to translate a few electrodes into this new, emotionally satisfying experience without scientists really knowing how the whole thing works.

Speaker 47 There's so much we still don't understand about the brain and how it tries to make sense of the world, and it just makes me that much more excited for everything we're going to learn along the way.

Speaker 47 This is just the first episode of our Making Sense series. Next week, touch and its evil twin, pain.

Speaker 53 Think of yourself if you have a toothache or if you have a problem, if someone holds your hand or someone pats your back or give you a hug, that relieves actually. Gentle human touch can be very good.

Speaker 47 After next week, we'll be talking about more perplexing sense mysteries like how scientists still don't really know how smell works, how many tastes there could be, why some people can't see images in their heads, and even a sixth sense.

Speaker 47 This episode was edited by Catherine Wells, Meredith Hodenat, and Brian Resnick. It was produced and scored by me, Noam Hasenfeld.

Speaker 47 Christian Ayala handled the mixing and sound design with an ear from Afim Shapiro. Richard Seema checked the facts.
Tori Dominguez is our audio fellow. Manding Wynn is keeping things sunny.

Speaker 12 And Bird Pinkerton is dreaming of bioluminescence.

Speaker 47 If you want to check out more about Diana Deutsch and auditory illusions, we've got a link in our show description where you can find more illusions to listen to and a ton of info about the illusions she's discovered.

Speaker 47 To read more about some of the topics we cover on our show or to find episode transcripts, check out our site at vox.com unexplainable.

Speaker 47 And if you have thoughts about the show, you can always email us at unexplainable at vox.com. Or you could leave us a review or a rating, which we would love to.

Speaker 47 Unexplainable is part of the Vox Media Podcast Network, and we'll be back in touch with episode two of our Sense series next week.

Speaker 55 This month on Explain It To Me, we're talking about all things wellness. We spend nearly $2 trillion on things that are supposed to make us well.

Speaker 55 Collagen smoothies and cold plunges, Pilates classes, and fitness trackers. But what does it actually mean to be well? Why do we want that so badly?

Speaker 55 And is all this money really making us healthier and happier? That's this month on Explain It To Me, presented by Pureleaf.

Speaker 56 Support for the show comes from Mercury. What if banking did more?

Speaker 56 Because to you, it's more than an invoice. It's your hard work becoming revenue.
It's more than a wire. It's payroll for your team.
It's more than a deposit. It's landing your fundraise.

Speaker 56 The truth is, banking can do more. Mercury brings all the ways you use money into a single product that feels extraordinary to use.

Speaker 56 Visit mercury.com to join over 200,000 entrepreneurs who use Mercury to do more for their business. Mercury, banking that does more.