How do you turn facial expressions into music?

This BBC podcast is supported by ads outside the UK.

Ranked number one in innovation 10 consecutive years, Arizona State University isn't just ahead of the curve, it's creating new paths to success.

Learn from notable clinical and research faculty online.

That's a degree better.

Explore programs at asuonline.asu.edu.

Most home fire and carbon monoxide fatalities are preventable with the right safety products, including smoke and carbon monoxide alarms that can alert you when a hazard has been detected.

Teach kids that when they hear beeps at last, they need to get out fast.

Join KIDDA in highlighting the importance of fire and carbon monoxide safety preparedness in homes across the country so our families and especially our children can always feel safe.

To learn more, get involved, and help us spread the word about the importance of fire and carbon monoxide readiness.

Visit causeforalarm.org.

BBC Sounds, Music, Radio, Podcasts.

Hello, and welcome to BBC Inside Science, a programme first broadcast on Thursday, the 3rd of July, 2025.

I'm Victoria Gill, and you join me on a very hot street in central London because I am looking at the gleaming white columns of the Royal Society on Carlton House Terrace.

This week, the UK's world-famous scientific academy opens its doors for its annual Summer Science Exhibition, an event that showcases the latest research from across the UK and invites the public in.

And with me to explore it is science journalist, broadcaster, and fellow seeker of free exhibitions that provide an escape from this terrifying weather, Caroline Steele.

Hello Caroline.

How are you?

Hi, good.

Thank you.

Thanks for having me on.

It's an absolute pleasure.

There's a lot to explore, so I'm glad to have a partner in crime.

This exhibition has a very long history, doesn't it?

But the public weren't always invited in.

Yeah, so the very first Royal Society Summer Science Exhibition was back in 1778.

Apparently in 1896, visitors got to x-ray their hands, so my hopes are high.

But in all seriousness, it'll be interesting to see what's on display this year, because it's been a difficult year for science, which is something the Royal Society has acknowledged.

Earlier this year, they released a statement saying that science is under threat because of huge cuts and a rise of misinformation.

So it'll be interesting to see what they've chosen to sort of showcase to that year.

Yeah, so and why do you think amid all of those risks and the Royal Society coming out and saying that science is under threat from

all of the challenges that you cited there, why is it important to open the doors and invite people in, do you think?

Why is this something that the Royal Society showcases?

Well, it's going to be great to have scientists and the general public in the same room.

Although, it's worth saying, you know, thousands of people will pass through these doors this week, but it is only on in London, so it's not something everyone is going to get the chance to experience.

Yeah, that's true, and hopefully, we'll give you a little snippet of that experience in audio form.

So let's go and have an explore.

Let's do it.

We are starting our tour in the quantum zone.

There is an entire zone for quantum at this exhibition because the UN has declared this year, 2025, the International Year of Quantum, celebrating 100 years of quantum science and technology since groundbreaking work was published by physicists like Erwin Schrödinger and Werner Heisenberg.

I'm here with Daisy Shearer from the National Quantum Computing Centre, who is the person who has put this whole zone together.

Hello, Daisy.

Hello.

Thank you for joining us.

Why does quantum have an entire zone at the exhibition this year?

Well as you said it's all about the International Year of Quantum Science and Technology so we've come together to bring everybody in well most people in quantum in the UK to celebrate their science and showcase the amazing collaboration that we do nationally.

Quantum can be a bit baffling can't it and a bit sort of hidden inside technology that we're you know coming to depend on or that we do depend on.

How are you showing it and showing off how just important it is?

Absolutely.

So one of the amazing things about quantum is that we all use these quantum mechanical effects to build different technologies.

So we're showcasing some of those effects like entanglement that you may have heard of and optics as well.

And then we're showing how we can apply this to different technologies.

So we've got quantum computing, quantum sensing, quantum timing, quantum imaging and quantum communications as well.

What do you want people to take away from this?

Excitement.

I would love to see young people being able to see themselves in quantum careers, but also just to help people demystify quantum because it has a bit of a reputation for being quite complex and we're here to showcase that you know there's all this awesome stuff happening and everybody can get involved.

Complex but exciting and useful.

Absolutely.

And I understand there's some quantum music and since we're an audio programme I think I think we might go and investigate that.

Thank you Daisy.

Clelia Altamonte is a PhD student at King's College London who has been experimenting with how to create music from quantum circuits.

Hi Clelia.

Hi.

Lovely to to meet you.

I'm intrigued.

First of all, though, we're going to have to start with the basics for me, I'm afraid.

What is a quantum circuit?

So, I guess we might have to start from the difference between a bit and a qubit.

So, a bit is the classical unit of information, so it can be either zero or one.

And bits are used in classical computers, for example, in binary codes.

The qubit is the quantum version of that.

So, it is like a bit that obeys the rules of quantum mechanics.

And because of that, it has some extra properties.

For example it can be neither in a state of zero or one but in a state that is in between and we call that a superposition state.

Right.

Or if we bring together more than one qubit they can be entangled with one another which means that they are not independent of one another anymore and we can get correlations that are beyond what would be classically possible.

Those quantum properties not having to be in a state of zero or one

and the quantum concepts of entanglement, that's what makes quantum circuitry so powerful, more powerful than superficial.

Yes, that's right.

And in fact, many quantum algorithms use these properties of entanglement and superposition.

And because of that, some can exceed what is possible classically.

So where did the idea come from to turn quantum circuits into music?

So the idea first came to me as I was working with my supervisor.

And we were working on a quantum circuit.

He noticed that it looked very much like a music score.

Say lines and symbols,

these changes, that's right.

And since that time, every time I've spoken to people about quantum circuits, I've always used this analogy.

And people told me that they found it quite useful in order to quickly understand how quantum circuits work.

And then a couple of months ago, I was at a concert and I was really enjoying the music and really felt like I wanted to play an instrument.

And I've not been playing in a long time, even though when I was little, I actually learned how to play the piano and the guitar.

The easiest way for me to get back to music was trying to put to the test that music analogy with the quantum algorithms.

Right, so the easiest way for you to get back into making music was to quantum circuits.

Yes, simple music.

That's right.

What's your first example?

Take us through.

So we're going to start with quite a simple one.

The easiest one is a single qubit first put in a superposition state and then we perform some rotations on it and it sounds like this.

That does sound simple.

So what are we hearing there?

There's a chord, we kind of have two different notes and they're changing in relation to each other.

So the way we mapped the state of the qubit to music notes is through the block sphere.

So it is possible to represent the state of a qubit as a point on a sphere.

So, we can have two angles representing that state, and each of these two angles was mapped to music notes on a different octave.

And in this case, because we have only a single qubit, we only have one instrument.

The next example is we have first entangled the two qubits.

Okay, so now we have what Einstein called spooky action at a distance.

Yes.

We have entangled qubits.

Okay, let's have a listen to that.

Entanglement's quite harmonious.

It is.

When they're not independent, when they're related to each other, there's that sort of

harmony.

Fascinating.

I feel like I'm hearing quantum in a way that's clarifying it for me.

It's not something that I'm doing.

Yeah, that's fantastic.

Is there anything else that you would like to play for us that you think is particularly melodic quantum text?

I'm not sure it is melodic, but I think it sounds interesting because the next thing that we did was to try and see how some very famous quantum circuits and quantum algorithms sound like.

For example, this is Grower's search algorithm.

It is used to search an unstructured database.

It feels a bit dark, but I can show you.

This is quite a scary quantum algorithm there.

Where can we download the quantum album then?

Are you going to be releasing it anytime soon?

Yes, so there is a GitHub website with a theory about how to turn quantum algorithms into music, and also the code is available.

Listeners can make their own quantum music.

Yeah.

We'll have to put that link on the programme page.

Clearly, this has been fascinating.

Thank you so much for this audio tour of your quantum research.

Thanks.

Caroline, that music is quite a fun way of visualizing, wrapping your head around quantum circuitry, isn't it?

Yeah, because it can be really complicated, sticky math.

So to be able to kind of listen to it, like hear what entanglement could sound like, I think it's really helpful and good fun.

Yeah.

Yeah.

I feel almost like I understand it now.

There's such a variety of different research and different technology on display, and it's amazing what you can try out here and really get hands-on.

Speaking of which, would you like to find out a little bit more about spider and elephant sensors?

Ooh yeah.

Well I know just where to go, just the woman.

I'm Dr.

Beth Mortimer.

I am a Royal Society University Research Fellow and I work at the Department of Biology at the University of Oxford.

And your lab is the Animal Vibration Lab.

That's right.

I did think very hard before picking that name.

I've gone for that because we don't just study one type of animal, but we study a range of animals from spiders and elephants.

And what they all have in common is that they use vibrations through materials for information.

Explain that to me.

What does that actually mean?

Like if you give me the example of elephants, how are they using vibrations?

Yeah, sure.

So when you think of vibrations, you might more commonly think about hearing.

And vibration sensing is linked to hearing, but it is slightly different.

So sound usually travels through air, and usually, we would say hearing on ear is linked to vibrations through air.

But what we're studying is how they're able to use vibrations that move along surfaces or through materials rather than through air.

So, different animals do it in different ways, but essentially, usually the surface that they're standing on, they're using how that moves to gain information about their environment, but also information from other animals as well.

Picking up vibrations through their feet?

That's right.

So, in the case of the spiders, they've got thousands of sensors that are distributed throughout their legs.

And essentially, when whatever they're standing on vibrates, that moves their exoskeleton and then they detect that through these sensors.

So, we're interested in studying spiders because if we can work out how they're able to not only tell what is vibrating but where it might be, and understanding what role their posture or their body shape actually has in that, then we might be able to design robots that have embedded sensors in them as well that can take information from the environment.

Spiders, I guess, it makes sense to me that their world is quite vibration-filled.

Their webs vibrate, they have multiple legs and lots of places for sensors on that body plan.

What about elephants?

Like, how are they using vibrations?

In terms of how they detect it, effectively, your ear can detect vibrations that travel through your bone, not just vibrations through the air.

So, this is how we think that elephants might be detecting vibrations through the ground.

The reason this is important is because the elephants actually generate ground-based vibrations when they vocalize.

So, when they communicate using rumble vocalizations, which is one type of vocalization made by an elephant, it vibrates the air, but it also vibrates the ground.

So, yeah, we know that they can communicate using just seismic vibrations.

So, is that sort of like the way very, very bassy music will kind of move a wall or sort of how a speaker works?

Like there's a physical vibration from that deep, deep rumble?

Yeah, that's right.

So it's basically elephants are so large, they've got such a big diaphragm, they're such low frequency.

All of that together means they're basically bombarding the ground with this very, very low frequency, loud vibration that causes the ground to vibrate.

And then they're able to transmit information just from ground vibration.

So without any airborne vibration, they're able to detect and respond to elephant rumbles in a way that shows they transfer information between each other.

So, Beth, how are you bringing that science to life in your exhibit?

So, one of the main things we're very proud of for the exhibit is that we've got a multi-sensory experience.

So, you can go inside a pod, but it's just a fancy name for a gazebo, really.

And there's a vibrating floor where we're going to be playing the real vibrations that we've recorded from both spiders and elephants, either in the lab or in their natural environment.

So, because we're not very good with our own seismic sense, to kind of aid people in interpreting that, we've basically got a visual representation of the seismic vibration, as well as an audible version as well.

And we have a very enthusiastic Caroline Steele who is keen to give this vibration gazebo a try.

Is that all right?

Could you show me the vibrating gazebo?

Absolutely, come over here for the multi-sensory experience.

It's a large black gazebo with a black floor and a huge screen across one back wall.

So how does this work?

That's right.

So what we'd like you to do is if you want to get the full experience, then take off your shoes

because you do have vibration sensors embedded in your skin which you can investigate through this experience.

So this is a vibrating floor and you're going to feel, see and hear the vibrations that are made by spiders and elephants.

Okay, shoes off.

Can I keep my socks on?

You can keep your socks on, yeah.

Great.

Okay, so I'm going to step up onto the black floor.

Okay, there you go.

Right, are you ready?

Yeah, I think so.

Prepare to experience the seismic world.

Cupianius spiders, found in South and Central America, are nocturnal predators.

With limited vision, they largely hunt by feel.

The floor's vibrating.

In the Kenyan savannah, researchers have buried seismic sensors just under the ground surface.

A low-frequency rumble rolls across the ground as an elephant calls out to its family.

Whoa!

My face is vibrating.

That's a really big rumble.

It's not a sound, but a seismic signal.

A message.

We're here and we belong together.

Aww.

That was so cool.

There you go.

So that's the short version of the multi-sensory experience.

So how did you find it?

I absolutely loved it.

I mean, I just, I kind of can't believe it's almost like a whole extra sense that I feel like I'm not making the most of that these spiders and elephants have.

Thank you so much.

That was a lot of fun.

You're welcome.

Don't forget your shoes.

Ranked number one in innovation 10 consecutive years, Arizona State University isn't just ahead of the curve.

It's creating new paths to success.

Learn Learn from notable clinical and research faculty.

Online, that's a degree better.

Explore programs at asuonline.asu.edu.

Most home fire and carbon monoxide fatalities are preventable with the right safety products, including smoke and carbon monoxide alarms that can alert you when a hazard has been detected.

Teach kids that when they hear beeps that last, they need to get out fast.

Join KIDA in highlighting the importance of fire and carbon monoxide safety preparedness in homes across the country so our families, and especially our children, can always feel safe.

To learn more, get involved and help us spread the word about the importance of fire and carbon monoxide readiness, visit causeforalarm.org.

You are listening to BBC Inside Science with me, Victoria Gill.

And this week we're exploring the Royal Society Summer Science Exhibition.

Now from understanding animal senses to using animals as sensors.

Caroline, why is this particular exhibit that seems to be all about water and tiny creatures caught caught your eye?

Yeah, well I'm a really keen open water swimmer and I'm quite worried about the state of our rivers and lakes and I'm hoping there's a tiny creature here that might tell us a bit more.

Let's go and find out.

I'm Dr.

Casey Riley from the University of Birmingham.

I'm an environmental scientist and my research specialises in freshwater pollution and ecotoxicology.

Just describe the exhibition and particularly the little tiny creatures that we've seen on the screens and in the kind of microscopic images.

Yeah, so a lot of my research works with Daphnia.

So they're a small freshwater crustacean.

They're very similar to kind of like sea monkeys.

They're super useful in freshwater environments because we can use them as an indicator of good water health.

And they've been used so much for research, we have a really good baseline to understand and rank environmental hazards using them.

How do you do that?

What are you looking for in the Daphnia?

Is it sort of presence, absence, or do they give you all the clues?

So the most basic test that we can do in terms of ranking their performance is when they stop moving, so immobilization, and we use chemicals and like vary the doses and then see and observe the effects.

So, when they stop moving, that's like known as the effect concentration.

Daphnia tend to need good levels of oxygen in the water and things like that, so it's a good marker that the ecosystem is healthy enough to support something so sensitive.

Can you see them, or are they so small that you don't even notice?

No, you can see them.

I'll show you.

Really?

Yeah, so a rough markathon would be it's a fully grown Daphnia magna, which is one of the larger species, would be about a third of the size of your little fingernail if you like imagine it like that.

What have you got in your exhibition?

How are you kind of presenting these teeny tiny little pollution detectors to the public?

Often we look at like the bigger animals, so we might look at like turtles and things like that.

We don't often look at them microscopic, so we're trying to really elevate the importance of like these much smaller species.

So to do that, we've got some Daphnia that have eaten some fluorescent particles.

So these are fluorescent plastics.

So it'll be really good to kind of highlight the different tools that we can use in environmental science.

So like the fluorescent stains we can use to assess some of these growing problems like plastic pollution.

And you can see inside their bodies.

Yeah, so they're completely transparent, which is super cool for a test organism.

So we can see how much is in their gut, how long it's in their gut.

Next time I swim, I'm going to be looking out for these tiny flea-like things in the water.

Maybe we can have a look at them now under the microscope.

Can we take a look at the tip?

Can we take a look?

Yes, please.

That'd be great.

I always get slightly excited when I see a microscope.

It's like, ah, science is out there.

It's like the symbol of science, isn't it?

Hopefully, you'll be able to see a Daphnia under the microscope, full of plastic.

Okay, so these are the Daphnia that have eaten the brightly coloured plastic.

Yeah, and we have literally fed them 10 minutes ago.

Ooh, okay, so I see a background that looks to me like space, lots of sort of different specks of light, and in the middle, there is...

Oh,

I think a Daphnia.

Oh, yeah, it's moving.

Very cute.

Really reminds me of the sea monkeys I had as pets, as a kid.

And what do they eat when you're not feeding them bits of plastic?

Algae.

Rice.

Yeah, so just normal like the green algae that you'd find in water.

And so if they're healthy algae munching Daphnia they'd appear green because you could see into yeah we have one just on the next microscope which is one of the things.

When you control

Caroline's straight in there with the microscope.

Ooh ah.

Ooh ah.

Those are really good noises.

It almost looks like a speck of algae that is moving because it is so algae coloured.

What do you want people to take away from your exhibition and these tiny little microscopic creatures?

I really hope that it gets people invested in the work and shows them that we are kind of working towards solutions, but also I hope it highlights the scientific process for some of the things so that when people come to advocate for their spaces, which I really encourage people to do, that they have the rationale and stuff to be able to like map their wants and desires for their spaces.

Why do you think it's important that people grasp that if they care about their waterway, whether they swim or they just kind of enjoy the outdoors?

So when we're saying, like, oh, we can only prioritise one site and we need to decide what's the most polluted, like, what do we mean by the most polluted?

Do we mean the one that has the most chemical pollution, which is leading to like harmful algal blooms and things like that?

Or do we mean the one that has the most plastic in it, which we don't really understand the consequences fully of yet?

Katie, thank you so much.

This is really interesting, and good luck with the rest of the exhibition.

Thank you.

Well, I have to admit, Caroline, I found the quantum music that we listened to a little bit earlier a tad unsettling.

Yeah, it was a bit sort of spooky, apart from the entanglement part.

Yeah, so we want to find something maybe a little more melodic to round off our scientific tour.

And we've come to an exhibition that is all about using technology to make it possible for people with disabilities to make music.

I'm here with Sally Currie, who is CEO of the charity Drake Music.

Hi, Sally.

Hello, how are you?

I'm good, thank you.

Enjoying the exhibition.

How is it going?

Oh, yeah, it's going really, really, really well.

We've got a lot of interest in our all the noises coming out of our room.

Well, it's an amazing room because we're sort of surrounded by musical instruments.

So, what is your charity about?

How are you making music more accessible?

Okay, so Drake Music is a charity that's been running since 1993.

We operate at the intersection between music, disability, and technology.

And our vision is a world where disabled and non-disabled people work together as equals.

We believe that access to music matters and not everybody has it.

And so, we specifically use technology to break down those barriers through a number of different projects.

We have a couple of principles that we work with.

So, the first one is co-design, where you co-design with the people that are actually going to be using the product.

And the second one is the social model of disability.

So, that means that, for example, I'm partially deaf, I'm wearing hearing aid.

We contrast it with the medical model, which would say that there's something wrong with me and that I need to be cured.

But the social model means that actually the person is not disabled because of something wrong with them, they're disabled by society and the barriers that it presents.

So, barriers for me may be accessing something might be people not providing captions.

It's not because I'm deaf, it's because they haven't provided captions.

So, yeah, that's how we work.

They're the overarching principles, and as you've realised, we do a lot of accessible instrument development.

And we're showcasing a number of examples here and celebrating our long-term legacy in instrument development.

We're celebrating over 15 years' worth of work here.

Wow, so it's breaking down those barriers so that music is not something that is inaccessible to people with certain disabilities, it's just opening it all up.

Everybody has a fundamental human right to access music, like many other things.

So, we're focusing on music here, and so it's all about equity.

You know, it's about giving people the same chances, and different people may need different things to have that first starting point.

But the exhibits that we've got in the exhibition is an example of three different types of work that we do.

So, the first thing is our long-running partnership with the London Philharmonic Orchestra, our Ork Lab programme.

You will see a number of adapted orchestral instruments that have been designed with residents of homes where we go to deliver work.

The second thing is the Mimu Gloves, who was invented by Imogen Heap and her team.

And the final thing, the centrepiece, is the photosynth.

And the photosynth was invented invented by Zen Alenski.

And we awarded Zen a grant through our DM Lab programme to develop this instrument.

He's been working on it for 14 years.

And it is a mixed reality

sort of application.

It tracks your facial expressions and it makes music based on your facial expressions.

And I really feel like Zen can give a better explanation.

Well, I think you gave a beautiful explanation.

I'm very excited to try it because as as is the tradition with the Royal Society exhibition, we can have a play with this, can't we?

We can

actually, and to be honest, I insist.

My name's Zen Onalenski, and I'm a creative designer from London.

So, Caroline and I are very excited about this.

This is making music out of our positions and facial expressions.

So, we have in front of us a screen that we can see ourselves on.

So, there's a camera pointing at us, and then there's a synthesizer.

How does this work, Zen?

This is very fun.

A smile-powered synthesizer.

This is a new era of music making.

Rather than making the music the traditional route, we're trying to augment your existing abilities in exciting new ways to give you instant musical superpowers.

Amazing.

And how does that work?

We're going to turn up the volume in a minute so you can hear.

It's tracking our faces.

Yeah, if I sort of smile, a smiley face pops above my smile.

And if I frown, a frowny face pops above my frown.

Yeah, it's tracking every single facial expression, also the positions of our bodies as well.

So how are the different sounds created, Zen?

So by analyzing your facial expression, we have certain neural networking models that convert your facial expressions into what we analyze as an intention, and then we convert that intention back into music.

Okay, can you give us a demo?

Making music with you?

Yes, I do.

I'll join you.

And are we just going to sort of freestyle jazz it, or do you have a suggested sort of pattern of behavior for us to carry out so we can make something a bit melodic?

I think if we all smile together we'll make something quite oh okay that sounds lovely.

That does sound lovely.

Okay all right.

Should we all smile?

Okay one two three

quite a dramatic chord though it is yeah

And so what do you

do you see bands using this?

Is this about playing a game?

What do you hope people are going to get out of this?

So originally it was made for the disabled community to allow everybody, regardless of their background, to be able to make music.

Part of the setup with the camera allows people like lying down in bed or in a wheelchair to use it.

There isn't sort of any expectations about your body type or your abilities.

Could you maybe demo for us some of the different ways in which this responds to your face?

I feel like you might be a more skilled user of Photosynth than we are.

That was amazing!

That was so good!

That was beautiful!

Thank you so much, Sam.

Thank you for coming around.

Well, very sadly, that is all we have time to explore today.

Thank you so much, Caroline Steele, for joining me on this slightly hot and sticky whirlwind scientific tour.

Thanks for having me, it's been fun.

It has been a joy.

And you have been listening to BBC Inside Science with me, Victoria Gill, and with Caroline Steele.

The producers were Dan Welsh and Claire Salisbury.

The show was made in Cardiff by BBC Wales and West.

And don't forget, if you have any scientific questions for the team, you can email us at insidescience at bbc.co.uk.

For now, though, thanks for listening and see you next time.

Want to know my ultimate beauty secret?

It's a prescription eye drop that lifts low-lying eyelids in minutes.

Just one drop and boom, lifted eyelids that last up to eight hours.

My go-to for looking more awake?

It's Upnik.

Oxymetazoline HDL ophthalmic solution, 0.1%.

Drooping eyelids can be caused by other more serious conditions such as stroke.

Tell your doctor your symptoms and medical history, including blood pressure or blood flow issues and heart, brain, or eye disease.

This is not a complete list of risks.

Learn more at upnik.com.

That's upnu.com.

With a wealthfront cash account, your uninvested cash earns 4% annual percentage yield from partner banks with free instant withdrawals, even on weekends and holidays.

4% APY is not a promotional rate, and there's no limit to what you can deposit and earn.

Wealthfront.

Money works better here.

Go to WealthFront.com to start today.

Cash account offered by Wealthfront Brokerage LLC member Fenra SIPC.

Wealthfront is not a bank.

The APY on cash deposits as of December 27, 2024 is representative, subject to change, and requires no minimum.

Funds in the cash account are swept to partner banks where they earn the variable APY.