How do you turn facial expressions into music?

Speaker 1 This BBC podcast is supported by ads outside the UK.

Speaker 2 It's the coziest time of year on Britbox.

Speaker 1 Very cozy.

Speaker 2 That means basking in the ambience. We know the bodies turned up.
How often do people get murdered around here? Unboxing the unexpected.

Speaker 5 Well, we know it wasn't an accident.

Speaker 2 And starting new traditions.

Speaker 1 Oh, so you're telling me to behave myself. Oh, shut up.

Speaker 2 Stream Brick Box original series based on best-selling novels, including Lindley and a new season of Karen Pirry.

Speaker 7 Smashy, smashy, breaky, breaky.

Speaker 2 It's all a bit warmer with Britbox. See holidays differently when you stream the best of British TV with Britbox.

Speaker 8 It's time your hard-earned money works harder for you. With the Wealthfront Cash Account, your uninvested cash earns a 3.5% APY, which is higher than the average savings rate.

Speaker 8 No account fees, no minimums, and free instant withdrawals to eligible accounts anytime. Join over a million people who trust Wealthfront to build wealth at wealthfront.com.

Speaker 8 Cash account offered by Wealthfront Brokerage LLC, member FINRA SIPC, and is not a bank. APY on deposits as of November 7th, 2025 is representative, subject to change and requires no minimum.

Speaker 8 Funds are swept to programme banks where they earn the variable APY.

Speaker 6 BBC Sounds, Music, Radio, Podcasts. Hello, and welcome to BBC Inside Science, a programme first broadcast on Thursday, the 3rd of July, 2025.

Speaker 6 I'm Victoria Gill, and you join me on a very hot street in central London because I am looking at the gleaming white columns of the Royal Society on Carlton House Terrace.

Speaker 6 This week, the UK's world-famous scientific academy opens its doors for its annual summer science exhibition, an event that showcases the latest research from across the UK and invites the public in.

Speaker 6 And with me to explore it is science journalist, broadcaster, and fellow seeker of free exhibitions that provide an escape from this terrifying weather, Caroline Steele.

Speaker 3 Hello, Caroline. Hi,

Speaker 3 good. Thank you.
Thanks for having me on.

Speaker 6 Oh, it's an absolute pleasure. There's a lot to explore, so I'm glad to have a partner in crime.

Speaker 6 This exhibition has a very long history, doesn't it? But the public weren't always invited in.

Speaker 3 Yeah, so the very first Royal Society Summer Science Exhibition was back in 1778. Apparently in 1896, visitors got to x-ray their hands, so my hopes are high.

Speaker 3 But in all seriousness, it'll be interesting to see what's on display this year, because it's been a difficult year for science, which is something the Royal Society has acknowledged.

Speaker 3 Earlier this year, they released a statement saying that science is under threat because of huge cuts and a rise of misinformation.

Speaker 3 So it'll be interesting to see what they've chosen to sort of showcase to the year.

Speaker 6 Yeah, so and why do you think it amid all of those risks and the Royal Society coming out and saying that science is under threat from you know all of the challenges that you that you cited there why is it important to open the doors and invite people in do you think?

Speaker 6 Why is this something that the Royal Society showcases?

Speaker 3 Well it's going to be great to have scientists and the general public in the same room although it's worth saying you know thousands of people will pass through these doors this week but it is only on in London so it's not something everyone is going to get the chance to experience.

Speaker 6 Yeah that's true and hopefully we'll give you a little snippet of that experience in audio form. So let's go and have an explore.

Speaker 3 Let's do it.

Speaker 6 We are starting our tour in the quantum zone.

Speaker 6 There is an entire zone for quantum at this exhibition because the UN has declared this year 2025 the International Year of Quantum celebrating 100 years of quantum science and technology since groundbreaking work was published by physicists like Erwin Schrödinger and Werner Heisenberg.

Speaker 6 I'm here with Daisy Shearer from the National Quantum Computing Centre, who is the person who has put this whole zone together.

Speaker 3 Hello, Daisy. Hello.

Speaker 6 Thank you for joining us. Why does quantum have an entire zone at the exhibition this year?

Speaker 10 Well, as you said, it's all about the International Year of Quantum Science and Technology.

Speaker 10 So we've come together to bring everybody in, well, most people in quantum in the UK to celebrate their science and showcase the amazing collaboration that we do nationally.

Speaker 6 Quantum can be a bit baffling, can't it? And a bit sort of hidden inside technology that we're you know coming to depend on or that we do depend on.

Speaker 6 How are you showcasing it and showing off how just important it is?

Speaker 10 Absolutely. So, one of the amazing things about quantum is that we all use these quantum mechanical effects to build different technologies.

Speaker 10 So, we're showcasing some of those effects like entanglement that you may have heard of, and optics as well.

Speaker 10 And then we're showing how we can apply this to different technologies. So, we've got quantum computing, quantum sensing, quantum timing, quantum imaging, and quantum communications as well.

Speaker 6 What do you want people to take away from this?

Speaker 10 Excitement.

Speaker 10 I would love to see young people being able to see themselves in quantum careers, but also just to help people demystify quantum because it has a bit of a reputation for being quite complex.

Speaker 10 And we're here to showcase that there's all this awesome stuff happening and everybody can get involved.

Speaker 6 Complex but exciting and useful. Absolutely.
And I understand there's some quantum music. And since we're an audio programme, I think we might go and investigate that.
Thank you, Daisy.

Speaker 6 Clelia Altamonte is a PhD student at King's College London who has been experimenting with how to create music from quantum circuits.

Speaker 3 Hi, Clelia. Hi.

Speaker 6 Lovely to meet you. I'm intrigued.
First of all, though, we're going to have to start with the basics for me, I'm afraid. What is a quantum circuit?

Speaker 11 So I guess we might have to start from the difference between a bit and a qubit. So a bit is the classical unit of information, so it can be either zero or one.

Speaker 11 And bits are used in classical computers, for example in binary codes. The qubit is the quantum version of that.
So it is like a bit that obeys the rules of quantum mechanics.

Speaker 11 And because of that, it has some extra properties. For example, it can be neither in a state of zero or one, but in a state that is in between.
And we call that a superposition state. Right.

Speaker 11 Or if we bring together more than one qubit, they can be entangled with one another, which means that they're not independent of one another anymore and we can get correlations that are beyond what would uh be classically possible.

Speaker 6 Those quantum properties, not having to be in a state of zero or one

Speaker 6 and the quantum concepts of entanglement, that's what makes quantum circuitry so powerful, more powerful than classical computing.

Speaker 11 And in fact, many uh quantum algorithms use these properties of uh entanglement and superposition and because of that some can exceed what what is possible classically.

Speaker 6 So where did the idea come from to turn quantum circuits into music?

Speaker 11 So the idea first came to me as I was working with my supervisor and we were working on a quantum circuit. He noticed that it looked very much like a music score.

Speaker 6 Say lines and symbols and

Speaker 6 these changes.

Speaker 3 That's right.

Speaker 11 And since that time, every time I've spoken to people about quantum circuits, I've always used this analogy.

Speaker 11 And people told me that they found it quite useful in order to quickly understand how quantum circuits work.

Speaker 11 And then a couple of months ago I was at a concert and I was really enjoying the music and really felt like I wanted to play an instrument and I've not been playing in a long time.

Speaker 11 Even though when I was little I actually learned how to play the piano and the guitar.

Speaker 11 The easiest way for me to get back to music was trying to put to the test that music analogy with the quantum algorithms.

Speaker 6 Right, so the easiest way for you to get back into making music was to quantum circuits.

Speaker 3 Yes, similar music. That's right.

Speaker 6 What's your first example? Take us through. So we're going to start with quite a simple one.

Speaker 11 The easiest one is a single qubit first put in a superposition state and then we perform some rotations on it and it sounds like this.

Speaker 6 That does sound simple. So what are we hearing there? There's a chord.
We kind of have two different notes and they're changing in relation to each other.

Speaker 11 So, the way we mapped the state of the qubit to music notes is through the block sphere. So, it is possible to represent the state of a qubit as a point on a sphere.

Speaker 11 So, we can have two angles representing that state. And each of these two angles was mapped to music notes on a different octave.

Speaker 11 And in this case, because we have only a single qubit, we only have one instrument. The next example example is we have first entangled the two qubits.

Speaker 6 Okay, so now we have what Einstein called spooky action at a distance. Yes.

Speaker 6 We have entangled qubits.

Speaker 3 Okay, let's have a listen to that.

Speaker 6 Entanglement's quite harmonious.

Speaker 11 It is.

Speaker 6 When they're not independent, when they're related to each other, there's that sort of

Speaker 6 harmony. That's fascinating.
I feel like I'm hearing quantum in a way that's clarifying it for me. It's not something that I think is.

Speaker 3 That's great.

Speaker 6 Yeah, that's fantastic. Is there anything else that you would like to play for us that you think is particularly melodic quantum tech?

Speaker 11 I'm not sure it is melodic, but I think it sounds interesting because the next thing that we did was to try and see how some very famous quantum circuits and quantum algorithms sound like.

Speaker 11 For example, this is Grower's search algorithm. It is used to search an unstructured database.

Speaker 11 It feels a bit dark, but I can show you.

Speaker 6 This is quite a s a scary quantum algorithm there. Where can we download the quantum album then? Do are you going to be releasing it any time soon?

Speaker 11 Yes, so there is a GitHub website with um theory about how to turn quantum algorithms into music, and also the code is available.

Speaker 6 Listeners can make their own quantum music. We'll have to put that link on the programme page.
Clearly, this has been fascinating. Thank you so much for this audio tour of your quantum research.

Speaker 6 Thanks.

Speaker 6 Caroline, that music is quite a fun way of visualizing, wrapping your head around quantum circuitry, isn't it?

Speaker 3 Yeah,'cause it can be really complicated, sticky math. So to be able to kind of listen to it, like hear what entanglement could sound like, I think it's really helpful and good fun.

Speaker 6 Yeah, yeah, I feel almost like I understand it now.

Speaker 6 There's such a variety of different research and different technology on display, and it's amazing what you can try out here and really get hands-on.

Speaker 6 Speaking of which, would you like to find out a little bit more about spider and elephant sensors?

Speaker 3 Ooh, yeah.

Speaker 6 Well, I know just where to go, you know, just the woman.

Speaker 13 I'm Dr. Beth Mortimer.
I am a Royal Society University Research Fellow and I work at the Department of Biology at the University of Oxford.

Speaker 6 And your lab is the Animal Vibration Lab?

Speaker 13 That's right. I did think very hard before picking that name.
I've gone for that because we don't just study one type of animal, but we study a range of animals from spiders and elephants.

Speaker 13 And what they all have in common is that they use vibrations through materials for information.

Speaker 6 Explain that to me. What does that actually mean? Like, if you give me the example of elephants, how are they using vibrations?

Speaker 13 Yeah, sure. So, when you think of vibrations, you might more commonly think about hearing.
And vibration sensing is linked to hearing, but it is slightly different.

Speaker 13 So, sound usually travels through air, and usually we would say hearing or an ear is linked to vibrations through air.

Speaker 13 But what we're studying is how they're able to use vibrations that move along surfaces or through materials rather than through air.

Speaker 13 So, different animals do it in different ways but essentially usually the surface that they're standing on they're using how that moves to gain information about their environment but also information from other animals as well.

Speaker 6 Picking up vibrations through their feet?

Speaker 13 That's right so in the case of the spiders they've got thousands of sensors that are distributed throughout their legs and essentially when whatever they're standing on vibrates that moves their exoskeleton and then they detect that through these sensors.

Speaker 13 So we're interested in studying spiders because if we can work out how they're able to not only tell what is vibrating but where it might be

Speaker 13 and understanding what role their posture or their body shape actually has in that, then we might be able to design robots that have embedded sensors in them as well that can take information from the environment.

Speaker 6 Spiders, I guess, it makes sense to me that their world is quite vibration filled. Their webs vibrate, they have multiple legs and lots of places for sensors on that body plan.
What about elephants?

Speaker 6 Like, how are they using vibrations?

Speaker 13 In terms of how they detect it, effectively, your ear can detect vibrations that travel through your bone, not just vibrations through the air.

Speaker 13 So, this is how we think that elephants might be detecting vibrations through the ground.

Speaker 13 The reason this is important is because the elephants actually generate ground-based vibrations when they vocalize.

Speaker 13 So, when they communicate using rumble vocalizations, which is one type of vocalization made by an elephant, it vibrates the air, but it also vibrates the ground.

Speaker 13 So yeah, we know that they can communicate using just seismic vibrations.

Speaker 6 So is that sort of like the way very, very bassy music will kind of move a wall or sort of how a speaker works? Like there's a physical vibration from that deep, deep rumble?

Speaker 13 Yeah, that's right. So it's basically elephants are so large, they've got such a big diaphragm, they're such low frequency.

Speaker 13 All of that together means they're basically bombarding the ground with this very, very low frequency, loud vibration that causes the ground to vibrate.

Speaker 13 And then they're able to transmit information just from ground vibration.

Speaker 13 So, without any airborne vibration, they're able to detect and respond to elephant rumbles in a way that shows they transfer information between each other.

Speaker 6 So, Beth, how are you bringing that science to life in your exhibit?

Speaker 13 So, one of the main things we're very proud of for the exhibit is that we've got a multi-sensory experience. So, you can go inside a pod, but it's just a fancy name for a gazebo, really.

Speaker 13 And there's a vibrating floor where we're going to be playing the real vibrations that we've recorded from both spiders and elephants, either in the lab or in their natural environment.

Speaker 13 So because we're not very good with our own seismic sense, to kind of aid people in interpreting that, we've basically got a visual representation of the seismic vibration, as well as an audible version as well.

Speaker 6 And we have a very enthusiastic Caroline Steele who is keen to give this vibration gazebo a try.

Speaker 3 Is that all right? Could you show me the vibrating gazebo?

Speaker 13 Absolutely, come over here for the multi-sensory experience.

Speaker 3 It's a large black gazebo with a black floor and a huge screen across one back wall. So how does this work? That's right.

Speaker 13 So what we'd like you to do is if you want to get the full experience, then take off your shoes

Speaker 13 because you do have vibration sensors embedded in your skin, which you can investigate through this experience.

Speaker 13 So, this is a vibrating floor, and you're going to feel, see, and hear the vibrations that are made by spiders and elephants.

Speaker 3 Okay, shoes off. Can I keep my socks on?

Speaker 13 You can keep your socks on, yeah. Great.

Speaker 3 Okay, so I'm going to step up onto the black floor. Okay, there you go.
Right, are you ready? Yeah, I think so.

Speaker 14 Prepare to experience the seismic world.

Speaker 14 Cupianius spiders, found in South and Central America, are nocturnal predators. With limited vision, they largely hunt by feel.

Speaker 3 The floor's vibrating.

Speaker 14 In the Kenyan savanna, researchers have buried seismic sensors just under the ground surface. A low-frequency rumble rolls across the ground as an elephant calls out to its family.

Speaker 3 Whoa!

Speaker 3 My face is vibrating. That's a really big rumble.

Speaker 14 It's not a sound, but a seismic signal. A message.
We're here and we belong together. Aww.

Speaker 3 That was so cool.

Speaker 13 There you go. So that's the short version of the multi-sensory experience.
So how did you find it?

Speaker 3 I absolutely loved it. I mean, I just, I kind of can't believe it's almost like a whole extra sense that I feel like I'm not making the most of that these spiders and elephants have.

Speaker 3 Thank you so much. That was a lot of fun.
You're welcome.

Speaker 13 Don't forget your shoes.

Speaker 1 Be our guest at Disney's enchanting musical, Beauty and the Beast. Fill your heart with joy and Disney magic.
Brought to life like never before.

Speaker 1 Coming to the Orpheum Theater July 14th through August 9th. Tickets at BroadwaySF.com.

Speaker 4 Dashing through the store, Dave's looking for a gift. One you can't ignore, but not the socks he picks.
I know, I'm putting them back. Hey, Dave, here's a tip.
Put scratchers on your list.

Speaker 1 Oh, scratchers, good idea.

Speaker 4 It's an easy shopping trip. We're glad we could assist.

Speaker 5 Thanks, random singing people.

Speaker 4 So be like Dave this holiday and give the gift of play. Scratchers from the California lottery.
A little play can make your day.

Speaker 5 Please play responsibly. Must be 18 years or older to purchase play or claim.

Speaker 6 You are listening to BBC Inside Science with me, Victoria Gill. And this week we're exploring the Royal Society Summer Science Exhibition.

Speaker 6 Now, from understanding animal senses to using animals as sensors. Caroline, why is this particular exhibit that seems to be all about water and tiny creatures caught your eye?

Speaker 3 Yeah, well, I'm a really keen open water swimmer and I'm quite worried about the state of our rivers and lakes. And I'm hoping there's a tiny creature here that might tell us a bit more.

Speaker 3 Let's go and find out.

Speaker 12 I'm Dr. Katie Riley from the University of Birmingham.
I'm an environmental scientist, and my research specialises in freshwater pollution and ecotoxicology.

Speaker 6 Just describe the exhibition, and particularly the little tiny creatures that we've seen on the screens and in the kind of microscopic images.

Speaker 12 Yeah, so a lot of my research works with daphnia. So they're a small freshwater crustacean.
They're very similar to kind of like sea monkeys.

Speaker 12 They're super useful in freshwater environments because we can use them as an indicator of good water health.

Speaker 12 and they've been used so much for research, we have a really good baseline to understand and rank environmental hazards using them.

Speaker 6 How do you do that? What are you looking for in the Daphnia? Is it sort of presence, absence, or do they give you all the clues?

Speaker 12 So, the most basic test that we can do in terms of ranking their performance is when they stop moving, so mobilization, and we use chemicals and like vary the doses and then see and observe the effects.

Speaker 12 So, when they stop moving, that's like known as the effect concentration.

Speaker 12 Daphnia tend to need good levels of oxygen in the water and things like that, so it's a good marker that the ecosystem is healthy enough to support something so sensitive.

Speaker 3 Can you see them, or are they so small that you don't even notice them? I know you can see them. I'll show you.
Really?

Speaker 12 Yeah,

Speaker 12 so a rough mark of thumb would be: it's a fully grown Daphnia magna, which is one of the larger species, would be about a third of the size of your little fingernail, if you like, imagine it like that.

Speaker 6 What have you got in your exhibition? How are you kind of presenting these teeny, tiny little pollution detectors to the public?

Speaker 12 Often we look at like the bigger animals, so we might look at like turtles and things like that.

Speaker 12 We don't often look at the microscopic so we're trying to really elevate the importance of like these much smaller species.

Speaker 12 So to do that we've got some Daphnia that have eaten some fluorescent particles so these are fluorescent plastics so it'll be really good to kind of highlight the different tools that we can use in environmental science.

Speaker 12 So like the fluorescent stains we can use to assess some of these growing problems like plastic pollution.

Speaker 6 You can see inside their bodies.

Speaker 12 Yeah so they're completely transparent which is super cool for a test organism. So we can see how much is in their gut, how long it's in their gut.

Speaker 3 Next time I swim I'm going to be looking out for these tiny flea-like things in the water.

Speaker 6 Maybe we can have a look at them now under the microscope. Can we take a look?

Speaker 12 Yes, please, that'd be great.

Speaker 6 I always get slightly excited when I see a microscope.

Speaker 3 It's like, ah, science is there. It's like the symbol of science, isn't it?

Speaker 12 Hopefully, you'll be able to see a Daphnia under the microscope full of plastic.

Speaker 6 Okay, so these are the Daphnia that have eaten the brightly coloured plastic.

Speaker 3 Yeah, and we have literally fed them 10 minutes ago. Oh, okay, so I see a background that looks to me like space, lots of sort of different specks of light, and in the middle there is,

Speaker 3 oh,

Speaker 9 I think a daphnia.

Speaker 3 Oh, yeah, it's moving. Very cute.
Really reminds me of the sea monkeys I had as pets, as a kid.

Speaker 6 And what do they eat when you're not feeding them bits of plastic?

Speaker 3 Algae.

Speaker 12 Rice. Yeah, so just normal, like the green algae that you'd find in water.

Speaker 6 And so if they're healthy algae-munching daphnia, they'd appear green because you could see into.

Speaker 12 Yeah, we have one just on the next microscope, which is like a controller.

Speaker 3 Yep. Paralleline's straight in there with the microscope.

Speaker 3 Ooh, ah, ooh, ah.

Speaker 3 Those are really good noises.

Speaker 3 It almost looks like a speck of algae that is moving because it is so algae coloured.

Speaker 6 What do you want people to take away from your exhibition and these tiny little microscopic creatures?

Speaker 12 I really hope that it gets people invested in the work and shows them that we are kind of working towards solutions, but also I hope it highlights the scientific process for some of the things so that when people come to advocate for their spaces, which I really encourage people to do, that they have the rationale and stuff to be able to map their wants and desires for their spaces.

Speaker 6 Why do you think it's important that people grasp that if they care about their waterway, whether they swim or they just kind of enjoy the outdoors?

Speaker 12 So, when we're saying, like, oh, we can only prioritise one site and we need to decide what's the most polluted, like, what do we mean by the most polluted?

Speaker 12 Do we mean the one that has the most chemical pollution, which is leading to like harmful algal blooms and things like that?

Speaker 12 Or do we mean the one that has the most plastic in it, which we don't don't really understand the consequences fully of yet.

Speaker 6 Katie thank you so much this is really interesting and good luck with the rest of the exhibition.

Speaker 3 Thank you.

Speaker 6 Well I have to admit Caroline I found the quantum music that we listened to a little bit earlier a tad unsettling.

Speaker 3 Yeah it was a bit sort of spooky apart from the entanglement part.

Speaker 6 Yeah so we want to find something maybe a little more melodic to round off our scientific tour and we've come to an exhibition that is all about using technology to make it possible for people with disabilities to make music.

Speaker 6 I'm here with Sally Currie, who is CEO of the charity Drake Music. Hi, Sally.

Speaker 3 Hello, how are you?

Speaker 6 I'm good, thank you. Enjoying the exhibition.
How is it going?

Speaker 15 Oh, yeah, it's going really, really, really well. We've got a lot of interest in our all the noises coming out of our room.

Speaker 6 Well, it's an amazing room because we're sort of surrounded by musical instruments. So, what is your charity about? How are you making of music more accessible?

Speaker 15 Okay, so Drake Music is a charity that's been running since 1993.

Speaker 15 We operate at the intersection between music disability and technology, and our vision is a world where disabled and non-disabled people work together as equals.

Speaker 15 We believe that access to music matters and not everybody has it, and so we specifically use technology to break down those barriers and through a number of different projects.

Speaker 15 We have a couple of principles that we work with.

Speaker 15 So, the first one is co-design, where you co-design with the people that are actually going to be using the product, and the second one is is the social model of disability.

Speaker 15 So, that means that, for example, I'm partially deaf, I'm wearing hearing aid. We contrast it with the medical model, which would say that there's something wrong with me and that I need to be cured.

Speaker 15 But the social model means that actually the person is not disabled because there's something wrong with them, they're disabled by society and the barriers that it presents.

Speaker 15 So, barriers for me may be accessing something might be people not providing captions. It's not because I'm deaf, it's because they haven't provided captions.

Speaker 15 So, yeah, that's how we work. They're the overarching principles, and as you've realised, we do a lot of accessible instrument development.

Speaker 15 We're showcasing a number of examples here and celebrating our long-term legacy in instrument development. We're celebrating over 15 years' worth of work here.

Speaker 6 Wow, so it's breaking down those barriers so that music is not something that is inaccessible to people with certain disabilities, it's just opening it all up.

Speaker 15 Everybody has a fundamental human right to access music, like many other things.

Speaker 15 So, we're focusing on music here. And so, it's all about equity.
You know, it's about giving people the same chances, and different people may need different things to have that first starting point.

Speaker 15 But the exhibits that we've got in the exhibition is an example of three different types of work that we do.

Speaker 15 So, the first thing is our long-running partnership with the London Philharmonic Orchestra, our Orch Lab programme.

Speaker 15 You will see a number of adapted orchestral instruments that have been designed with residents of homes where we go to deliver work.

Speaker 15 The second thing is the Mimu Gloves, who was invented by Imogen Heap and her team. And the final thing, the centrepiece, is the photosynth.

Speaker 15 And the photosynth was invented by Zen Alenski, and we awarded Zen a grant through our DM Lab programme to develop this instrument. He's been working on it for 14 years, and it is a mixed reality

Speaker 15 sort of application. It tracks your facial expressions and it makes music based on your facial expressions.
And I really feel like Zen can give a better explanation.

Speaker 6 Well, I think you gave a beautiful explanation, and I'm very excited to try it because, as is the tradition with the Royal Society exhibition, we can have a play with this, can't we?

Speaker 15 We can act music can, and to be honest, I insist.

Speaker 7 My name's Zenona Lenski, and I'm a creative designer from London.

Speaker 6 So, Caroline and I are very excited about this. This is making music out of our positions and facial expressions.
So, we have in front of us a screen that we can see ourselves on.

Speaker 6 So, there's a camera pointing at us, and then there's a synthesizer. How does this work, Zen? This is very fun.

Speaker 7 A smile-powered synthesizer. This is a new era of music making.

Speaker 7 Rather than making the music the traditional route, we're trying to augment your existing abilities in exciting new ways to give you instant musical superpowers.

Speaker 6 Amazing. And how does that work? We're going to turn up the volume in a minute so you can hear.
It's tracking our faces.

Speaker 3 Yeah, if I sort of smile, a smiley face pops above my smile. And if I frown, a frowny face pops above my frown.

Speaker 6 Yeah, it's tracking every single facial expression, also the positions of our bodies as well. So how are the different sounds created, Zen?

Speaker 7 So by analyzing your facial expression, we have certain neural networking models that convert your facial expressions into what we analyze as an intention, and then we convert that intention back into music.

Speaker 6 Okay, can you give us a demo?

Speaker 7 Making music with you? Yes, I do.

Speaker 3 I joined it.

Speaker 6 And are we just gonna sort of freestyle jazz it, or do you have a suggested sort of pattern of behavior for us to carry out so we can make something a bit melodic?

Speaker 7 I think if we all smile together, we'll make something quite.

Speaker 3 Oh, okay. That sounds lovely.

Speaker 6 That does sound lovely.

Speaker 3 Okay, all right. Through your smile?

Speaker 6 Okay, one, two, three.

Speaker 3 It's quite a dramatic chord, though. It is.

Speaker 3 And so, what do you.

Speaker 6 Do you see bands using this?

Speaker 6 Is this about playing a game? What do you hope people are going to get out of this?

Speaker 7 So, originally, it was made for the disabled community to allow everybody, regardless of their background, to be able to make music.

Speaker 7 Part of the setup with the camera allows people like lying down in bed or in a wheelchair to use it. There isn't sort of any expectations about your body type or your abilities.

Speaker 6 Could you maybe demo for us some of the different ways in which this responds to your face? I feel like you might be a more skilled user of Photosynth than we are.

Speaker 6 That was amazing! That was so good!

Speaker 3 That was beautiful!

Speaker 6 Thank you so much, Sam. Thank you for coming around.

Speaker 6 Well, very sadly, that is all we have time to explore today. Thank you so much, Caroline Steele, for joining me on this slightly hot and sticky whirlwind scientific tour.

Speaker 3 Thanks for having me, it's been fun.

Speaker 6 It has been a joy. And you have been listening to BBC Inside Science with me, Victoria Gill, and with Caroline Steele.
The producers were Dan Welsh and Claire Salisbury.

Speaker 6 The show was made in Cardiff by BBC Wales and West. And don't forget, if you have any scientific questions for the team, you can email us at insidescience at bbc.co.uk.

Speaker 6 For now, though, thanks for listening and see you next time.

Speaker 16 This holiday, deck their countertops with peppermint white chocolate mochas, gingerbread lattes, and pumpkin cream cold brews. Ninja Lux Cafe is the ultimate gift.

Speaker 16 It's the number one best-selling espresso machine in the US. Perfect for the coffee lovers in your life.
If they can dream it, they can brew it. No skills needed.
Espresso balanced. Drip coffee, rich.

Speaker 16 Cold brew in a flash. Hands-free silky microfoam.
Dairy or plant-based milks. Bring joy into their holiday and every day after with Ninja Lux Cafe.
Shop now.

Speaker 17 This is the story of the one. As head of maintenance at a concert hall, he knows the show must always go on.

Speaker 17 That's why he works behind the scenes, ensuring every light is working, the HVAC is humming, and his facility shines.

Speaker 17 With Granger's supplies and solutions for every challenge he faces, plus 24-7 customer support, his venue never misses a beat. Call quickgranger.com or just stop by.

Speaker 17 Granger, for the ones who get it done.