Dr Lorenzo Picinali, Senior Lecturer in Audio Experience Design at Imperial College London, visited GOV.UK to talk about his work. He works on acoustic virtual and augmented reality. He’s recently worked on 3D binaural sound rendering, spatial hearing, interactive applications for visually impaired people, hearing aids technologies, audio and haptic interaction.
Vision contains much more information than sound. If there’s audio and visual input, our brains generally prioritise the visual.
e.g. the McGurk illusion: visual input shapes our understanding of sound.
https://www.youtube.com/watch?v=G-lN8vWm3m0
Echo location. This blind man gets information on size, layout, texture and density by making a clicking noise and listening to the echoes. He trained his brain to better localise echoes.
(To learn more, check out this episode of the Invisibilia podcast)
In some contexts sound is better than vision:
- It’s 360 degrees. You don’t have to be looking at it.
- It’s always active. e.g. good for alarms.
- Occlusions don’t make objects inaudible. (You can often hear things even if there’s another object in the way, whereas line of sight is generally blocked by other objects.)
- Our brain is really good at comparing sound signals
- We’re better at memorising tonal sequences than visual sequences.
Examples of good interfaces that use sound:
- Sound can be useful to give people information in busy situations. e.g. a beeping noise to help you reverse park.
- Music to help pilots fly level at night. With this interface, the left or right volume would change if the plane was tilting, and the pitch would go up or down if the plane was pointing up or down. This worked really well.
- A drill for use in space. Artificial sound communicated speed and torque.
Acoustic augmented reality is a frontier that hasn’t been explored yet. We can match the real world and the virtual world more convincingly than with visual elements of augmented reality, where it’s quite clear that they aren’t real.
Our ears are good at making sense of differences in volume and the time that sound reaches them. This lets us work out where in space sounds are coming from. Our binaural audio processing skills mean that we can create artificial 3d soundscapes.
https://imperialcollegelondon.app.box.com/s/3ki1gg770nmzhykhvnqzzdd7ipx6ioem
Plugsonic – a platform that lets you create 3d soundscapes on the web using your own sound file and pictures.