Archontis Politis is a PhD researcher at Communication Acoustics Research Group in Aalto University, Finland. We sat down with him after the Kuopio Symphony Orchestra trio’s home concert to discuss the current state of 3-D audio, primarily how it is recorded and what you need for playback.
Antti Vanne: Thank you, Archontis, for helping us out with the recording of this special event. Could you tell us briefly how it’s possible to record immersive 3-D audio and video?
Archontis Politis: No problem, Antti! To record 3-D audio, we use microphone arrays, which are basically a combination of microphones. When we combine the separate mic signals, we can create a more accurate audio perception for the listener. The same perception they would have in the actual setting, by capturing where the sounds are originating from and preserving that information in the recording.
The equipment we used today is an experimental 3-D microphone with 32 individual microphones on top of a sphere, called Eigenmike from MH Acoustics. It is probably the best you can get on the market today in terms of spatial resolution. It’s still an experimental device, though, and mostly used in research and development. The video was captured with a rig of 6 GoPro cameras, set on the faces of a cube. This can provide high-resolution immersive video, lets say about 8000 by 4000 resolution, which is enough for many immersive virtual-reality (VR) applications.
And then there is also the method of playback for reproducing the 3-D sound. We can use loudspeaker arrays, but if we want to create true immersion we have to go beyond the traditional surround systems like 5.1 or 7.1. We have to have loudspeakers above and below the listener to accurately reproduce the sounds coming from above and below the listener. We can also use headphones, but then we have to make sure that the head of the listener is tracked and use that in playback so that we can keep the sound scene stable while the listener moves his head to make it sound as natural as it sounds in real life.
AV: How do you think 3-D recording and playback will evolve in the near future?
AP: Well it has already evolved to a stage where it is quite usable. It has pretty good quality in terms of playback. I think it will become much more ubiquitous and you’ll start to see it integrated in more and more devices, and it will follow the development of spherical cameras. Most of these cameras will begin to utilize some kind of spatial sound recording method. The playback end will have headphones with integrated head tracking, the same kind of technology used in VR devices.
AV: What do you expect from the recording tonight? Do you think we can reproduce the music and environment, the players and so on?
AP: I think the performance was great and the acoustics of this space are quite nice. Congrats on that. The aim of today’s recording is to preserve the natural impression of the concert as accurately as possible. There is always a trade off in preserving the acoustics and also in getting good fidelity in general, but hopefully we’ll manage to make it sound quite realistic in that sense. Using this type of advanced experimental device like the 3-D microphone that we were using, it’s much more straightforward to do that compared to traditional methods that we’ve used for surround sound recordings that are much harder to set up in general.
AV: Can the recording also be used for research purposes?
AP: Definitely, there are many things that can be tested with this type of recording. Since we have both spatial sound and visual immersive information at the same time, we can perform experiments to test audio-visual perception. Until now, there had not been enough material for large scale tests at the universities, the recording process was too complicated. But now we can perform these types of experiments. Also, in basic acoustical research, for example, focusing different directions of the recording, extracting different sounds from the recording and then having the video that gives you additional information, a little bit like acoustic cameras.
AV: So is it mostly about science, what about art?
AP: No, it’s not only science. It’s always a pleasure attending the performance and trying to capture it. That’s equally important, especially in audio, the artistic performance is as important as the science. It is about listening and fine tuning and getting the right thing out of the performance to listen to, so you can recreate the experience and give others the chance to appreciate the art too.