Hey steemians,
This is a post on work I've been researching and implementing for the last 2 years or so. I've been an audio enthusiast from very young, and finding these techniques has brought a load of fresh ideas about creating interesting spaces using sound.
The research into 3D spatial audio has been booming recently, and with what is considered the 'first year of VR' behind us, there has been an even greater push to understanding these techniques. Whether using headphones or large 3D multichannel speaker arrays, spatial audio is a powerful technique to bring interesting aspects to you composition. The psychoacoustic and perceptual principles that underly how the human brain determines distance and location is a very interesting field of research and I would encourage audio engineers of all levels to look into it. I'll add some books at the bottom that readers can have a look at.
Personally, I've been involved in building a couple art installations which really would've not been the same without an added audio dimension. It's a whole new way for an artist to create entirely fictional spaces that a viewer can get lost in, and there is ample technology available for you to get started in creating immersive experiences!
Here are some photos of an installation I worked on last year,
There is so much info and research on this topic, I will need to extend this post into a 4-5 part series that will go through the main techniques, aswell as current systems that implement these techniques and how you can start mixing immersive audio! So without further addo, let's jump in!
Current immersive audio systems
I want to point readers in the direction of some current implementations first, hopefully after listening and learning briefly about the capabilities of these systems, will incite an interest to go and study how to implement these techniques.
The 4D soundsystem is truly an awe inspiring rig, with 36.6 omnidirectional sound and the ability to move and modulate audio around a dancefloor. Situated in Budapest, Hungary It has the ability to project holophonic sound sources which means it can put sounds right by the listener's ear, or further back in the distance. It removes the necessity for a predetermined 'sweet spot' and allows for a truly flexible, interesting listening experience where the listener may explore the space and find sounds hidden in different parts of the room. This system has been used by many music producers, recognizable names such as Stimming and Max Cooper.
Envelop is another project that was started by Chris Willits in San Franscisco. This was a crowd funded project that has built a mobile immersive system. It makes use of the Higher Order Ambisonics algorithm to pan the audio around the space. The main difference between this system and the 4D system is that the speakers surround listeners who are located at the centre. This obviously means for listeners to perceive the location of an audio source, they would need to be sitting in the centre of the array.
The sound dome constructed in the fields of Pischeldorf, Styria makes use of Higher Order Ambisonics to place audio around the edge of the dome. Once again, this setup requires the listeners to be located at the centre of the dome, or 'sweet spot', of the dome to achieve a decent effect. As can be seen in the video below, the materials used to make the dome play a big role in the quality of the sound. Generally, lots of reverberation and non-absorbent materials make it harder for the immersive audio to be felt.
So we have a dome, but what about a cube? The Cube is housed at Virginia Tech in the States. It makes use of Wave field Synthesis and uses a Dante framework to allow for one of the largest immersive speaker systems in the world. This rig is one of the more advanced systems out there, similar to that of the 4D soundsystem. It has 145 audio channels, and they intend to add more speakers.
Software that is currently available to for use
There are many software implementations that are freely available that may be used to achieve audio immersion. Don't stress if you don't have the funds to fork out millions to buy a large scale speaker array, most of the spatial audio effects used in these systems can be monitored binaurally using headphones HRTF filters (Head Related Transfer Functions) and most multiplayer gamers should already know about these. The effect is really believable and I would greatly encourage audio enthusiasts to check out these tools! There are a number of youtube videos available that use headphones to give the viewer an immersive experience.
Some open source software used for spatial audio is listed below,
- Ambisonics Toolkit - This toolkit is available in two flavours. Native Reaper JSfx plugins, or SuperCollider
- SoundScape Render - This tool has many more spatial audio algorithms available for use.
- IRCAM's SPAT - This tool is available for use with MaxMSP, and also has many algorithms available for use.
- Expansion of Meyer Sound's SpaceMap called MIAP - This tool has been developed outside of an academic context and has shown extremely promising results. It's available as an external for both MaxMSP and Pure Data.
I personally can vouch for the Ambisonics Toolkit. If you understand how to use a DAW, it gives a very cheap and easy way to experiment with spatial audio on headphones and would really recommend checking it out. Follow their tutorials to learn how to set up your Reaper project to handle the 4 channel configuration required for the Ambisonics encoding.
This first post should give some indication to the current immersive audio technologies out there, and I hope my fellow audio heads will go and check this stuff out. It's really exciting stuff!
Here are some books and articles that you can read if you'd like to learn a bit more!
Next up, 'What exactly is audio immersion and where did it start?"