AudioVirtualizer (PC)

The AudioVirtualizer is a VR audio visualiser for the HTC Vive and Oculus Rift headsets. A participant explores three different Op Art inspired scenes revealing different mappings between musical input and visual output. In each scene the user can select from a range of different soundtracks, or live microphone input, to investigate how the visuals respond to the sound. The graphics are based on drawings and 3D models by Adinda van ’t Klooster. The audio engine tracks low- as well as mid-level features in the sound, allowing more complicated visual returns than previous FFT filterbank based audiovisualisers. The mid level audio feature extraction explored here includes the detection of onsets and beats, continuously held pitches, and aspects of timbre (our Unity library for this is freely available at: https://github.com/sicklincoln/MusicalMachineListeningUnityPlugin).

The AudioVirtualizer begins with an entrance space where navigation can be practiced. Using the large circular touch pad on the VIVE controller, or the joystick on an Oculus controller one can move forwards (up), to the right (right), the left (left) and backwards (down). Going up and down in the 3D space is achieved by pressing the trigger as well as keeping the finger on the up or down position. Teleportation between scenes is effected by navigating to a red dildo-like sculpture and selecting it. A choice of four music tracks, or live microphone input, is available. Pressing the menu button on the controller overlays the scene with five sculptural objects, each of these selects a different sound source. Pressing the menu button again will remove the options and play the current selection.