Projects

Impulse response modelling has been a well studied topic in the acoustics and audio signal processing fields. Advancements in deep learning have led to such methods being applied to room impulse response (RIR) generation, and has yielded some interesting and comparable results to traditional methods.

Recently, the development of Neural Radiance Fields NeRF in computer vision has been a breakthrough in neural representation learning, and has been good at interpolating unseen views within a complex 3D scene trained on sparse input views. NeRF has been adapted to acoustic scene representation and has shown to produced RIRs at unseen source-listener pair positions in Neural Acoustic Fields NAFs. This seminar research paper reviews the contribution, affordances, and performance of several NeRF-based and non-NeRF-based deep learning-based RIR generation methods.

Velvet noise, sparse pseudo-random noise, in the synthesis of impulse response tails proves to be simple and computationally efficient. This project is the implementation and audio demo of late reverberation synthesis using filtered velvet noise.

Binaural signals can be digitally constructed with head-related impulse responses (HRIR), a set of laboratory-measured pairwise signals which encode the sound source position in relation to the listener. An impulsive sound is played from a loud speaker at some distance, horizontal, and vertical angle in an anechoic chamber and HRIRs are recorded by two in-ear microphones in a dummy head or human test subject. Thus a personalized set of HRIRs with radius, azimuth angle, and elevation angle properties are captured for each subject. This project explores different supervised machine learning methods to predict the associated azimuth angle of an unknown subjects’ HRIR using machine learning. The results of linear regression, polynomial regression, and multilayer perceptron methods are compared and are shown to be promising for this application. The motivation for the project was to deepen my understanding of spatial sound, binaural techniques, auditory modelling, and machine learning applications for audio.

Sound plays a complex and important role in how humans infer the world. This project explores the landscapes that can be created with spatialized audio in the Unity game engine, and the degree to which sounds can create presence of unseen beings and psychologically immerse inhabitants in a virtual environment. The application of realistic sound for virtual environments is important and can be manipulated to create surprising and striking experiences. It is this edge of human experience that I am excited to expand using sound. This project used the native audio spatializer SDK in Unity and the Resonance Audio SDK developed by Google.

The four symbols sound zoo is a soundscape created in Unity, in which the inhabitant moves around a perforated circular room, guided by sound emitting from outside of the room. Four mythological creatures from Chinese lore, the Azure Dragon of the East, the Vermillion Bird of the South, the White Tiger of the West, and the Black Tortoise of the North are placed into different spatial environments ocean, sky, forest, and cave. Neither the mythological creatures nor their habitats appear visually, but based on the direction of the sound, reverb effects, and movement of the sound through space, the presence of four large animals is felt by the inhabitant.

"Where do we go when we fall asleep" is a virtual reality experience that experiments with the psychological liminal space between awakeness and dreaming. An experience where the dreamer (user) is physically laying down on a bed and experiences multiple dreamlike worlds. When the dreamer gets up from the bed the dream vanishes and they remain in the virtual bedroom.

"Welcome home, Computer" imagines how digital entities such as computer generated character Lil Miquela, with over 2.7 million Instagram followers, and Apple virtual assistant Siri exist in space just for them. How a human intruder would experience their world? This project is an immersive reality experience built in Unreal Engine. Distance, movement, and communication within a home environment exclusively for virtual entities are altered to imagine the machine subjectivity of spatial inhabitation and experience.