- Course period: Third Quarter
- Course year: 1,2
- Course type: Elective
- Credit: 2
- Coordinator: Julián Villegas
- Instructor(s): Julián Villegas, Michael Cohen, and Jie Huang.
The purpose of this course is to study the fundamentals of spatial hearing and its application to virtual environments. By using two ears, humans among other species are able to determine the direction from where a sound is being emitted in a real environment. For virtual environments (e.g., movies, games, recorded or live concerts) it is desirable to provide the spatial cues found in nature to increase the realism of a scene. Besides reviewing the underlying theories of spatial hearing, this course focuses on practical implementations of binaural hearing techniques, so the course is rich in hands-on exercises, assignments, and projects, mainly based on the Pure-data programming language (Official site, Japanese version).
- Students who pass this course will understand the basic underlying mechanisms of spatial hearing, as well as the literature and terminology on this topic.
- Given some application constraints (real-time, computing power, etc.) students at the end of the term should be able to decide which of the presented techniques is best for creating the 3D aural illusion.
- Upon completion of this course, students should be able to successfully implement virtual 3D sound environments based on head-related transfer functions (HRTFs) and multi-speaker systems.
|1||NA||Introductions and Pd||Syllabus review, introduction to 3D sound, Explanation of mid- and final-term project, antiphonal music, applications, introduction to Pd programming, connections, architecture, subpatches, abstractions, DFS, Jargon, Audio objects||Characteristics of sound sources (Begault, pp.17–25)||Genuit Puerpera Regem, Unknown||800||Inverse distance law, violin section|
|2||NA||Quantification of sound||Power and intensity, SPL, SIL, sound, weighted sound pressure level, sound propagation, directivity, Frequency, Decibels, Fourier Analysis, Time Domain vs. Frequency Domain, wavelength and antropometry||Perception (Begault, pp. 25–31)||Spem in Alium, Tallis||1573||Sound level meter lab|
|3||NA||Spatial hearing and psychoacoustics||Huggins pitch, Virtual Auditory Space: Context, Acoustics, and Psychoacoustics, Intensity vs. Loudness, SL vs. phons, Frequency vs. Pitch ,mel, Spectrum vs. Timbre, Threshold of hearing, limens, Masking, Freq. Weighting, Cognition||Interaural Time and Intensity Cues (Begault, pp. 31–36)||The battle of Vitoria, Op. 91, Beethoven||1813||Binaural recordings|
|4||NA||Binaural difference cues I||Audiometry, Interaural Intensity Difference (IID), Interaural Time Difference (ITD), group delay, Thresholds, Head shadowing, Lateralization, Cone of confusion, Physical Basis of Lateralization||Widely separated ears; Auditory demonstrations 68 — 74||Tuba Mirum section of Requiem, Berlioz||1837||Hearing screening|
|5||NA||Binaural difference cues II||ITD Envelope Cue, Perception of Lateralization, Externalization,Head Movement, Localization Blur, Front/Back Confusion,||Spectral Cues Provided by the Pinnae (Begault, pp. 40–49)||Symphonie Pathetique no. 6, Tchaikovsky||1893|
|6||NA||Head related impulse responses||Transfer functions, linear systems, all-pass filter, minimum-phase systems, Capturing, Equalization, Data Reduction and Computational Shortcuts.||Environmental Context Simulation (Begault pp. 138–145)||The Unanswered Question, Charles Ives||1908|
|7||NA||Room impulse responses||measurement of room impulse responses, convolution, reverberation techniques. Delivery mid-term project||Distance Cues (Begault, pp. 69–82)||4'33, Cage||1952||HRTF-based spatialization in Pd|
|8||NA||Motion and distance perception I||Doppler effect, Doppler shift, coloration, acoustic propagation constant, acoustic attenuation,acoustic phase, group velocity, transmission loss, absorption, divergence and refraction, scattering loss. Motion parallax||Moore & King, Auditory perception: The near and far of sound localization||Poème électronique, Varèse||1958||Capturing room impulse response|
|9||NA||Motion and distance perception II||Free field, far field, direct field, diffuse field, stationary waves, nodes, antinodes, reverberant field, diffuse field, Echo, Reverberation, reverberation time,||The Precedence Effect (Begault, p. 36)||Kontakte, Stockhausen||1960|
|10||NA||Special topics in sound spatialization||The Precedence (Haas) Effect, Speech intelligibility, dealing with multiple sources, localization in noise, computational source localization.||OSC protocol definition||HPSCHD, Cage||1969||Create a Doppler effect in PD|
|11||NA||Headphone techniques||head tracker, open vs. closed headphones, diffuse field equalization, free field equalization||Loudspeakers and cross-talk cancellation (Begault, pp. 174 –177)||Turenas, Chowing||1972|
|12||NA||Loudspeaker techniques I||Loudspeaker definition, Panning, VBAP, Dolby Surround, Wave field synthesis||Surround versus 3-D Sound (16 – 17)||Répons, Boulez||1984|
|13||NA||Loudspeaker techniques II||Loudspeaker lab, Ambisonics, transaural, Wave-field synthesis, holophony, Encoding and Decoding Multi-channel Audio, recording and reproduction, Dirac||Aeronautical Applications (Begault, pp. 188 — 190)||Post-praeludium per donau, Nono||1987||Transaural audio in PD|
|14||NA||Workshop on final project||NA||Dissecting the Brain with Sound||Versions/Stages: II. Dionysus||1993|
|15||NA||Applications I||Binaural navigation systems, information display,||Music to your ears||Prelude, Op. 3 No. 2, Rachmaninoff||2009|
|16||NA||Applications II||Music spatialization, Course evaluation, Presentation of final projects||None||Golden Music, Villegas||2010|
- Durand R. Begault, 3-D Sound for Virtual Reality and Multimedia, Academic Press, 2000. (online)
- Various materials prepared by the instructors
This class does not have prerequisites, but it is recommended that students be familiarized with Pure-data programming paradigm (currently taught in ITC02: Introduction to Sound and Audio), and general audio signal processing techniques. These are some classes that students are encouraged to take concurrently or before this class:
- ITC02 Introduction to Sound and Audio
- ITA07 Advanced Signal Processing
NOTE: Students are expected to read their emails frequently.
- Course website: http://arts.u-aizu.ac.jp/spatialHearing/
- J. Blauert, Spatial Hearing: The Psychophysics of Human Sound Localization. MIT Press, 1997.
- Bregman, Albert S., Auditory Scene Analysis: The Perceptual Organization of sound. Cambridge, Massachusetts: The MIT Press, 1990 (hardcover)/1994 (paperback).