The term "Virtual Reality" -also known as Artificial Reality- was first introduced by (Jaron Lanier) in 1989 referring to 3D interactive environment produced by a computer system. There are also two terms commonly used "Virtual Environments & Virtual Worlds" which refer to the worlds existing entirely in the memory of a computer.
The virtual reality interfaces try to simulate three major systems of the human body:
The Visual system.
The Tactile system.
The Auditory system (which is the main focus of this project):
Our sense of sources of sounds comes from three main effects:
Interaural Time Differences (ITD): which is the measurement of differences between the time in which the sound waves reach the left ear to the time in which the same sound waves reach the right ear.
Interaural Intensity differences (IID): which represents the measurement of the level of intensity which decreases with distance.
Hearing Shadows : which is the effect of the high frequency of the sound that is created by the objects
between the listener and the source of sound.
Note : In this page I only explained the auditory
system which is the most relevant to the project, for more information about
the others there are many resources out there on the: Visual systems and the Tactile systems.
An Introduction to 3D Sound
To be able to create 3D sound we have to understand the methods that the human being locates sounds sources. At the end of the 19th century, Lord Rayleigh contributed a study addressing this point, and he was one of the pioneers in this field. His studies and others' led to the main points mentioned above. Here are some explanation for them:
Interaural Time Differences (ITD):
The interaural time differences are the least complex of the directional cues that the human auditory system depends on to locate sound sources. As mentioned above, the ITD is the difference in time-of-arrival between the left and right ears depending on the position of the sound source. In the following diagram the sound wave will reach the right ear before a considerable amount of time in which it will reach the left ear.
The ITD is useful unless the sound source is in one of the following positions: Directly in front, directly behind, directly above or directly below). In these positions the sound will reach the left ear and the right one at exactly the same time.
Reproducing 3D sounds:
One of the theories that explain reproducing 3D sound in the virtual world states that a sound-producing device should be placed for every sound source to be created in the virtual world. It's obvious that this theory is so complicated and too costly as a sound source should be processed each time the user or one of the virtual objects move in the virtual world. To overcome this problem the time differences needed for determining the source can be processed automatically by our auditory system.
Sound level:
Researches found that the sound frequency level plays a major role in determining the sound source using the ITD. For high frequencies, the listener's head creates the "shadow effect" on the sound waves, which is another problem to be dealt with. Sounds with frequencies higher than 4000 Hz suffer from the shadow effect more than smaller frequencies. The reason for this is that the wave length of high frequencies is relatively short which causes more crashes between these waves and the listener's head which obstruct these waves from reaching on of the ears. For these kinds of sounds the IPD is used to determine the sound source.
Note: Research showed that for frequencies between 1000 Hz and 4000 Hz, the shadow effect depends mainly on the listener's perceptual threshold, in other words it differs from one listener to another.
The engineering basics of the ITD:
A huge amount of research literature tried to determine the actual delays required for reproducing ITDs. Probably the most accurate set of equations that represent this process is the following:
For frequencies equal to or less than 4000 Hz, the ITD is calculated as follows: ITD = ( 3 × r × sin Ø ) ÷ S
Where: S : the speed of sound. Ø : the sound source's deviation angle (given that " 0° " degree is the angle directly behind the listener's head, the " 90° " is the left ear's deviation angle and the " 270° " is the right ear's deviation angle). r : the radius of the listener's head given in cm.
For frequencies greater than 4000 Hz,the ITD is calculated as follows:
ITD = ( 2 × r / S ) × sin Ø
To fully understand these formulas, let's assume the followings:
The speed of sound
S = 343 m/sec
The listener's head radius
r = 9 cm
The sound source's angle deviation of the listener's head
Ø = 45°
In the given conditions, the ITD for frequencies less or equal to 4000 Hz is calculated as follows:
ITD = (3 × 9 × sin 45) / 343 = 0.05566 sec
This means that a delay of (0.05566 sec) has to occur between the instance the sound reaches the left ear and the instance the sound reaches the right ear.
To simulate this process in our virtual environment, we have to generate a sound from the right sound speaker and then generate the same sound from the left sound speaker after 0.05566 sec. It is clear that if the sound source was located on an angle deviation of " 45° " on the left of the listener, then the angle "Ø " that should be used in the formula above is "315° " and the result will be the same but in negative value.
Notes:
Some sources adopted a different formula for calculating the ITD:
ITD = 0.0008 × sin ø
The ITD is most helpful in locating the sound source in the horizontal level for low frequencies and is rarely used for high frequencies where the IPD is more used.
I'm not going to go through the IPDs and the spectral cues because they are a little more complicated, but if you have any questions about them then check these links.
