When you're looking at the road, you don't hear very well the passengers/driver by your side. You have to turn your head to them to hear them good, yes it is about "local voice".
Update : Might be related to the left/right hear in Stereo mode. Somehow one the ear is transmitting on very low volume (the one at the opposite of the sound emitter location) while the other is also on a lower volume but not as low as the other ear. Then when looking directly at direction emitting the sound both ears return to "100% volume".
What lots of games forgot (so not just yours) is that even if a sound come from the left, that doesn't mean left ear got 100% volume while right ear is 10% volume. Which provoke this unbalanced audio effect which i find disturbing (among the fact that you barely ear anything).