A system and method for localization of sounds in 3-D space using Head Related Transfer Functions (HRTFs) are disclosed herein. A plurality of audio signals and/or sound sources, or "voices", represented by data, such as one or more audio files or streaming audio data, is transmitted from an audio source, such as a compact disc player, to an audio processing system. The audio processing system, in one embodiment, prioritizes the voices based on characteristics of the voices, such as the relative listening benefit of sound localization of a particular voice. In one embodiment, a finite impulse response (FIR) filter is used to apply a HRTF to the voices. The audio processing system distributes up to a fixed number of coefficients among the voices, where the distribution of the coefficients, in one embodiment, is based on the priority of each channel. Those voices having a higher priority than other voices could have a greater number of coefficients assigned to them than the lower priority voices. The voices, in one embodiment, are processed by the FIR filter using the assigned number of coefficients for each voice. The results of the FIR filter can then be further processed, if needed, and output to a set of speakers, a storage device, and the like. By distributing a fixed number of coefficients among the plurality of voices, a dynamic tradeoff between sound localization quality and processing effort and/or time can be achieved based on the voices' priorities.