A hybrid audio encoding technique incorporates both ABR, or CBR, and VBR encoding modes. For each audio coding block, after a VBR quantization loop meets the NMR target, a second quantization loop might be called to adaptively control the final bitrate. That is, if the NMR-based quantization loop results in a bitrate that is not within a specified range, then a bitrate-based CBR or ABR quantization loop determines a final bitrate that is within the range and is adaptively determined based on the encoding difficulty of the audio data. Excessive bitrates from use of conventional VBR mode are eliminated, while still providing much more constant perceptual sound quality than use of conventional CBR mode can achieve.