A telephone includes a transmit channel and a receive channel, each including a bank of sub-band filters having a VAD coupled one to each sub-band filter. Each VAD measures the spectral energy in a sub-band, compares the spectral energy to a first threshold, and produces an output signal representative of whether or not the first threshold is exceeded. The voice activity detector also includes a threshold circuit for calculating a dynamically adjustable noise threshold based upon averaged measured spectral energy. A wide band or system VAD monitors echo canceling circuitry to detect voice activity and double talk. A dynamic balance control circuit maintains the transmit and receive channels in a predetermined energy ratio during double talk.