Two signals, which are mutually correlated, are subjected to a principal component analysis and converted into two signals being put in an orthogonal relation, thereby generating two signals being non-correlated. Those two signals are reproduced by speakers, and the voice generated from the speakers are collected by microphones. The cross spectra of a signal as the result of subtracting an echo canceling signal from a voice collected by each microphone, and a voice before it is generated from the speaker, are obtained. Those cross spectra are ensemble-averaged for a predetermined period of time, and inverse Fourier transformed, thereby producing impulse response estimation errors of each filter. Impulse responses of those filters are updated so as to cancel those impulse response estimation errors.