An apparatus for canceling an echo signal in a mobile terminal of a mobile communication system. A double talk detector (DTD) receives a first signal by canceling an estimated echo signal from a signal received through a microphone, outputs the first signal, and outputs the first and a second signal comprising a background noise signal and a residual echo signal during a non-double talk. An Auto-Regressive (AR) analysis and inverse filtering unit receives the second signal from the DTD, and whitens the second signal. A pitch analysis and inverse filtering unit receives the whitened signal, and cancels a pitch value remaining therein by performing pitch analysis and inverse filtering on the whitened signal. A noise canceller receives the pitch-cancelled whitened signal and the first signal output from the DTD, canceling a residual echo signal and a background noise signal from the first signal using the pitch-cancelled whitened signal.