AF is started in response to the depression of a shutter button 61 by the user, and a plurality of images are obtained in succession with a time interval while a first image and a second image are obtained in this order with a time interval by a CCD 11. With respect to the AF area, of the first image, set so as to be used for the evaluation of the focus condition, the number of pixels belonging to a region in a predetermined color space is counted, and with respect to a plurality of color evaluation areas (the area corresponding to the AF area and a plurality of areas in the vicinity thereof) set for the second image, the number of pixels belonging to the region in the predetermined color space is counted. Then, the amount of shift between the color evaluation area where the number of pixels closest to the number of pixels counted with respect to the first image is counted, and the AF area is detected as the object movement amount.