The present invention provides an auto focus system capable of having an AF area which is a target range of auto focus (AF) automatically follow a follow-up target subject as a focusing target, wherein the system changes a size of a range for capturing an image of the follow-up target subject as a reference pattern image out of the range of a photographing image according to a target subject distance and a focal length of a photographing lens so as to appropriately perform detection of the image of the target subject and automatic follow-up of the AF area even in the case where the size of the image of the target subject in the photographing image of the camera changes such as the case where the distance between the camera and the follow-up target subject changes or the case where the focal length of the photographing lens (optical system of the camera) changes.