A three-dimensional data generating apparatus of the present invention comprises an obtaining portion for obtaining plural image data representing images obtained by photographing an object from different viewpoints, the plural image data each representing respective one of the images; a first detector for detecting corresponding points of the images by a gradient method; a second detector for detecting corresponding points of the images by a correlation method; a selector for selecting either one of the first detector or the second detector; and a generator for generating three-dimensional image of the object based on the corresponding points detect by the selected detector. The present invention is capable of generating highly reliable three-dimensional data irrelevant to a type of an object.