After a variable-density image of a target object is inputted and edge pixels representing the contour line of the target object are extracted from the image, the directions of these edge pixels are calculated, while straight-line portions in the contour are labeled according to their directions. Those of the edge pixels, which have the same direction as that of any of the straight-line portions, are assigned the same label. Line segments are then extracted as assemblies of those of the edge pixels which are continuous on the image and have a same assigned label. For labeling the directions of the straight-line portions, a histogram may be prepared, based on the numbers of edge pixels having different directions. Peaks are then extracted from the histogram and the labels are set to the directions corresponding to these peaks. Alternatively, labels may be set to each of expected directions of straight-line portions of the contour of the target object, the same labels being assigned to edge pixels having the same direction.