In document recognition processing for accurately recognition a document image at high speed, a shape feature is extracted from a component included in an input document image, and the degree of similarity of shape between the document image and the registered document is calculated based on the extracted shape feature. When it is determined that the component of the document image has color information, the color information is extracted from the document image, thus calculating the degree of similarity of color information from the document image. The degree of similarity of shape is corrected based on the degree of similarity of color information, thus determining that the registered document is similar to the document image.