A document image processing apparatus. A layout analysis section analyzes a layout structure of an input image. A layout information memory section stores layout information representing a relationship between the layout structure and a corresponding area in the input image. An image display section displays the corresponding area in the input image according to the layout information. An indication input section inputs an indication to modify the corresponding area in the input image displayed. A modification section modifies the corresponding area in the input image and the layout structure of the corresponding area in the layout information according to the indication.