An image display apparatus includes a diaphragm (9b) arranged at a pupil position. The shape of an aperture of the diaphragm (9b) is similar to the shape of a light source image formed by an illuminating optical system at the pupil position. An area varying mechanism (90) varies the area of the aperture of the diaphragm (9b) in response to a focal length varying operation of a projection lens (9). This prevents an excessive opening from being formed around a light source image cast by the illuminating optical system, thereby preventing the deterioration of displayed images due to unwanted light that may pass through the excessive opening.