An apparatus of evaluating a layer matching deviation based on CAD information of the invention, is provided with means for storing CAD data and a function of displaying to overlap a scanning microscope image of a pattern of a semiconductor device formed on a wafer and a design CAD image read from the storing means and a function of evaluating acceptability of formation of the pattern by displaying to overlap a pattern image of the semiconductor device formed on the wafer and the design CAD image of the pattern, in addition thereto, a function capable of evaluating acceptability of formation of the pattern also with regard to a shape and positional relationship with a pattern formed at a later step by displaying to overlap a design CAD image of the pattern formed at the later step.