An apparatus and a method for automatically inspecting a defect by an electron beam using an X-ray detector. The composition of a defective portion is analyzed with higher rapidity and the cause of the defect is easily and accurately determined based on an X-ray spectrum. The X-ray spectrum and the image of foreign particles formed on a process QC wafer are registered as reference data, and the defects generated on a process wafer are classified by collation with the reference data. The use of both the X-ray spectrum and the detected image optimizes the operating conditions for X-ray detection. A defect of which the X ray is to be detected is selected based on the result of classification of defect images automatically collected, and the defect is classified according to the features including both the composition and the external appearance.