A method for inspecting a patterned surface employs reference data related to the pattern to provide a map for identifying regions which are expected to generate equivalent images. These regions are then compared in an image-to-image comparison to identify possible defects. In a first implementation, the regions are related by local symmetry operators. In a second, disjoint corner features or other features are classified and similar features compared.