A method of selecting a plurality of lithography process parameters for patterning a layout on a wafer includes simulating how the layout will print on the wafer for a plurality of resolution enhancement techniques (RETs), where each RET corresponds to a plurality of lithography process parameters. For each RET, the edges of structures within the simulated layout can be classified based on manufacturability. RETs that provide optimal manufacturability can be selected. In this manner, the simulation tool can be used to determine the optimal combination of scanner setup and reticle type for minimizing the variation in wafer critical dimension (CD).