A system and method are provided for analyzing layout patterns via simulation using a lithography model to characterize the patterns and generate rules to be used in rule-based optical proximity correction (OPC). The system and method analyze a series of layout patterns conforming to a set of design rules by simulation using a lithography model to obtain a partition of the pattern spaces into one portion that requires only rule-based OPC and another portion that requires model-based OPC. A corresponding hybrid OPC system and method are also introduced that utilize the generated rules to correct an integrated circuit (IC) design layout which reduces the OPC output complexity and improves turnaround time.