A system and method for simultaneous construction of a corresponding CAD model and physical model, wherein the CAD model comprises a plurality of CAD representations each corresponding to a physical component part that is used to construct the physical model. During construction of a physical model using individual component parts, the CAD system can identify a given component part and retrieve its CAD representation from a CAD library. In addition, the CAD system allows the user to build a CAD representation of a given component part if its CAD representation is not stored in the CAD library. After the CAD representation of the component part is generated, the CAD system will track the motion (position and orientation) of the part as it is maneuvered into a desired position in the physical model. The position and orientation, as well as the CAD representation of the component part as it exists in the physical model is saved in a CAD model database. During assembly, the CAD system can render an image of the CAD model as each component part is placed in the physical model.