A system and method of placing constraints on a deformation map. The method includes the acts of generating a deformation map between two images, identifying a defined structure in one of the images, applying the deformation map to relate the defined structure from the one image onto the other image to create a deformation-based defined structure, modifying the deformation-based defined structure, and updating the deformation map in response to the step of modifying the deformation-based defined structure.