A method, system and software are disclosed for correcting defects formed in a physical medium of an original image. Multiple scans of the original image are recorded, where the multiple scans have different properties. For example, the angle of the light incident to the physical medium or the properties of the light may be changed between scans of original image. The multiple scans can be used to generate a reference image from which defect corrections are made. Alternatively, a reference image can be generated directly from the original image. The multiple scans can also be used to determine the degree of defectiveness and/or an estimate of the signal strength of each portion of the original image. A decision is made on whether or not an image portion having one or more defects should be corrected, where the decision can be based on an evaluation of the potential benefit compared to the potential damage caused by correction of an image portion. In one embodiment, the potential benefit is proportional to the degree of defectiveness, while the potential damage is proportional to the image information that may be removed by correction. If the decision is made to correct an image portion, a variety of methods may be implemented to correct the image portion, such as cloning information from non-defective image portions surrounding the defective image portion. The present invention finds particular use in image capturing systems, such as flatbed scanners, photocopiers, facsimile machines, and the like.