A gradient-based image enhancement and restoration method and system which applies an orientation-isotropy adaptive filter to the gradients of high structured regions, and directly suppresses the gradients in the noise or texture regions. A new gradient field is obtained from which image reconstruction can progress using least mean squares. The method generally comprises: inputting image data; calculating image gradients; defining the gradients as having large or small coherence; filtering the large coherence gradients for edge enhancement; suppressing the small coherence gradients for noise reduction; assembling an enhanced gradient field from the filtered large coherence and suppressed small coherence gradients; and optimizing the assembled gradient field into a restored image.