First similarity values along at least four directions are ascertained within a local area containing a target pixel and weighted averaging is performed by adding the pixel values of pixels around the target pixel value to the pixel value of the target pixel, adding weight along a direction having a small first similarity value (along a direction manifesting a high degree of similarity). By incorporating the pixel value level differences among a plurality of pixels on adjacent lines extending adjacent to the target pixel into the first similarity values, it becomes possible to effectively remove jaggies that are difficult to eliminate in the prior art. Furthermore, by making a judgment on degrees of similarity by incorporating color information such as characteristics differences among different color pixels, a more accurate judgment can be made with regard to the image structure to enable very accurate direction-dependent low-pass filtering.