In the conversion of a video signal from interlaced to progressive scanning, the value of each pixel on an interpolated scan line is calculated by a procedure that includes calculating similarity values for pairs of pixel blocks located in point-symmetrical positions on opposite sides of the interpolated pixel; deciding whether similar edges are present in corresponding positions in the two pixel blocks constituting each pixel block pair; selecting an interpolation direction corresponding to the most similar pixel block pair among the pixel block pairs in which the similar edges are present; and using the pixels disposed at or closest to the centers of the two pixel blocks in this pixel block pair as reference pixels. Restricting diagonal interpolation directions to pixel block pairs in which similar edges are present improves the accuracy of the interpolation direction.