In an edge detecting apparatus and a noise eliminating apparatus, low AC components are detected from frequency components obtained by orthogonally transforming an image signal in rectangular units. Edge components are included in the low AC components among the frequency components. Based on this characteristic, based on the result of detecting the low AC components, in a rectangular-unit image corresponding to the low AC components, it is determined whether edge components corresponding to the low AC components are included in a rectangular unit image, and an edge direction is detected if the edge components are included. This enables ensured detection of edge components in any portions of a screen.