A method and system for smoothing a frame to remove jagged edges are presented. The method and system generates a smoothing filter with consolidated pixels. Edges within the smoothing filter are analyzed to select an edge direction used for smoothing. A smoothed pixel is generated based on a normalized linear combination of a first edge end pixel, a second edge end pixel and center entry of the smoothing filter. Subtle structure checking can be used to determine whether to use the smoothed pixel in place of the current pixel.