An angle de-skew device and the method thereof are used in an image processing system to rotate a skewed image with respect to a reference point as a rotated center. First and second accumulator circuits respectively are used to accumulate first and second offsets for the X coordinates and Y coordinates of the first pixel in the skewed image, and respectively output the sums into the image processing system. A counter is connected to the two accumulator circuits to control the number of accumulation times.