An image processing system interpolates image data of an image array by ascertaining shift invariant points and non-shift invariant points within the array. The average illumination and the second order derivative are determined for the shift invariant locations. The second order derivative and the intensity at the non-shift invariant locations for each of the non-shift invariant points are estimated. The color data for each color element is determined from the image data and second order derivative. The second order derivative is multiplied by a scaling factor for selectively smoothing and sharpening the second order derivative. The color data values of adjacent color element to enhance a resolution of the image data.