Chroma upsampling is performed by setting a chroma of a target pixel according to luma of neighboring pixels. When a luma of the target pixel differs from the luma of both of the reference pixels, the chroma of the target pixel is set to a value of (k*C1+(m-k)*C3+m/2) shifted n bits to the right. In this formula, n is an integer, m is equal to 2.sup.n, and k is equal to {0.about.m, min(abs(abs(Y1-Y2)*k-abs(Y2-Y3)*(m-k)))}. Since luma are considered, and a precise calculation is used in this method, the calculations required are simplified and reduced, and video quality is improved.