An image processing apparatus includes a size acquiring unit that acquires a block size of an orthogonal transformation from encoded data of an image compressed by performing the orthogonal transformation and quantization for each block; a filter selecting unit that selects a filter for each block, the filter reducing an encoding distortion from each block of a decoded image; and an encoding-distortion reducing unit that reduces an encoding distortion by the filter on all pixels within the block. The filter selecting unit selects the filter so that a strength of the filter strengthens by increasing of the block size, or selects the filter so that the number of taps of the filter decreases by reduction of the block size.