With the use of a vector quantization technique, an index image, which is obtained by replacing an image on a block-by-block basis, each block consisting of a predetermined number of pixels, with any of a plurality of different pixel patterns and representing the pixel patterns by index values of the respective pixel patterns, is decoded by referring to a code book containing the pixel patterns and the index values of the pixel patterns in association with each other. When decoding an index image on the block-by-block basis into pixel patterns corresponding to the index values, the pixel pattern of each block is scaled, and additionally pixels near a contour in the pixel pattern after scaling are redrawn based on contour vector information corresponding to the pixel pattern.