A memory controller controls so that the number of maximum macro blocks for which inter coding is performed in one image frame is set to a smaller value than the number of macro blocks configuring the image frame, the capacity of the reference image memory is reduced to capacity for the maximum macro blocks for which inter coding is performed, macro block positions for which intra coding and inter coding are performed in the next frame are determined based upon image information and the quantity of codes in the coding of a present image and reference image macro blocks equivalent to only macro block positions for which inter coding is determined to be performed in the next frame are stored in the reference image memory.