An image data compression device comprises a quantizing section for quantizing image data with a quantization step changing in accordance with a quantization parameter, a FIFO buffer section for buffering the quantized data corresponding to a plurality of frames, an encoded data generating section for generating encoded data obtained by encoding the quantized data retrieved from the FIFO buffer section asynchronously with writing thereto, and a rate control section for controlling a data size of the encoded data by changing the quantization step. The rate control section finds a predicted data size of the encoded data of the precedent frame from the size of the quantized data of the precedent frame, and sets or releases an upper limit threshold of the quantization parameter based on the number of assignable bits assignable to a compressing process or the number of remaining frames. When setting the upper limit threshold of the quantization parameter, the quantization parameter is found so as to be equal to or less than the upper limit threshold of the quantization parameter using the predicted data size.