A method of calibrating image luminance values is provided. At first, a calibration chart is scanned, and a dark actual luminance value and a white actual luminance value are obtained accordingly. Next, a document is scanned, and a scan luminance value is obtained accordingly. Then, a white ideal luminance value, a dark ideal luminance value, an ideal luminance value and a predetermined calibrated luminance value are provided. Next, a calibrated luminance value is determined according to the dark actual luminance value, the white actual luminance value, the white ideal luminance value, the dark ideal luminance value, the ideal luminance value, the predetermined calibrated luminance value and the scan luminance value.