In a variable displacement swash plate type refrigerant compressor (10), a stopper (28) is provided for setting an initial angle of the inclination angle of a swash plate (23) when a drive shaft (19) is not driven. When the drive shaft (19) is driven by a driving power source of the compressor (10) under a non-operation condition of a refrigerant circuit, the stopper (28) is moved to permit the inclination angle of the swash plate (23) to be reduced from the initial angle in response to increase of compression work of the compressor (10) so as to suppress the increase of the compression work to save the driving power for the compressor (10).