A regulator circuit receives a power supply and provides a regulated power supply output suitable for integrated circuitry. It has a controllable current source circuit, a controller and a capacitor, such that an output of the controllable current source circuit can provide a lower frequency current part of the regulated power supply output, and the capacitor can supply a higher frequency current part of the regulated power supply output. The controllable current source circuit is controlled according to feedback from the regulated power supply output, and to restrict a rate of change of the output of the controllable current source circuit. The amount of EMI noise caused by high rate of change of current in power supply lines to the regulator circuit can be reduced. This can be done more efficiently or using a smaller capacitor than known arrangements.