A high-speed voltage regulating apparatus and a method for high-speed voltage regulation. The apparatus includes: (A) a regulator, adapted to provide a regulated voltage; (B) switching circuitry, connected to the regulator, adapted to either (i) connect the regulator to an output node or (ii) disconnect the regulator from the output node; whereas the output node is connected to a dynamic power consuming device and to a load capacitor; (C) control logic, connected to the regulator, adapted to receive at least an indication reflecting a voltage of the output node and to control the switching circuitry such that the regulator is disconnected from the output node to facilitate a decrease in the voltage of the output node. The method includes: (A) determining whether to (i) decrease a voltage of an output node, (ii) to maintain the voltage of the output node or to (iii) increase the voltage of the output node; (B) allowing a voltage of an output node to decrease by disconnecting a regulator from the output node; whereas the output node is coupled to a dynamic power consuming device and to a load capacitor; and (C) providing a regulated voltage corresponding to a required voltage of the output node, if determining to maintain the voltage of the output node or to increase the voltage of the output node.