A heat sink retaining device for attaching a heat sink on a chip unit is provided, which includes a retaining component and a rotating component. The retaining component is locked to the heat sink in a state of pressing by a hook portion thereof, and the rotating component is pivotally disposed on the retaining component and pivotally rotated between a release position and a locking position. When the rotating component is at the locking position, a clamping portion of the rotating component is snapped onto the chip unit and presses the retaining component against the heat sink and the chip unit, thus improving the stability, reliability and fastness of the integral combination and meanwhile achieving the goal of being extremely convenient in assembling or disassembling.