A retracting mechanism for a lens barrel includes a non-rotatable member having an engagement surface; and a frame movable along an axis relative to the non-rotatable member without rotation and having a holder, the holder being movable between an aligned position in which an imaging element thereof is substantially aligned with the axis and a displaced position in which the imaging element is displaced relative to the aligned position. The holder includes an engagement surface which contacts the engagement surface of the non-rotatable member during movement of the frame towards the non-rotatable member, and moves the holder from the aligned position to the displaced position. The non-rotatable member includes a guide key or a key groove, and the frame includes a corresponding key groove or guide key, the guide key and the key groove being slidably engageable with each other and guiding the frame.