A rotary driving device includes a rotary shaft supported rotatably about an axis thereof on a predetermined supporting member, a rotary load body mounted on the rotary shaft to project radially outward from the rotary shaft in such a manner that the rotary load body can rotate integrally with the rotary shaft about the axis thereof, a driver for rotating the rotary shaft about the axis thereof, a disk mounted on the rotary shaft coaxially therewith for integral rotation with the rotary shaft, the disk having a mounting hole formed therein, a pendulum loosely fitted in the mounting hole, and an adjustment mechanism for adjusting a relative position relationship between a central axis position of the mounting hole and a center of gravity position of the pendulum under conditions where the disk is rotating.