A rotary tapered tool holder includes a body or shank including a front contact portion and a rear contact portion. An adapter sleeve is disposed about a portion of the shank. The adapter sleeve includes slits that allow radial expansion and contraction of the adapter sleeve. The adapter sleeve includes an inner surface formed at a first non-zero taper angle with respect to a longitudinal axis of the shank, and an outer surface formed at a second non-zero taper angle with respect to the longitudinal axis of the shank. The second taper angle is different than the first taper angle such that an interference fit between the shank and the inner surface of the adapter sleeve forces the outer surface of the adapter sleeve into positive contact with a tapered bore of a spindle of the rotary tapered tool holder.