Described herein are implantable medical devices, such as intravascular stents, for delivering therapeutic agents to a patient, and methods for making such medical devices. The medical devices comprise a substrate having at least a cavity therein and a pellet disposed in the cavity. The pellet comprises a non-polymeric material having a plurality of pores therein. A therapeutic agent is disposed in at least some of the pores.