The present invention relates to computer graphics applications involving scene rendering using objects modeled at multiple levels of detail. In accordance with an aspect of the invention, a ray tracer implementation allows users to specify multiple versions of a particular object, categorized by LOD ID's. A scene server selects the version appropriate for the particular scene, based on the size of the object on the screen for example, and provides a smooth transition between multiple versions of an object model. In one example, the scene server will select two LOD representations associated with a given object and assign relative weights to each representation. The LOD weights are specified to indicate how to blend these representations together. A ray tracer computes the objects hit by camera rays associated with pixels in the camera window, as well as secondary rays in recursive implementations, and rays striking LOD objects are detected and shaded in accordance with the weights assigned to the different representations. Embodiments are disclosed for level of detail control using both forward ray tracing and backward ray tracing, including handling of camera rays, reflected rays, refracted rays and shadow rays.