A model-based object recognition system operates to recognize an object on a predetermined world surface within a world space. An image of the object is acquired. This image is a distorted projection of the world space. The acquired image is processed to locate one or more local features of the image, with respect to an image coordinate system of the image. These local features are mapped a world coordinate system of the world surface, and matched to a model defined in the world coordinate system. Annotations can be arranged as desired relative to the object in the world coordinate system, and then inverse-mapped into the image coordinate system for display on a monitor in conjunction with the acquired image. Because models are defined in world coordinates, and pattern matching is also performed in world coordinates, one model definition can be used by multiple independent object recognition systems.