Techniques and tools relating to annotating program source code facilitate inferring annotations from source code based at least in part on a description (or specification) generated with a programmable tool. Described techniques and tools provide flexibility in annotation inference across different code bases and program states or properties of interest, and can reduce the overhead of adding annotations to "legacy" source code. For example, a specification is generated with a programmable specification tool that is separate from an inference engine. In the inference engine, one or more annotations for a computer program are inferred based at least in part on the specification.