A multimedia archive description scheme is provided for characterizing a multimedia archive having records and associated record descriptions. The multimedia archive description scheme provides a data structure which relates records by similarity measures. The principle data structure in the multimedia archive description scheme is a cluster. A cluster includes one or more attributes of the records in the archive and can include one or more cluster relationships. Cluster attributes can include feature space attributes, semantic attributes, media attributes and meta attributes of the records in the archive. The cluster relationships can relate records to clusters or clusters to clusters. Cluster relationships can include feature space (syntactic) relationships, semantic relationships, media relationships and meta relationships. The multimedia archive description scheme provides an efficient form for describing a collection of records.