A technique for generating an animated character based on visual and audio input from a live subject. Further described is a technique of extracting phonemes to select corresponding visemes to model a set of physical positions of the subject or emotional expression of the subject.