In a combined holistic and analytic recognition system, the holistic recognition module will recognize an input word or phrase image by matching an input string of character features for the whole word or phrase against a string of prototype features for a plurality of reference words in a lexicon. This will yield a holistic answer list of recognized word or phrase candidates for the input word or phrase along with a confidence value for each answer on the list. At the same time based on each answer in the answer list, the holistic recognition modules will generate a list of character features and segment the character features into sets for each character in an answer. The analytical recognition module uses segmentation hypotheses from the segmented character feature sets to cut the image of the input string of characters into individual character images. A plurality of character images for the various segmentation hypotheses will be recognized to produce an analytical answer list having a plurality of word or phrase answers for the input word or phrase. Each analytic word answer will have a confidence value based on the combined confidence of recognizing each character. The holistic answer list and the analytic answer list will be examined to find the best answer from the two lists as the recognition of the input handwritten text.