A discriminative machine learning system for labels text and non-text strokes in digital ink. The learning system considers stroke features and the context of the strokes, such as temporal information about one or more strokes, in a probabilistic framework. The learning system can also consider gap features within the probabilistic framework to label associated strokes.