This invention uses Bayesian techniques to prioritize alerts or alert groups generated by intrusion detection systems and other information security devices, such as network analyzers, network monitors, firewalls, antivirus software, authentication services, host and application security services, etc. In a preferred embodiment, alerts are examined for the presence of one or more relevant features, such as the type of an attack, the target of an attack, the outcome of an attack, etc. At least a subset of the features is then provided to a real-time Bayes network, which assigns relevance scores to the received alerts or alert groups. In another embodiment, a network manager (a person) can disagree with the relevance score assigned by the Bayes network, and give an alert or alert group a different relevance score. The Bayes network is then modified so that similar future alerts or alert groups will be assigned a relevance score that more closely matches the score given by the network manager.