A monitoring system for an infant, child, invalid, or other person requiring care uses computer vision and hearing, and inputs of other modalities, to analyze the status of a caretaker and/or cared-for person and its environment. The conditions are classified as normal or alarm conditions and an informative alarm signal is generated which may include records of the vision audio and other inputs. The system also has the ability to solicit responses from the occupants to stimulate a classifiable input to reduce ambiguity in its state signal.