An interchangeable panel, modular display system for use as a doll house or miniature display case includes a modular frame with at least one wall section having grooves for slidably receiving a wall panel therein. The frame also includes an opening for inserting a wall panel into and removing a wall panel from the wall section. Each wall panel includes a flat, rigid support, a magnetically attractable layer, and a decorative cover layer. Magnetically attractable decorations can be removably attached at selected locations on the wall panel.