A method and interlocking building system for construction of buildings capable of multiple stories from a variety of standard size panels and a variety of connectors for interlocking the panels. The panels may be horizontally disposed to form floors, ceilings or roofs or may be vertically disposed to form walls. The panels are formed of a plastic shell and contain a plurality of metal strips spaced therein and received within internal guide tracks or integral internal vertically-oriented trussing. The metal strips or trussing have interlocking tooth profiles which project beyond the plastic panel shell. When the panels are used as wall panels, the panels are preferably disposed so that the frame members are vertical. Panels may be interlocked, one vertically above the other, to form multiple stories, with the frame members of all stacked wall panels being in horizontal alignment to form generally continuous vertical strength members. Panels can also be interlocked side by side with integral mate and female sides to form continuous walls in a horizontal direction. A variety of 2-way, 3-way and 4-way connectors with male and/or female receptacles allow the panels to be interconnected in virtually unlimited ways to allow interconnection of both vertical and horizontal panels.