A method of manufacturing a semiconductor device includes forming a plurality of conductive layers above a substrate; forming a plurality of interlayer insulating layers; forming with dry etching a first hole penetrating the upper interlayer insulating layer to reach the lower insulating layer; forming a protective film on the first hole; and forming by etching a second hole penetrating the lower interlayer insulating layer via the first hole having the protective film formed thereon to form a contact hole.