A semiconductor memory device having a self-aligned charge trapping layer and a method of manufacturing the same in which a consistent length of an ONO layer is ensured. Here, an insulating stacked structure is self-aligned to a bottom surface of conductive spacers.