A gate insulator film and a gate electrode are formed on a semiconductor substrate, and then a layered stack of a SiO₂ film and a SiN film is formed on the entire surface. Subsequently, sidewalls made of polysilicon film are formed adjacent to the gate electrode via the layered stack of the SiO₂ film and the SiN film. Then, using as a mask the gate electrode, portions of the layered stack adjacent to the gate electrode, and the sidewalls, an ion dopant is implanted into a device active region to thereby form source/drains therein, and the sidewalls are then removed. At this stage, since the gate insulator film is completely covered with the layered stack, the gate insulator film is not ablated or retreated even on a device isolation insulator film.