The technique capable of reducing the power consumption in the MISFET by suppressing the scattering of the carriers due to the fixed charges is provided. A silicon oxynitride film with a physical thickness of 1.5 nm or more and the relative dielectric constant of 4.1 or higher is formed at the interface between a semiconductor substrate and an alumina film. By so doing, a gate insulator composed of the silicon oxynitride film and the alumina film is constituted. The silicon oxynitride film is formed by performing a thermal treatment of a silicon oxide film formed on the semiconductor substrate in a NO or N.sub.2O atmosphere. In this manner, the fixed charges in the silicon oxynitride film are set to 5.times.10.sup.12 cm.sup.-2 or less, and the fixed charges in the interface between the silicon oxynitride film and the alumina film are set to 5.times.10.sup.12 cm.sup.-2 or more.