It is intended to achieve the reduction in number of heat treatments carried out at high temperature (at least 600.degree. C.) and the employment of lower temperature processes (600.degree. C. or lower), and to achieve step simplification and throughput improvement. In the present invention, a barrier layer (105), a second semiconductor film (106), and a third semiconductor layer (108) containing an impurity element (phosphorus) that imparts one conductive type are formed on a first semiconductor film (104) having a crystalline structure. Gettering is carried out in which the metal element contained in the first semiconductor film (104) is allowed to pass through the barrier layer (105) and the second semiconductor film (106) by a heat treatment to move into the third semiconductor film (107). Afterward, the second and third semiconductor films (106) and (107) are removed with the barrier layer (105) used as an etching stopper.