A semiconductor memory device comprises a silicon layer having a first diffused region and a second diffused region formed therein, a gate electrode formed through an insulating film on one side of the silicon layer between the first and the second diffused regions, a capacitor formed on said one side of the silicon layer and having a storage electrode connected to the first diffused region, and a bit line formed on the other side of the silicon layer and connected to the second diffused region, whereby a semiconductor memory device of SOI structure can be easily fabricated. The bit line connected to the second diffused region is formed on the other side of the semiconductor layer, whereby the bit line can be arranged without restriction by the structure, etc. Of the capacitor. Short circuit between the capacitor and the bit line can be prevented.