A magnetic memory device having a TMR element with high performance and high reliability, which can prevent oxidation or reduction of a tunnel dielectric layer in the TMR element. A nonvolatile magnetic memory device (1) includes a TMR element (13) configured by sandwiching a tunnel dielectric layer (303) between a ferromagnetic magnetization pinned layer (302) and a ferromagnetic storage layer (304), wherein information is stored by utilizing a change in resistance according to whether the direction of spins in the ferromagnetic layers (302) and (304) are parallel or antiparallel. The magnetic memory device (1) further includes a write word line (11) as a first wiring and a bit line (12) as a second wiring intersecting each other at different levels with the TMR element (13) interposed therebetween. The write word line (11) is electrically insulated from the TMR element (13), and the bit line (12) is electrically connected to the TMR element (13). The side surface of the TMR element (13) is covered with a side barrier layer (61) for blocking impurities.