Recent human activities have caused extensive forest fragmentation, which can profoundly affect the population genetic structure of soil microbes, as has been well documented for plants and animals. However, such information remains scarce for microbes, even for ectomycorrhizal (ECM) fungi, which play key roles in the growth and survival of dominant forest trees. To investigate the long-term effects of forest fragmentation on the population genetic structure of ECM fungi, we examined ice-age relict forests of Pinus pumila across nine alpine sites that have been isolated for more than 10,000 years. We focused on two phylogenetically related ECM fungi, Rhizopogon alpinus and Suillus spraguei, whose spores are dispersed primarily by animals and wind, respectively. The genetic structure of these fungi and their host trees was assessed via 28 microsatellite markers, 14 of which were newly developed in this study. R. alpinus, an animal-dispersed species, exhibited substantially greater genetic differentiation among populations than its host tree P. pumila, which disperses pollen via wind and seeds. In contrast, the wind-dispersed S. spraguei presented little evidence of population differentiation. Inbreeding levels were highest in P. pumila, intermediate in R. alpinus, and negligible in S. spraguei. This study provides the first direct evidence that forest fragmentation has contrasting effects on the population genetic structure of eukaryotic microbes with different dispersal mechanisms. The limited gene flow and elevated inbreeding observed in R. alpinus indicate a heightened risk of extinction—paralleling patterns observed in fragmented populations of plants and animals—and highlight the need for targeted conservation attention.