Arbuscular mycorrhizal (AM) fungi provide a range of functions in natural and managed ecosystems. However, the trajectory of AM fungal diversity after land degradation is poorly known. We studied the succession of AM fungi along an agricultural recultivation chronosequence after open-cast mining near Cologne, Germany. We used high-throughput sequencing of the large-subunit ribosomal RNA genes to characterize the soil AM fungal communities of 10 agricultural fields spanning 52 years of recultivation. During three years, soils are recultivated with a legume, and then converted to agriculture to be later returned to local farmers implementing conventional agriculture. Our data reveal a quick and strong recovery of AM fungal richness after a few years of recultivation, but also a rapid decline following years of conventional agriculture. The community structure was strongly correlated to mineral nitrogen and phosphorus, richness peaking at high N:P ratio. This work represents the first molecular data documenting temporal patterns of AM fungal communities in agriculture; it shows the deleterious effect of conventional agricultural practices on AM fungal communities developing over time. Nonetheless, the highly dynamic nature of AM fungal communities suggests strategies for site-level management for which considering N:P stoichiometry is crucial.