Understanding how communities assemble is a central goal of ecology. This is particularly relevant for communities of arbuscular mycorrhizal fungi (AMF), because the community composition of these beneficial plant symbionts influences important ecosystem processes. Moreover, AMF may be used as sensitive indicators of ecological soil quality if they respond to environmental variation in a predictable way. Here we use a molecular profiling technique (T-RFLP of 25S rRNA gene fragments) to test which factors determine AM fungal community composition in 40 agricultural soils in the Netherlands. In particular, we test whether species richness, dominance structure and community nestedness are influenced by management type (in pairs of organically and conventionally farmed fields), and we examine the contribution of crop species (maize versus potato), soil type (sand versus clay textured soils), and habitat (plant root versus bulk soil) on AMF community characteristics. AMF richness varied from 1 to 11 taxa per field. Communities from species poor fields were found to be subsets of those in richer fields, indicating nestedness and a progressive “loss” from the species pool. AMF taxa richness and occurrence in soil and plant roots were highly correlated, and richness was related to management intensity (phosphate availability and grass cropping history together explained 34% and 50% of richness in roots and soils). Soil type together with soil chemical parameters explained only 17% of variance in AMF community structure. We synthesize these results by discussing the potential contribution of a “bottle-neck effect” on AMF communities through increased stochastic effects under environmental stress.