Livestock overgrazing influences both microbial communities and nutrient cycling in terrestrial ecosystems. However, the role of overgrazing in regulating the relationship between soil biodiversity and nitrogen availability remains largely unexplored. We performed long-term grazing exclusion experiments across eight sites along precipitation gradient covering three major types of grassland in northern China to compare the linkage between soil microbial diversity and N availability in overgrazed versus non-grazed conditions. We found a positive association between fungal diversity and soil available N in non-grazed grasslands. However, the positive association was absent in overgrazed environments. Bacterial diversity is not related to soil available N in either non-grazed or overgrazed grasslands. Moreover, in bacterial community, we found a positive link between the relative abundance of Actinobacteria with soil available N in non-grazed, but not overgrazed, grasslands. Instead we found the links between relative abundance of Bacteroidetes and Acidobacteria with soil available N in overgrazed grasslands, but not non-grazed, grasslands. Our work provides evidence that the relationships between microbial diversity and ecosystem functions are context-dependent, and so microbial community diversity is likely not the major driver of soil N mineralization in overgrazed grasslands. Our study suggests that high intensity anthropogenic activities in grasslands restrains the capacity of diverse soil microbial communities to sustain ecosystem function, and more broadly the capacity of entire ecosystems to maintain important ecosystem processes such as plant production. Our study also indicates that the fundamental microbial communities associated with N availability change with differing land management strategies (e.g. livestock grazing).