The spatial heterogeneity of limiting soil resources is an essential factor determining ecosystem processes and function. It has been reported that large herbivores can strongly impact the variation and spatial distribution pattern of soil nitrogen (N). However, it remains unclear how large herbivores affect soil spatial heterogeneity and whether this influence is dependent on plant community diversity. Here we examined effects of different herbivore assemblages [no grazing; cattle grazing (CG); sheep grazing (SG); and mixed grazing (MG) of cattle and sheep] on soil N spatial heterogeneity in grasslands with high and low pre-grazing plant diversity in an eastern Eurasian steppe. We found that herbivore grazing generated and maintained spatial patterns of soil nutrients, depending on herbivore assemblage and the level of pre-grazing plant diversity. CG increased the spatial heterogeneity of soil available N in Leymus chinensis-dominated steppe meadows, which were independent of pre-grazing plant diversity. However, the effects of SG and MG strongly depended on grassland plant diversity, with an increased spatial heterogeneity of soil available N in the high-diversity grassland, but not in the low-diversity grassland. Synthesis and applications. We concluded that in a L. chinensis-dominated eastern Eurasian steppe, cattle ranching could be considered as an optimal grazing management protocol to improve soil spatial heterogeneity because cattle grazing (CG) consistently increased soil spatial heterogeneity in the context of both low and high plant diversity. Nevertheless, soil spatial heterogeneity could be improved by any herbivore grazing regime [CG and/or sheep grazing (SG)] when high plant diversity is maintained. These findings highlight the importance of conserving plant diversity to maintain grassland structure and ecosystem function. In grassland systems with high plant diversity, herbivore grazing and plant diversity would jointly improve soil spatial heterogeneity, thus feeding back to maintain higher plant diversity. Therefore, high plant diversity could generate a positive feedback loop of herbivore–plant–soil interactions in grazed grassland systems. Our findings indicate the importance of herbivore assemblages in maintaining spatial heterogeneity in low- and high-diversity grassland systems.