Managed grassland soils represent a large reservoir of soil organic carbon (SOC), playing a crucial role in climate regulation. While optimal herbivore grazing is presumed to be non-detrimental and may even increase soil carbon sequestration, there is a limit of long-term experimental studies to validate this effect, especially those that incorporate multiple herbivore species and their combinations to assess impacts on SOC stocks. Here, we conducted a 14-year controlled grazing experiment with moderate intensity, incorporating common herbivore species (sheep, cattle, or both) in a temperate grassland. The results showed that more than a decade of moderate grazing did not alter total SOC stocks and also particulate organic carbon (POC), regardless of livestock assemblages, while diversified livestock grazing by cattle and sheep increased the mineral-associated organic carbon (MAOC) content and its proportion of bulk soil carbon. This effect of grazing on soil MAOC is primarily attributed to an increase in soil microbial carbon inputs, driven directly by diversified livestock grazing and indirectly by improved plant diversity and soil conditions. Our study indicates that moderate grazing had a strong effect on MAOC, a key indicator of long-term soil carbon stability, but had little influence on total SOC. We further suggest that improving grassland soil carbon sequestration requires not only optimizing grazing intensity, but also incorporating diverse herbivore assemblages that mimic natural grazing systems, with diversified moderate grazing emerging as an effective management strategy for promoting carbon storage and contributing to climate change mitigation.