The long-term influence of persistent small herbivores on successional plant community configuration is rarely studied. We used an herbivore exclusion experiment along the successional gradient in a salt-marsh system, to investigate the effects of hares and geese, and hares alone, on plant diversity at five successional stages (the earliest, two early, the intermediate and the late successional stages) in the short and long term, i.e. 7 and 22 years, respectively. Plant diversity declined over time at all successional stages except for the earliest one. Small herbivores slowed down species decline, but only at one early successional stage. Small herbivores slowed down species decline via decreasing dominance of preferred grass Festuca rubra in the short term, and less preferred Elytrigia atherica in the long term. The effects of hares and geese were more pronounced than hares alone, indicating an important additive role of geese, especially in the long term. Synthesis. Small herbivores can have a strong and long-lasting impact on plant diversity, but it highly depends on the abundance of small herbivores, which in turn depends on the quality and abundance of forage plants. A diverse herbivore community may have more positive effects on regulating plant communities.