Macroevolutionary consequences of competition among large clades have long been sought in patterns of lineage diversification. However, mechanistically clear examples of such effects remain elusive. Here we postulated that the limited phenotypic diversity and insular gigantism in lagomorphs could be explained at least in part by an evolutionary constraint placed on them by potentially-competing ungulate-type herbivores (UTHs). Our analyses yielded three independent lines of evidence supporting this hypothesis: (1) the minimum UTH body mass is the most influential predictor of the maximum lagomorph body mass in modern ecoregions; (2) the scaling patterns of local-population energy use suggest universal competitive disadvantage of lagomorphs weighing over ~6.3 kg against artiodactyls, closely matching their observed upper size limit in continental settings; and (3) the trajectory of maximum lagomorph body mass in North America from the late Eocene to the Pleistocene (37.5–1.5 Ma) was best modeled by the body mass ceiling placed by the smallest contemporary perissodactyl or artiodactyl. Body size evolution in lagomorphs has likely been regulated by the conflicting forces of competition within the clade, increased predation in open habitats, and importantly, competition from other ungulate-type herbivores. Our finding highlights the conditionally-coupled dynamics of phenotypic boundaries within an adaptive zone, offering a complementary, if nuanced, perspective to Van Valen’s Red Queen hypothesis.