What are the costs and benefits for animals living in groups of different sizes? Balancing the trade-offs between within-group competition (which favors smaller groups) and between-group competition (which favors larger groups) suggests that intermediate-sized groups may be best, yet empirical support for this prediction has largely been lacking. Using long-term data on wild baboons, we provide novel evidence that individuals living in intermediate-sized groups have energetically optimal space-use strategies and lower glucocorticoid (stress hormone) concentrations than individuals in either large or small groups. Our results offer new insight into the costs and benefits of group living.