Trade-offs maintain diversity and structure communities along environmental gradients. Theory indicates that if covariance among functional traits sets a limit on the number of viable trait combinations in a given environment, then communities with strong multidimensional trait constraints should exhibit low species diversity. We tested this prediction in winter annual plant assemblages along an aridity gradient using multilevel structural equation modelling. Univariate and multivariate functional diversity measures were poorly explained by aridity, and were surprisingly poor predictors of community richness. By contrast, the covariance between maximum height and seed mass strengthened along the aridity gradient, and was strongly associated with richness declines. Community richness had a positive effect on local neighbourhood richness, indicating that climate effects on trait covariance indirectly influence diversity at local scales. We present clear empirical evidence that declines in species richness along gradients of environmental stress can be due to increasing constraints on multidimensional phenotypes.