1. The effective number of species (ENS) has been proposed as a robust measure of species diversity that overcomes several shortcomings of both diversity indices and species richness measures. However, it is not yet clear if ENS improves interpretation and comparison of biodiversity monitoring data, and ultimately resource management decisions. 2. We used simulations of five stream macroinvertebrate assemblages and spatially extensive field data of stream fishes and mussels to show (1) how different ENS formulations respond to stress and (2) how diversity-environment relationships change with values of q, which weight ENS measures by species abundances. 3. Values of ENS derived from whole simulated assemblages with all species weighted equally (true species richness) steadily decreased as stress increased, and ENS-stress relationships became weaker and more different among assemblages with increased weighting. 4. The amount of variation in ENS across the fish and mussel assemblages that was associated with environmental gradients decreased with increasing q. 5. Synthesis and applications: ENS does not improve interpretability of how diversity responds to stress or natural environmental gradients, and incorporating relative abundance into species diversity measures as implemented in ENS can actually weaken detection of diversity responses. Ecologists need to be cautious about use and interpretation of diversity measures whose values are jointly influenced by richness and evenness, including ENS, and instead separately assess species richness, species evenness, and compositional change in ecological communities.