Floral traits are hypothesized to evolve primarily in response to selection by pollinators. However, selection can also be mediated by other environmental factors. To understand the relative importance of pollinator-mediated selection and its variation among trait and pollinator types, we analyzed directional selection gradients on floral traits from experiments that manipulated the environment to identify agents of selection. Pollinator-mediated selection was stronger than selection by other biotic factors (e.g. herbivores), but similar in strength to selection by abiotic factors (e.g. soil water), providing partial support for the hypothesis that floral traits evolve primarily in response to pollinators. Pollinator-mediated selection was stronger on pollination efficiency traits than on other trait types, as expected if efficiency traits affect fitness via interactions with pollinators, but other trait types also affect fitness via other environmental factors. In addition to varying among trait types, pollinator-mediated selection varied among pollinator taxa: selection was stronger when bees, long-tongued flies, or birds were the primary visitors than when the primary visitors were Lepidoptera or multiple animal taxa. Finally, reducing pollinator access to flowers had a relatively small effect on selection on floral traits, suggesting that anthropogenic declines in pollinator populations would initially have modest effects on floral evolution.