The heritability (h2) of fitness traits is often low. Although this has been attributed to directional selection having eroded genetic variation in direct proportion to the strength of selection, heritability does not necessarily reflect a trait's additive genetic variance and evolutionary potential (“evolvability”). Recent studies suggest that the low h2 of fitness traits in wild populations is caused not by a paucity of additive genetic variance (VA) but by greater environmental or nonadditive genetic variance (VR). We examined the relationship between h2 and variance-standardized selection intensities (i or βσ), and between evolvability (IA:VA divided by squared phenotypic trait mean) and mean-standardized selection gradients (βμ). Using 24 years of data from an island population of Savannah sparrows, we show that, across diverse traits, h2 declines with the strength of selection, whereas IA and IR (VR divided by squared trait mean) are independent of the strength of selection. Within trait types (morphological, reproductive, life-history), h2, IA, and IR are all independent of the strength of selection. This indicates that certain traits have low heritability because of increased residual variance due to the age at which they are expressed or the multiple factors influencing their expression, rather than their association with fitness.