Temporal fluctuations in the strength and direction of selection are often proposed as a mechanism that slows down evolution, both over geological and contemporary time-scales. Both the prevalence of fluctuating selection and its relevance for evolutionary dynamics remain poorly understood however, especially on contemporary time scales: Unbiased empirical estimates of variation in selection are scarce, and the question of how much of the variation in selection translates into variation in genetic change has largely been ignored. Using long-term individual-based data for a wild rodent population, we quantify the magnitude of fluctuating selection on body size. Subsequently, we estimate the evolutionary dynamics of size, and test for a link between fluctuating selection and evolution. We show that, over the past 11 years, phenotypic selection on body size has fluctuated significantly. However, the strength and direction of genetic change have remained largely constant over the study period, i.e., the rate of genetic change was similar in years where selection favored heavier versus lighter individuals. This result suggests that over shorter timescales, fluctuating selection does not necessarily translate into fluctuating evolution. Importantly however, individual-based simulations show that the correlation between fluctuating selection and fluctuating evolution can be obscured by the effect of drift, and that substantially more data is required for a precise and accurate estimate of this correlation. We identify new challenges in measuring the coupling between selection and evolution, and provide methods and guidelines to overcome them.