Explaining the strong variation in lifespan among organisms remains a major challenge in evolutionary biology. Whereas previous work has concentrated mainly on differences in selection regimes and selection pressures, we hypothesize that differences in genetic drift may explain some of this variation. We develop a model to formalize this idea and show that the strong positive relationship between lifespan and genetic diversity predicted by this model indeed exists among populations of Daphnia magna, and that ageing is accelerated in small populations. Additional results suggest that this is due to increased drift in small populations rather than adaptation to environments favoring faster life histories: First, the correlation between genetic diversity and lifespan remains significant after statistical correction for potential environmental covariates. Second, no trade-offs are observed; rather, all investigated traits show clear signs of increased genetic load in the small populations. Third, hybrid vigor with respect to lifespan is observed in crosses between small but not between large populations. Together, these results suggest that the evolution of lifespan and ageing can be strongly affected by genetic drift, especially in small populations, and that variation in lifespan and ageing may often be non-adaptive, due to a strong contribution from mutation accumulation to this variation.