What causes species' niche and range margins to shift is not only a fundamental theoretical question, but also directly affects how we assess the resilience of natural populations in current and future environments. Yet despite the urgent need for a theory that can predict evolutionary and ecological responses in times of accelerated climate change, the assumptions of current eco-evolutionary theory remain restrictive, with predictions neglecting important interactions between ecological and evolutionary forces. In this study, I provide quantitative, testable predictions on limits to adaptation in changing environments, which arise from the feedback between selection, genetic drift, and population dynamics. This eco-evolutionary feedback creates a tipping point beyond which adaptation fails as genetic drift overwhelms selection, and species' ranges contract from the margins or fragment abruptly -- even under gradual environmental change. This "expansion threshold'' is determined by three parameters: two quantifying the effects of spatial and temporal variability on the fitness of the population, and one capturing the impact of genetic drift: the reduction of local genetic diversity across generations in finite populations. Genetic drift is strong in populations with small "neighbourhood size''. Increasing dispersal, such as via assisted migration, enlarges neighbourhood size, counteracting the loss of genetic variation due to genetic drift. This increases adaptive potential and can facilitate evolutionary rescue in changing environments. Conversely, beyond the expansion threshold, local genetic variance becomes depleted, increasing extinction probability. The theory provides general predictions for species' range and niche dynamics beyond standard ecological niche models, highlighting the fundamental impact of eco-evolutionary interactions.