Understanding why populations of some migratory species show a directional change over time, i.e. increase or decrease, while others do not, remains a challenge for ecological research. One possible explanation is that species with smaller non-breeding ranges may have more pronounced directional population trends, and their populations are thus more sensitive to the variation in environmental conditions in their non-breeding quarters. According to the serial residency hypothesis, this sensitivity should lead to higher magnitudes (i.e. absolute values) of population trends for species with smaller non-breeding ranges, with the direction of trend being either positive or negative depending on the nature of the environmental change. We tested this hypothesis using population trends over 2001–2012 for 36 sub-Saharan migratory passerine birds breeding in Europe. Namely, we related the magnitude of the species' population trends to the size of their sub-Saharan non-breeding grounds, whilst controlling for factors including number of migration routes, non-breeding habitat niche and wetness, breeding habitat type and life-history strategy. The magnitude of species’ population trends grew with decreasing absolute size of sub-Saharan non-breeding ranges, and this result remained significant when non-breeding range size was expressed relative to the size of the breeding range. After repeating the analysis with the trend direction, the relationship with the non-breeding range size disappeared, indicating that both population decreases and increases are frequent amongst species with small non-breeding range sizes. Therefore, species with small non-breeding ranges are at a higher risk of population decline due to adverse factors such as habitat loss or climatic extremes, but their populations are also more likely to increase when suitable conditions appear. As non-breeding ranges may originate from stochasticity of non-breeding site selection in naive birds (‘serial-residency’ hypothesis), it is crucial to maintain a network of stable and resilient habitats over large areas of birds’ non-breeding quarters.