Species that exhibit very peculiar ecological traits combined with limited dispersal ability pose a challenge to our understanding of ecological and evolutionary mechanisms. This is especially true when they have managed to spread over long distances, overcome physical barriers and colonise large areas. Climate and landscape changes, trophic web relations as well as life history all interact to shape migration routes and present-day species distributions and their population genetic structures. Here we analysed the post-glacial colonization of northern Europe by the gall midge Contarinia vincetoxici, which is a monophagous parasite on the perennial herb White swallowwort (Vincetoxicum hirundinaria). This insect not only has a narrow feeding niche but also limited dispersal ability and an exceptionally long dormancy. Gall midge larvae (n = 329) were collected from 16 sites along its distribution range in Denmark, Sweden, and Finland. Using microsatellite loci and knowledge of the species and the regions’ history, we investigated the role of landscape change, host plant distribution, insect population dynamics, and life history in shaping the population genetic structure of the insect. We devoted particular interest to the role of the insect’s presumed poor dispersal capacity in combination with its exceptionally extended diapause. We found significant levels of local inbreeding (95% highest posterior density interval = 0.42–0.47), low level within-population heterozygosity (mean HE = 0.45, range 0.20–0.61) with private alleles in all populations except two. We also found significant (p < 0.001) regional isolation-by-distance patterns, suggesting regularly recurring mainly short-distance dispersal. According to approximate Bayesian computations, C. vincetoxici appears to have colonized the study area via wind-aided flights from remote areas approximately 4600 to 700 years before present when the land has gradually risen above the sea level. Extremely long dormancy periods have allowed the species to “disperse in time”, thereby aiding population persistence despite generally low census population sizes.