Documenting and preserving the genetic diversity of populations, which conditions their long-term survival, has become a major issue in conservation biology. The loss of diversity often documented in declining populations is usually assumed to result from human disturbances; however, historical biogeographic events, otherwise known to strongly impact diversity, are rarely considered in this context. We apply a multi-locus phylogeographic study to investigate the late-Quaternary history of a tree frog (Hyla arborea) with declining populations in the northern and western part of its distribution range. Mitochondrial and nuclear polymorphisms reveal high genetic diversity in the Balkan Peninsula, with a spatial structure molded by the last glaciations. While two of the main refugial lineages remained limited to the Balkans (Adriatic coast, southern Balkans), a third one expanded to recolonize Northern and Western Europe, loosing much of its diversity in the process. Our findings show that mobile and a priori homogeneous taxa may also display substructure within glacial refugia (“refugia within refugia”), and emphasize the importance of the Balkans as a major European biodiversity center. Moreover, the distribution of diversity roughly coincides with regional conservation situations, consistent with the idea that historically impoverished genetic diversity may interact with anthropogenic disturbances, and increase the vulnerability of populations. Phylogeographic models seem important to fully appreciate the risks of local declines and inform conservation strategies.