Understanding what shapes variation in genetic diversity among species remains a major challenge in evolutionary ecology, and it has been seldom studied in parasites and other host-symbiont systems. Here, we studied mtDNA variation in a host-symbiont non-model system: 418 individual feather mites from 17 feather mite species living on 17 different passerine bird species. We explored how a surrogate of census size, the median infrapopulation size (i.e., the median number of individual parasites per infected host individual), explains mtDNA genetic diversity. Feather mite species genetic diversity was positively correlated with mean infrapopulation size, explaining 34% of the variation. As expected from the biology of feather mites, we found bottleneck signatures for most of the species studied but, in particular, three species presented extremely low mtDNA diversity values given their infrapopulation size. Their star-like haplotype networks (in contrast with more reticulated networks for the other species) suggested that their low genetic diversity was the consequence of severe bottlenecks or selective sweeps. Our study shows for the first time that mtDNA diversity can be explained by infrapopulation sizes, and suggests that departures from this relationship could be informative of underlying ecological and evolutionary processes.