Despite an on-going struggle to conserve the endangered black rhinoceros (Diceros bicornis) since the 1980’s, huge capital investment and several genetic surveys, the level of genetic structure and connectivity among populations in southern Africa is not well understood. Here, we undertake a major population genetic study of black rhinoceros in the Zimbabwe Lowveld, an area inhabited by over half of that country’s original Zambezi descendants plus one large population sourced from the relict KwaZulu stock of South Africa. Using nuclear microsatellite and mitochondrial DNA data, we found much higher levels of genetic diversity in the indigenous Zimbabwean populations, where observed multilocus heterozygosity was 0.54 vs 0.40 in KwaZulu, and maternal haplotype diversity was 0.77 vs 0.03. We show, for the first time, that both gene pools can be differentiated from each other on the basis of nuclear markers. This, along with the discovery of recent gene flow between all Lowveld populations, suggests that Zimbabwean and South African gene pools were prehistorically connected.