The 20th-century commercial whaling industry severely reduced populations of great whales throughout the Southern Hemisphere. The effect of this exploitation on genetic diversity and population structure remains largely undescribed. Here, we compare pre- and post-whaling diversity of mitochondrial DNA (mtDNA) control region sequences for three great whales in the South Atlantic, the blue, humpback and fin whale. Pre-whaling diversity is described from mtDNA extracted from bones collected near abandoned whaling stations, primarily from the South Atlantic island of South Georgia. These bones are known to represent the first stage of 20th-century whaling and thus pre-whaling diversity of these populations. Post-whaling diversity is described from previously published studies reporting large-scale sampling of living whales in the Southern Hemisphere. Despite relatively high levels of surviving genetic diversity in the post-whaling populations, we found evidence of a probable loss of mtDNA lineages in all three species. This is evidenced by the detection of a large number of haplotypes found in the pre-whaling samples that are not present in the post-whaling samples. A rarefaction analysis further supports a loss of haplotypes in the South Atlantic humpback and Antarctic blue whale populations. The bones from former whaling stations in the South Atlantic represent a remarkable molecular archive for further investigation of the decline and ongoing recovery in the great whales of the Southern Hemisphere.