The metaphor of “genomic islands of speciation” was first used to describe heterogeneous differentiation among loci between the genomes of closely related species. The biological model proposed to explain these differences was that the regions showing high levels of differentiation were resistant to gene flow between species, while the remainder of the genome was being homogenized by gene flow and consequently showed lower levels of differentiation. However, the conditions under which such differentiation can occur at multiple unlinked loci are restrictive; additionally, essentially all previous analyses have been carried out using relative measures of divergence, which can be misleading when regions with different levels of recombination are compared. Here we test the model of differential gene flow by asking whether absolute divergence is also higher in the previously identified “islands.” Using five species-pairs for which full sequence data is available, we find that absolute measures of divergence are not higher in genomic islands. Instead, in all cases examined we find reduced diversity in these regions, a consequence of which is that relative measures of divergence are abnormally high. These data therefore do not support a model of differential gene flow among loci, though islands of relative divergence may represent loci involved in local adaptation. Simulations using the program IMa2 further suggest that inferences of any gene flow may be incorrect in many comparisons. We instead present an alternative explanation for heterogeneous patterns of differentiation, one in which post-speciation selection generates patterns consistent with multiple aspects of the data.