Hybridization is often maladaptive, and in some instances has led to the loss of biodiversity. However, hybridization can also promote speciation, such as during homoploid hybrid speciation, thereby generating biodiversity. Despite examples of homoploid hybrid species, the importance of hybridization as a speciation mechanism is still widely debated, and we lack a general understanding of the conditions most likely to generate homoploid hybrid species. Here we show that the level of genetic divergence between hybridizing species has a large effect on the probability that their hybrids evolve reproductive isolation. We find that populations of hybrids formed by parental species with intermediate levels of divergence were more likely to mate assortatively, and discriminate against their parental species, than those generated from weakly or strongly diverged parental species. Reproductive isolation was also found between hybrid populations, suggesting differential sorting of parental traits across populations. Finally, hybrid populations derived from three species were more likely to evolve reproductive isolation than those derived from two, supporting arguments that hybridization-supplied genetic diversity can lead to the evolution of novel “adaptive systems” and promote speciation. Our results illustrate when we expect hybridization and admixture to promote hybrid speciation. Whether homoploid hybrid speciation is a common speciation mechanism in general, remains an outstanding empirical question.