Interspecific territoriality occurs when individuals of different species fight over space, and may arise spontaneously when populations of closely related territorial species first come into contact. But defence of space is costly, and unless the benefits of excluding heterospecifics exceed the costs, natural selection should favour divergence in competitor recognition until the species no longer interact aggressively. Ordinarily males of different species do not compete for mates, but when males cannot distinguish females of sympatric species, females may effectively become a shared resource. We model how reproductive interference caused by undiscriminating males can prevent interspecific divergence, or even cause convergence, in traits used to recognize competitors. We then test the model in a genus of visually orienting insects and show that, as predicted by the model, differences between species pairs in the level of reproductive interference, which is causally related to species differences in female coloration, are strongly predictive of the current level of interspecific aggression. Interspecific reproductive interference is very common and we discuss how it may account for the persistence of interspecific aggression in many taxonomic groups.