How reproductive isolation is related to divergent natural selection is a central question in speciation. Here we focus on several ecologically specialised taxa or “call types” of red crossbills (Loxia curvirostra complex), one of the few groups of birds providing much evidence for ecological speciation. Call types differ in bill sizes and feeding capabilities, and also differ in vocalizations, such that contact calls provide information on crossbill phenotype. We found that two call types of red crossbills were more likely to approach playbacks of their own call type than those of heterotypics, and that their propensity to approach heterotypics decreased with increasing divergence in bill size. Although call similarity also decreased with increasing divergence in bill size, comparisons of responses to familiar versus unfamiliar call types indicate that the decrease in the propensity to approach heterotypics with increasing divergence in bill size was a learned response, and not a byproduct of calls diverging pleiotropically as bill size diverged. Because crossbills choose mates while in flocks, assortative flocking could lead indirectly to assortative mating as a byproduct. These patterns of association therefore provide a mechanism by which increasing divergent selection can lead to increasing reproductive isolation.