Mirror-image flowers (enantiostyly) involve a form of sexual asymmetry in which a flower’s style is deflected either to the left or right side, with a pollinating anther orientated in the opposite direction. This curious floral polymorphism, which was known but not studied by Charles Darwin, occurs in at least 11 unrelated angiosperm families and represents a striking example of adaptive convergence in form and function associated with cross-pollination by insects. In several lineages, dimorphic enantiostyly (one stylar orientation per plant, both forms occurring within populations) has evolved from monomorphic enantiostyly, in which all plants can produce both style orientations. We use a modelling approach to investigate the emergence of dimorphic enantiostyly from monomorphic enantiostyly under gradual evolution. We show using adaptive dynamics that depending on the balance between inbreeding depression following geitonogamy, pollination efficiency and plant density, dimorphism can evolve from an ancestral monomorphic population. In general, the newly emergent dimorphic population is stable against invasion of a monomorphic mutant. However, our model predicts that under certain ecological conditions, e.g., a decline of pollinators, dimorphic enantiostyly may revert to a monomorphic state. We demonstrate using population genetics simulations that the observed evolutionary transitions are possible assuming a plausible genetic architecture.