The evolution of sex chromosomes is usually considered to be driven by sexually antagonistic selection in the diploid phase. However, selection during the haploid gametic phase of the lifecycle has recently received theoretical attention as possibly playing a central role in sex chromosome evolution, especially in plants where gene expression in the haploid phase is extensive. In particular, male-specific haploid selection might favour the linkage of pollen beneficial alleles to male sex determining regions on incipient Y chromosomes. This linkage might then allow such alleles to further specialise for the haploid phase. Purifying haploid selection is also expected to slow the degeneration of Y-linked genes expressed in the haploid phase. Here, we examine the evolution of gene expression in flower buds and pollen of two species of Rumex to test for signatures of haploid selection acting during plant sex chromosome evolution. We find that genes with high ancestral pollen expression bias occur more often on sex chromosomes than autosomes and that genes on the Y chromosome are more likely to become enriched for pollen expression bias. We also find that genes with low expression in pollen are more likely to be lost from the Y chromosome. Our results suggest that sex-specific haploid selection during the gametophytic stage of the lifecycle may be a major contributor to several features of plant sex chromosome evolution.