Sex differences in the prevalence, course and severity of infection are widespread, yet the evolutionary consequences of these differences remain unclear. Understanding how male–female differences affect the trajectory of infectious disease requires connecting the contrasting dynamics that pathogens might experience within each sex, to the number of susceptible and infected individuals that are circulating in a population. In this study, we build on theory using genetic covariance functions to link the growth of a pathogen within a host to the evolution and spread of disease between individuals. Using the Daphnia–Pasteuria system as a test case, we show that based on within-host dynamics alone, females seem to be more evolutionary liable for the pathogen, with higher spore loads and greater divergence among pathogen genotypes as infection progresses. Between-host transmission, however, appears to offset the lower performance of a pathogen within a male host, making even subtle differences between the sexes evolutionarily relevant, as long as the selection generated by the between-host dynamics is sufficiently strong. Our model suggests that relatively simple differences in within-host processes occurring in males and females can lead to complex patterns of genetic constraint on pathogen evolution, particularly during an expanding epidemic. This article is part of the theme issue ‘Linking local adaptation with the evolution of sex differences’.