Sterility virulence, or the reduction in host fecundity due to infection, occurs in many host-pathogen systems. Notably, sterility virulence is more common for sexually-transmitted infections (STIs) than for directly-transmitted pathogens, while other forms of virulence tend to be limited in STIs. This has led to the suggestion that sterility virulence may have an adaptive explanation. By focusing upon finite population models, we show that the observed patterns of sterility virulence can be explained by consideration of the epidemiological differences between STIs and directly-transmitted pathogens. In particular, when pathogen transmission is predominantly density-invariant (as for STIs), and mortality is density-dependent, sterility virulence can be favoured by demographic stochasticity, whereas if pathogen transmission is predominantly density-dependent, as is common for most directly-transmitted pathogens, sterility virulence is disfavoured. We show these conclusions can hold even if there is a weak selective advantage to sterilizing.