Many pathogens and parasites are transmitted through hosts that differ in species, sex, genotype, or immune status. In addition, virulence (here defined as disease-induced mortality) and transmission can vary during the infectious period within hosts of different state. Most models of virulence evolution assume that transmission and virulence are constant over the infectious period and that the host population is homogenous. Here, we examine a multi-species SIR model where transmission occurs within and between species, and transmission and virulence varied during the infectious period. This allows us to understand virulence evolution in a broader range of situations that characterize many emerging diseases. Because emerging pathogens are by definition new to their host populations, they should be expected to rapidly adapt after emergence. We illustrate these evolutionary effects using the framework of adaptive dynamics to examine how virulence evolves after emergence in response to the relative strength of selection on pathogen fitness and mutational variance for virulence. We illustrate the role of evolution by simulating adaptive walks to an evolutionarily stable virulence. We found that the magnitude of between-species transmission and the relative timing of transmission and mortality across species were of primary importance for determining the evolutionarily stable virulence.