The positive relationship between sexual size dimorphism (SSD) and harem size across pinnipeds is often cited as a textbook example of sexual selection. It assumes that female aggregation selected for large male size via male-male competition. Yet, it is also conceivable that SSD evolved prior to polygyny due to ecological forces. We analysed eleven life history traits in 35 pinniped species to determine their co-evolutionary dynamics and infer their most likely evolutionary trajectories contrasting these two hypotheses. We find support for SSD having evolved prior to changes in the mating system, either as a consequence of niche partitioning during aquatic foraging or in combination with sexual selection on males to enforce copulations on females. Only subsequently did polygyny evolve, leading to further coevolution as the strength of sexual selection intensified. Evolutionary sequence analyses suggest a polar origin of pinnipeds and indicate that SSD and polygyny are intrinsically linked to a suite of ecological and life history traits. Overall, this study calls for the inclusion of ecological variables when studying sexual selection and argues for caution when assuming causality between coevolving traits. It provides novel insights into the role of sexual selection for the co-evolutionary dynamics of SSD and mating system.