The operational sex ratio (OSR) and density are considered important factors affecting the strength of sexual selection. While there is increasing evidence that OSR and density affect the potential for sexual selection, few studies have addressed whether this is realized in phenotypic selection and how the two factors interact. We manipulated OSR (three levels) and male density (two levels) in 36 experimental breeding populations of Gobiusculus flavescens – a fish with paternal care. We measured mating competition behavior, the opportunity for selection (I), and selection on four morphological traits in males. We found sexual selection on two male traits, with the strongest selection being 20% of I. As predicted from OSR theory, increasing female scarcity caused males to become more competitive, concomitant with an increase in I and selection on morphological traits. Model simulations of I based on random mating (Imin) and maximum mate monopolization (Imax) demonstrated that the potential for sexual selection was close to its theoretical maximum across the range of OSRs. However, male density and its interaction with the OSR did not affect sexual selection. We argue that a multifaceted approach, combining mating behavior and selection analyses, can help us to understand how ecological factors affect sexual selection.