Inbreeding depression is widely regarded as a driving force in the evolution of dispersal, mate choice and sperm selection. However, due to likely costs of inbreeding avoidance, which are poorly understood, it is unclear to what extent selection to avoid inbreeding is expected in nature. Moreover, there are currently very few empirical estimates of the strength of selection against the act of inbreeding (mating with a relative), as opposed to the fitness costs of being inbred. Here, we use data from the individual-based study of red deer on the Scottish island of Rum, a strongly polygynous system which harbours a large inbreeding load, to estimate selection against the act of inbreeding for each sex. We use pedigree and genomic estimates of relatedness between individuals and measure fitness using both lifetime breeding success (number of calves born) and lifetime reproductive success (number of calves surviving to independence), with the latter incorporating inbreeding depression in calf survival. We find for both sexes that the repeatability of the act of inbreeding was low (<0.1), suggesting little among-individual variation for this trait on which selection can act. Using the genomic measures there was significant selection against the act of inbreeding in males, but not in females, and there was considerable uncertainty in the estimate in both sexes. We discuss possible explanations for these patterns and their implications for understanding the evolution of inbreeding avoidance in natural populations.