House mice are a major ecosystem pest, particularly threatening island ecosystems as a non-native invasive species. Rapid advances in synthetic biology offer new avenues to control pest species for biodiversity conservation. Recently, a synthetic sperm killing gene drive construct called t-Sry has been proposed as a means to eradicate target mouse populations due to a lack of females. A factor that has received little attention in the discussion surrounding such drive applications is polyandry. Previous research has demonstrated that sperm killing drivers are extremely damaging to a male's sperm competitive ability. Here we examine the importance of this effect on the t-Sry system using a theoretical model. We find that polyandry substantially hampers the spread of t-Sry such that release efforts have to be increased 3- to 6-fold for successful eradication. We discuss the implications of our finding for pest potential control programs, the risk of drive spread beyond the target population, and the emergence of drive resistance. Our work highlights that a solid understanding of the forces that determine drive dynamics in a natural setting are key for successful drive application, and that exploring the natural diversity of gene drives may inform effective gene drive design.