Deciphering the genetic and developmental causes of the disproportionate rarity, inviability and sterility of hybrid males, Haldane's rule, is important for understanding the evolution of reproductive isolation between species. Moreover, extrinsic and pre-zygotic factors can contribute to the magnitude of intrinsic isolation experienced between species with partial reproductive compatibility. Here we use the nematodes Caenorhabditis briggsae and C. nigoni to quantify the sensitivity of hybrid male viability to extrinsic temperature and developmental timing, and test for a role of mito-nuclear incompatibility as a genetic cause. We demonstrate that hybrid male inviability manifests almost entirely as embryonic, not larval, arrest and is maximal at the lowest rearing temperatures, indicating an intrinsic-by-extrinsic interaction to hybrid inviability. Crosses using mitochondrial substitution strains that have reciprocally introgressed mitochondrial and nuclear genomes show that mito-nuclear incompatibility is not a dominant contributor to post-zygotic isolation and does not drive Haldane's rule in this system. Crosses also reveal that competitive superiority of X-bearing sperm provides a novel means by which post-mating pre-zygotic factors exacerbate the rarity of hybrid males. These findings highlight the important roles of gametic, developmental, and extrinsic factors in modulating the manifestation of Haldane's rule.