Ecological speciation plays a primary role in driving species divergence and adaptation. Oryza rufipogon and Oryza nivara are two incipient species at the early stage of speciation with distinct differences in morphology, life history traits and habitat preference, and therefore provide a unique model for the study of ecological speciation. However, the population genetic structure of the ancestral O. rufipogon has been controversial despite substantial study, and the origin of the derivative O. nivara remains unclear. Here, based on sequences of 10 nuclear and two chloroplast loci from 26 wild populations across the entire geographic ranges of the two species, we conducted comprehensive analyses using population genetics, phylogeography and species distribution modelling (SDM) approaches. In addition to supporting the two previously reported major subdivisions, we detected four genetically distinct groups within O. rufipogon and found no correlation between the genetic groups and either species identity or geographical regions. The SDM clearly showed substantial change in the distribution range of O. rufipogon in history, demonstrating that the repeated extinction and colonization of local populations due to multiple glacial–interglacial cycles during the Quaternary was most likely the main factor shaping the confounding population genetic structure of O. rufipogon. Moreover, we found significant differences between the two species in climate preferences, suggestive of an important role for climatic factors in the adaptation, persistence and expansion of O. nivara. Finally, based on the genetic pattern and dynamics of the O. nivara populations, we hypothesize that O. nivara might have independently originated multiple times from different O. rufipogon populations.