Determining the driver mechanisms of phenotypic and genetic divergence of species have long been the central topic in evolutionary biology. However, the relative roles of isolation by distance (IBD) and environment factors in contributing to species divergence is largely undetermined in perennial herbs in East Asia. In this study, we collected population genetic samples of a cold-tolerant perennial herb Notopterygium oviforme in central China. We integrated the population transcriptomes, whole chloroplast genomes, genotyping-by-sequencing, microsatellite markers and ecological environment factors to determine the relative contributions of geography, climatic and soil factors on genetic, chemical and phenotypic trait divergence. The clearly genetic distinction was identified between the west Qinling (WQ) and east Qinling Mountains (EQ) geographical groups within N. oviforme. Interestingly, the two groups have existed the obviously niche conservatism, which might have caused the similarity of most leaf functional traits. Multiple matrix regression with randomization analysis showed that the spatial pattern of intraspecific divergence mainly resulted from effect of IBD, not isolation by environment. Meanwhile, transcriptomic expression divergence and some ecological factors have played the key roles in differentiation of intraspecific lineage. Approximate Bayesian Computation showed that both lineages have experienced the historically population expansion, and the recent range contraction. Molecular dating suggested that the intraspecific divergence was closely associated with the dramatically uplifts of the Qinling Mountains in East Asia. These results demonstrated that the geography, geological and environmental factors together shaped the cryptic intraspecific diversification and population dynamics shifts of perennial cold tolerant herb.