Variation in the environment can induce different patterns of genetic and phenotypic differentiation among populations. Both neutral processes and selection can influence phenotypic differentiation. Altitudinal phenotypic variation is of particular interest in disentangling the interplay between neutral processes and selection in the dynamics of local adaptation processes but remains little explored. We conducted a common garden experiment to study the phenotypic divergence in larval life-history traits among nine populations of the common toad (Bufo bufo) along an altitudinal gradient in France. We further used correlation among population pairwise estimates of quantitative trait (QST) and neutral genetic divergence (FST from neutral microsatellite markers), as well as altitudinal difference, to estimate the relative role of divergent selection and neutral genetic processes in phenotypic divergence. We provided evidence for a neutral genetic differentiation resulting from both isolation by distance and difference in altitude. We found evidence for phenotypic divergence along the altitudinal gradient (faster development, lower growth rate and smaller metamorphic size). The correlation between pairwise QSTs–FSTs and altitude differences suggested that this phenotypic differentiation was most likely driven by altitude-mediated selection rather than by neutral genetic processes. Moreover, we found different divergence patterns for larval traits, suggesting that different selective agents may act on these traits and/or selection on one trait may constrain the evolution on another through genetic correlation. Our study highlighted the need to design more integrative studies on the common toad to unravel the underlying processes of phenotypic divergence and its selective agents in the context of environmental clines.