A large brain is a defining feature of modern humans, yet there is no consensus regarding the patterns, rates, and processes involved in hominin brain size evolution. We use a reliable proxy for brain size in fossils, endocranial volume (ECV), to better understand how brain size evolved at both clade- and lineage-level scales. For the hominin clade overall, the dominant signal is consistent with a gradual increase in brain size. This gradual trend appears to have been generated primarily by processes operating within hypothesized lineages – 64% or 88% depending on whether one uses a more or less speciose taxonomy, respectively. These processes were supplemented by the appearance in the fossil record of larger-brained Homo species and the subsequent disappearance of smaller-brained Australopithecus and Paranthropus taxa. When the estimated rate of within-lineage ECV increase is compared to an exponential model that operationalizes generation-scale evolutionary processes, it suggests that the observed data were the result of episodes of directional selection interspersed with periods of stasis and/or drift; all of this occurs on too fine a time scale to be resolved by the current human fossil record, thus producing apparent gradual trends within lineages. Our findings provide a quantitative basis for developing and testing scale-explicit hypotheses about the factors that led brain size to increase during hominin evolution.