Mounting observational records demonstrate human-caused faunal decline in recent decades, while accumulating archaeological evidence suggests an early biodiversity impact of human activities during the Holocene. A fundamental question arises concerning whether modern wildlife population declines began during early human disturbance. Here, we performed population genomic analysis of six common forest birds in East Asia to address this question. For five of them, demographic history inference based on 25-33 genomes of each species revealed dramatic population declines by 4-48-fold over millennia (two to five thousand years ago). Nevertheless, ecological niche models predicted extensive range persistence during the Holocene and imply limited demographic impact of historical climate change. Summary statistics further suggest high negative correlations between these population declines and human disturbance intensities and indicate a potential driver of human activities. These findings provide deep-time and large-scale insight into the recently recognized avifaunal decline and support an early origin hypothesis of human effects on biodiversity. Overall, our study sheds light on the current biodiversity crisis in the context of long-term human-environment interactions and offers a multievidential framework for quantitatively assessing the ecological consequences of human disturbance.