Plant community response to climate change ranges from synchronous tracking to strong mismatch. Explaining this variation in climate change response is critical for accurate global change modeling. Here we quantify how closely assemblages track changes in climate (match/mismatch) and how broadly climate niches are spread within assemblages (narrow/broad ecological tolerance, or ‘filtering’) using data for the last 21 ka for 531 eastern North American fossil pollen assemblages. Although climate matching has been strong over the last 21 millennia, mismatch increased in 30% of assemblages during the rapid climate shifts between 14.5 to 10 ka BP. Assemblage matching rebounded towards the present day in 10-20% of assemblages. Climate-assemblage mismatch was greater in tree-dominated and high-latitude assemblages, consistent with persisting populations, slower dispersal rates, and glacial retreat. In contrast, climate matching was greater for assemblages comprising taxa with higher median seed mass. Over half of the assemblages were climatically filtered at any given time, with peak filtering occurring at 8.5 ka BP for nearly 80% of assemblages. Thus, vegetation assemblages have highly variable rates of climate mismatch and filtering over millennial scales. These climate responses can be partially predicted by species’ traits and life histories. These findings help constrain predictions for plant community response to contemporary climate change.