The ability of animals to find and consume hoarded seeds (i.e. seed recovery) is a key stage within the seed dispersal process. However, the ecology of seed recovery is still poorly understood. Here, we analyze the factors controlling seed recovery by scatter-hoarding rodents in an oak-dominated temperate forest. We examined the relative importance of intrinsic seed traits (i.e. plant-driven) and extrinsic seed factors (i.e. animal-driven) on the probability of seed recovery. We found that seed recovery is mainly driven by extrinsic seed factors, mostly related to animal behavior (pilfering frequency, microsite preference, predation risk, burial depth and cache size). Important intrinsic traits such as seed size, seed quality and seed-drop timing were, on average, of lower significance in the probability of seed recovery (2.8-times less important than extrinsic factors); only seed quality was an important intrinsic trait. On the other hand, larger and nutritionally more valuable seeds showed a removal–recovery tradeoff as they enhance seed removal and hoarding (increasing dispersal quality) but also favour seed recovery (increasing predation). We find that other mechanisms beyond seed traits (e.g. masting) are needed to decrease seed recovery and, thus, increase seed survival. We conclude that, as seed recovery is mostly driven by animal behavioural factors, it substantially differs from other previous stages of the seed dispersal process that are more dependent on seed traits. We argue that seed recovery needs further attention to advance our understanding of the ecology of seed dispersal and the role of secondary dispersers as a selective force for seeds.