Forest fragmentation and defaunation are considered the main drivers of biodiversity loss, yet the synergistic effects of landscape changes and biotic interactions on assemblage structure have been poorly investigated. Here, we use an extensive dataset of 283 assemblages and 105 species of small mammals to understand how defaunation of medium and large mammals and forest fragmentation change the community composition and diversity of rodents and marsupials in tropical forests of South America. We used structured equation models to investigate the relationship between small mammal species, functional and phylogenetic diversity with forest size, forest cover and the occurrence of medium and large mammals. The best-fit model showed that defaunation reduced functional diversity, and that species diversity of small mammals increased with forest patch size. Forest cover did not affect functional and phylogenetic diversity. Our results indicate that occurrence of medium and large sized mammals (probably acting as predators, or competitors of small mammals) and forest patch size help to retain species and functional diversity in small mammal communities. Further, the number of species in a small mammal community was critical to the maintenance of phylogenetic diversity, and may have a pronounced influence on the ecological functions played by small mammals. Identifying how phylogenetic and functional diversity change in function of human pressures allows us to better understand the contribution of extant lineages to ecosystem functioning in tropical forests.