Smallholder agriculture is the main driver of deforestation in the western Amazon, where terrestrial biodiversity reaches its global maximum. Understanding the biodiversity value of the resulting mosaics of cultivations and secondary forest is therefore crucial for conservation planning. However, Amazonian communities are organized across multiple forest‐types that support distinct species assemblages, and little is known about smallholder impacts across the range of forest habitats that are essential for sustaining biodiversity. We address this issue with a large‐scale field inventory of birds and trees in primary forest and smallholder agriculture in northern Peru, spanning three key forest‐types that structure Amazonian biodiversity. For birds, smallholder agriculture supported species richness comparable to primary forest within each forest‐type, but biotic homogenization across forest‐types resulted in substantial losses of biodiversity overall. These overall losses are invisible to studies that focus solely on upland (terra firme) forest. For trees, biodiversity losses in upland forests dominated the signal across all habitats combined, and homogenization across habitats did not exacerbate biodiversity loss. Proximity to forest strongly predicted the persistence of forest‐associated bird and tree species in the smallholder mosaic, and because intact forest is ubiquitous in our study area, our results probably represent a best‐case scenario for biodiversity in Amazonian agriculture. Land‐use planning both inside and outside of protected areas should recognize that tropical smallholder agriculture has pervasive biodiversity impacts that are not apparent in typical single‐habitat studies. The full range of forest‐types must be surveyed to accurately assess biodiversity losses, and primary forests must be protected to prevent landscape‐scale biodiversity loss.