Harvest of Brazil nuts from the large, iconic tree Bertholletia excelsa generates substantial income for smallholders, providing a strong incentive to conserve the mature forests where it grows. Although much previous work has focused on the impact of nut harvest on new seedling recruits into B. excelsa populations, the connection between harvest rates and long-term population stability is still unclear. Moreover, there is additional uncertainty for Brazil nut management in terms of population response to climate change and other anthropogenic influences. We drew on 14 years of research in two sites in Acre, Brazil with different B. excelsa nut harvest intensities (39% and 81%), to produce stochastic and deterministic matrix population models which incorporated parameter uncertainty in vital rates. Adult abundance was projected to remain close to the current observed abundance or higher through the next 50 years. Elasticity analyses revealed that the asymptotic population growth rate (λ) was most sensitive to stasis vital rates in sapling, juvenile, and adult stages. Deterministic transition matrices calculated using diameter growth rates dependent on rainfall yielded average λ values around 1.0 under extreme high, extreme low, and average annual rainfall. While sustained high rates of Brazil nut harvest and climate change could potentially negatively impact B. excelsa populations, changes in human use of the forested landscape are more immediate concern. To reduce the risk of population decline, smallholders and managers of B. excelsa rich forests should focus on conservation of pre-mature and mature individuals.