Objective: To examine associations between measures of obesity in middle to early-old age with later-life MRI markers of brain aging. Methods: We analyzed data from the Northern Manhattan MRI Sub-Study (n = 1,289). Our exposures of interest were body mass index (BMI), waist circumference (WC), waist-to-hip ratio, and plasma adiponectin levels. Our outcomes of interest were total cerebral volume (TCV), cortical thickness, white matter hyperintensity volume (WMHV), and subclinical brain infarcts (SBI). Using multivariable linear and logistic regression models adjusted for sociodemographics, health behaviors, and vascular risk factors, we estimated β coefficients (or odds ratios) and 95% confidence intervals (CIs) and tested interactions with age, sex, and race/ethnicity. Results: On average at baseline, participants were aged 64 years and had 10 years of education; 60% were women and 66% were Caribbean Hispanic. The mean (SD) time lag between baseline and MRI was 6 (3) years. Greater BMI and WC were significantly associated with thinner cortices (BMI β [95% CI] −0.089 [−0.153, −0.025], WC β [95% CI] −0.103 [−0.169, −0.037]) in fully adjusted models. Similarly, compared to those with BMI <25, obese participants (BMI ≥30) exhibited smaller cortical thickness (β [95% CI] −0.207 [−0.374, −0.041]). These associations were particularly evident for those aged <65 years. Similar but weaker associations were observed for TCV. Most associations with WMHV and SBI did not reach statistical significance. Conclusions: Adiposity in early-old age is related to reduced global gray matter later in life in this diverse sample. Future studies are warranted to elucidate causal relationships and explore region-specific associations.