As organisms age, the effectiveness of natural selection weakens, leading to age-related decline in fitness-related traits. The evolution of age-related changes associated with senescence is likely influenced by mutation accumulation (MA) and antagonistic pleiotropy (AP). MA predicts that age-related decline in fitness components is driven by age-specific sets of alleles, non-negative genetic correlations within trait across age, and an increase in the coefficient of genetic variance. AP predicts that age-related decline in a trait is driven by alleles with positive effects on fitness in young individuals and negative effects in old individuals, and is expected to lead to negative genetic correlations within traits across age. We build on these predictions using an association mapping approach to investigate the change in additive effects of SNPs across age and among traits for multiple stress-response fitness-related traits, including cold stress with and without acclimation and starvation resistance. We found support for both MA and AP theories of aging in the age-related decline in stress tolerance. Our study demonstrates that the evolution of age-related decline in stress tolerance is driven by a combination of alleles that have age-specific additive effects, consistent with MA, as well as non-independent and antagonistic genetic architectures characteristic of AP.