Critical thermal limits are important ecological parameters for studying thermal biology and for modelling species’ distributions under current and changing climatic conditions (including predicting the risk of extinction for species from future warming). However, estimates of the critical thermal limits are biased by the choice of assay and assay conditions, which differ among studies. Furthermore, estimates of the potential for phenotypic plasticity (thermal acclimation) to buffer against future warming are usually based on single assay conditions and (usually linear) extrapolation from a few acclimation temperatures. We produced high resolution estimates of adult acclimation capacity for upper tolerance limits at different assay conditions (ramping rates and knock-down temperatures) using CTmax (dynamic) and knock-down (static) thermal assays in the model species Drosophila melanogaster. We found the reaction norms to be highly dependent on assay conditions. We confirmed that progressively lower ramping rates or higher knock-down temperatures led to overall lower tolerance estimates. More surprisingly, extended assays (lower ramping rates or lower knock-down temperatures) also led to increasingly non-linear reaction norms for upper thermal tolerance across adult acclimation temperatures. Our results suggest that the magnitude (capacity) and direction (beneficial or detrimental) of acclimation responses are highly sensitive to assay conditions. The results offer a framework for comparison of acclimation responses between different assay conditions and a potential for explaining disparate acclimation capacity theories. We advocate cautious interpretation of acclimation capacities and careful consideration of assay conditions, which should represent realistic environmental conditions based on species’ ecological niches.