TAR DNA binding protein 43 (TDP-43) is closely related to the pathogenesis of amyotrophic lateral sclerosis (ALS) and translocates to stress granules (SGs). The role of SGs as aggregation-promoting “bioreactors” for TDP-43, however, is still under debate. We analyzed TDP-43 mobility and localization under different stress and recovery conditions using live cell single-molecule tracking and super-resolution microscopy. Besides reduced mobility within SGs, a stress induced decrease of TDP-43 mobility in the cytoplasm and the nucleus was observed. Stress removal led to a recovery of TDP-43 mobility, which strongly depended on the stress duration. ‘Stimulated-emission depletion microscopy’ (STED) and ‘tracking and localization microscopy’ (TALM) revealed not only TDP-43 substructures within stress granules but also numerous patches of slow TDP-43 species throughout the cytoplasm. The data provide new insights into the aggregation of TDP-43 in living cells and provide evidence suggesting that TDP-43 oligomerization takes place in the cytoplasm separate from SGs.