Global expansion of oxygen deficient (hypoxia) waters will have detrimental effects on marine life in the Northeast Pacific Ocean (NEP) where some of the largest proportional losses in aerobic habitat are predicted to occur. However, few studies have accounted for the high environmental variability in this region while including natural community-assembly dynamics. Here, we present results from a 14-month deployment of a benthic camera platform tethered to the VENUS cabled observatory in the seasonally hypoxic Saanich Inlet. Our time series continuously-recorded, natural cycles of deoxygenation and reoxygenation that allowed us to test whether a community from the NEP showed hysteresis in its recovery compared to hypoxia-induced decline and to address the processes driving temporal beta diversity under variable states of hypoxia. Using high-frequency ecological time series we reveal (1) differences in the response and recovery of the epibenthic community are rate-limited by recovery of the sessile species assemblage, (2) both environmental and biological processes influence community assembly patterns at multiple time-scales, and (3) interspecific processes can drive temporal beta diversity in seasonal hypoxia. Ultimately, our results illustrate how different time scale dependent drivers can influence the response and recovery of a marine habitat under increasing stress from environmental change.