Maintaining and manipulating sequential information online is essential for daily activities such as planning. In Parkinson’s disease (PD), deficits in sequential working memory have been associated with altered regional activation and functional connectivity within the basal ganglia. This study demonstrated that the substantia nigra (SN) integrity correlates with basal ganglia dysfunction during sequential working memory. We included 29 patients with PD and 29 healthy controls (HC). We assessed the SN integrity using neuromelanin-sensitive MRI and sequential working memory using functional MRI and a digit ordering task. In this task, participants either recalled a sequence of digits in the original order (‘pure recall’) or rearranged the digits in ascending order and recalled the new sequence (‘reorder & recall’). PD showed smaller SN areas than HC, especially on the left side. Compared to HC, PD showed lower task accuracy, hyper-activation of the subthalamic nucleus, hypo-activation of the caudate nucleus and globus pallidus, and weakened functional connectivity between the bilateral SN and all three basal ganglia regions. Moreover, PD showed distinct brain-behavior and structure-function relationships. Ordering-related accuracy cost (‘reorder & recall’ versus ‘pure recall’) correlated with the ordering-related subthalamic activation in HC, but with the ordering-related caudate activation in PD. In PD, the caudate activation correlated with the total SN area, whereas the subthalamic activation correlated with daily exposure to D2/3 receptor agonists. In conclusion, damage to the SN may lead to basal ganglia dysfunction sequential working memory deficits even in early PD.