Pod shattering, which causes the explosive release of the seeds from the pod, is one of the main sources of yield losses in cowpea in arid and semi-arid areas. Reduction of shattering has therefore been a primary target for selection in the domestication process of the species. Using a mini-core diversity panel of 368 cowpea accessions, four regions with a significant association with pod shattering were identified. Two genes (Vigun03g321100 and Vigun11g100600) involved in cell wall biosynthesis were identified as strong candidates for pod shattering. Microscopical analysis was conducted on a subset of accessions representing the full spectrum of shattering phenotypes. This analysis indicated that the extent of wall fiber deposition was highly correlated with shattering. The results from this study also confirm that pod shattering in cowpea is exacerbated by arid environmental conditions. Finally, using a subset of West African landraces, patterns of historical selection for shattering resistance related to precipitation in the environment of origin were identified. Together, these results shed light on sources of resistance to pod shattering, which will, in turn, improve climate resilience of a major global nutritional staple.