The restricted distribution and isolation of island endemics often produces unique genetic and phenotypic diversity of conservation interest to management agencies. However, these isolated species, especially those with sensitive life history traits, are at high risk for the adverse effects of genetic drift and habitat degradation by non-native wildlife. Here, we study the population genetic diversity, structure, and stability of a classic “island giant” (Xantusia riversiana, the Island Night Lizard) on San Clemente Island, California following the removal of feral goats. Using DNA microsatellites, we found that this population is reasonably genetically robust despite historical grazing, with similar effective population sizes and genetic diversity metrics across all sampling locations irrespective of habitat type and degree of degradation. However, we also found strong site-specific patterns of genetic variation and low genetic diversity compared to mainland congeners, warranting continued special management as an island endemic. We identify both high and low elevation areas that remain valuable repositories of genetic diversity and provide a case study for other low-dispersal coastal organisms in the face of future climate change.