Invasive non-indigenous species are defined by their impacts: they substantially change native communities or ecosystems. Accordingly, invasive species might transform their habitats in ways that eventually become unfavorable to them, causing population declines or even extirpations. Here we use over 40 years of systematically collected data on the abundance of the invasive rusty crayfish Faxonius rusticus from 17 lakes in northern Wisconsin, USA to explore if population declines of this invader are related to the prevalence of rocky habitat, which shelters crayfish from predators and is unchanged by crayfish. We predicted that lakes with rock-dominated substrates would be resistant to F. rusticus population declines, whereas lakes lacking rock-dominated substrates would experience F. rusticus declines due to crayfish destruction of shelter-providing macrophytes. We found that in nearly half (47%) of the study lakes, F. rusticus experienced population declines over the study time period, and these lakes had significantly lower proportions of rock substrate than lakes that did not experience population declines. We recommend that more studies should investigate the potential for invasive species-mediated community or ecosystem feedbacks to eventually contribute to their own population declines.