High levels of gene flow among partially isolated populations can overwhelm selection and limit local adaptation. This process, known as “gene swamping,” can homogenize genetic diversity among populations and reduce the capacity of a species to withstand rapid environmental change. We studied brown anole lizards (Anolis sagrei) distributed across seven islands in The Bahamas. We used microsatellite markers to estimate gene flow among islands and then examined the correlation between thermal performance and island temperature. The thermal optimum for sprint performance was correlated with both mean and maximum island temperature whereas performance breadth was not correlated with any measure of temperature variation. Gene flow between islands decreased as the difference between mean island temperatures increased, even when those islands were adjacent to one another. These data suggest that phenotypic variation is the result of either 1) local genetic adaptation with selection against immigrants maintaining variation in the thermal optimum, 2) irreversible forms of adaptive plasticity such that immigrants have reduced fitness, or 3) an interaction between fixed genetic differences and plasticity. In general, the patterns of gene flow we observed suggest that local thermal environments represent important ecological filters that can mediate gene flow on relatively fine geographic scales.