The importance of genetic drift in shaping patterns of adaptive genetic variation in nature is poorly known. Genetic drift should drive partially recessive deleterious mutations to high frequency, and inter‐population crosses may therefore exhibit heterosis (increased fitness relative to intra‐population crosses). Low genetic diversity and greater genetic distance between populations should increase the magnitude of heterosis. Moreover, drift and selection should remove strongly deleterious recessive alleles from individual populations, resulting in reduced inbreeding depression. To estimate heterosis, we crossed 90 independent line pairs of Arabidopsis thaliana from 15 pairs of natural populations sampled across Fennoscandia, and crossed an additional 41 line pairs from a subset of 4 of these populations to estimate inbreeding depression. We measured lifetime fitness of crosses relative to parents in a large outdoor common garden (8448 plants in total) in central Sweden. To examine the effects of genetic diversity and genetic distance on heterosis, we genotyped parental lines for 869 SNPs. Overall, genetic variation within populations was low (median expected heterozygosity = 0.02), and genetic differentiation was high (median FST = 0.82). Crosses between 10 of 15 population pairs exhibited significant heterosis, with magnitudes of heterosis as high as 117%. We found no significant inbreeding depression, suggesting that the observed heterosis is due to fixation of mildly deleterious alleles within populations. Widespread and substantial heterosis indicates an important role for drift in shaping genetic variation, but there was no significant relationship between fitness of crosses relative to parents and genetic diversity or genetic distance between populations.