According to theory, drift load in randomly mating populations is determined by past population size, because enhanced genetic drift in small populations causes accumulation and fixation of recessive deleterious mutations of small effect. In contrast, segregating load due to mutations of low frequency should decline in smaller populations, at least when mutations are highly recessive and strongly deleterious. Strong local selection generally reduces both types of load. We tested these predictions in 13 isolated, outcrossing populations of Arabidopsis lyrata that varied in population size and plant density. Long-term size was estimated by expected heterozygosity at 20 microsatellite loci. Segregating load was assessed by comparing performance of offspring from selfings versus within-population crosses. Drift load was the heterosis effect created by interpopulation outbreeding. Results showed that segregating load was unrelated to long-term size. However, drift load was significantly higher in populations of small effective size and low density. Drift load was mostly expressed late in development, but started as early as germination and accumulated thereafter. The study largely confirms predictions of theory and illustrates that mutation accumulation can be a threat to natural populations.