Reliable estimates of effective population size Ne are of central importance in population genetics and evolutionary biology. For populations that fluctuate in size, harmonic mean population size is commonly used as a proxy for (multi-) generational effective size. This assumes no effects of density dependence on the ratio between effective and actual population size, which limits its potential application. Here we introduce density dependence on vital rates in a demographic model of variance effective size. We derive an expression for the ratio Ne/N in a density regulated population in a fluctuating environment. We show by simulations that yearly genetic drift is accurately predicted by our model, and not proportional to 1/(2N) as assumed by the harmonic mean model, where N is the total population size of mature individuals. We find a negative relationship between Ne/N and N. For a given N, the ratio depends on variance in reproductive success and the degree of resource limitation acting on the population growth rate. Finally, our model indicate that environmental stochasticity may affect Ne/N not only through fluctuations in N, but also for a given N at a given time. Our results show that estimates of effective population size must include effects of density dependence and environmental stochasticity.