Conventional distance sampling is a logistically feasible method for estimating the densities of unmarked animals. The probability density function (PDF) of the sampling area specifies the expected proportion of the population that occurs at each distance from the observer and is a fundamental component of distance sampling models. Current approaches set this PDF either to equal probability at each distance, for line transects, or an increasing probability for point transects (because sampling area increases with radial distance from a point). Geographic Information Systems allow measurements of the area viewable from a given location (i.e., the viewshed), the structure of which may not reflect theoretical PDFs for either line (rectangular) or point (circular) transects. We simulated three datasets to test how variation in the viewshed structure affects estimates of detection probability, abundance, and density. We then implemented a novel application of Bayesian distance sampling models to test the magnitude of parameter bias recovered by accounting for incomplete viewsheds. Lastly, we compared parameter estimates from Bayesian hierarchical models that used either traditional or custom PDFs to analyze a dataset of 95 county-level spotlight surveys of white-tailed deer (Odocoileus virginianus) in Iowa, USA. For empirical data, viewable sampling area decreased with distance at an average rate of 3% every 100 m (range from 1–7% among counties). Our model correction decreased county-level density estimates by an average of 18% (range from 13–27% among counties), which depended on how sharply visibility declined. We suggest incomplete viewsheds be handled by considering the expected distribution of animals inside and outside of the viewshed. More generally, we show that customizing a PDF to more accurately reflect the study system improves density estimates and offers flexibility when the distribution of animals from the observer deviates from traditional assumptions.

Deer were sampled with nocturnal spotlight surveys along gravel roads in Iowa, USA.