Phenotypic differences among individuals are often linked to differential survival and mating success. Quantifying the relative influence of genetic and environmental variation on phenotype allows evolutionary biologists to make predictions about the potential for a given trait to respond to selection and various aspects of environmental variation. In particular, the environment individuals experience during early development can have lasting effects on phenotype later in life. Here, we used a natural full-sib/half-sib design as well as within-individual longitudinal analyses to examine genetic and various environmental influences on plumage color. We find that variation in melanin-based plumage color – a trait known to influence mating success in adult North American barn swallows (Hirundo rustica erythrogaster) – is influenced by both genetics and aspects of the developmental environment, including variation due to the maternal phenotype and the nest environment. Within individuals, nestling color is predictive of adult color. Accordingly, these early environmental influences are relevant to the sexually selected plumage color variation in adults. Early environmental conditions appear to have important lifelong implications for individual reproductive performance through sexual signal development in barn swallows. Our results indicate that feather color variation conveys information about developmental conditions and maternal care alleles to potential mates in North American barn swallows. Melanin-based colors are used for sexual signaling in many organisms, and our study suggests that these signals may be more sensitive to environmental variation than previously thought.