Carotenoid-based coloration plays an important role in signaling, is often sexually dimorphic, and is potentially subject to directional and/or sex-specific selection. To understand the evolutionary dynamics of such color traits, it is essential to quantify patterns of inheritance, yet nonautosomal sources of genetic variation are easily overlooked by classical heritability analyses. Carotenoid metabolism has recently been linked to mitochondria, highlighting the potential for color variation to be explained by cytoplasmically inherited factors. In this study, we used quantitative genetic animal models to estimate the importance of mitochondrial and sex chromosome–linked sources of genetic variation in coloration in two songbird populations in which dietary carotenoids are either unmodified (great tit plumage) or metabolized into alternative color forms (zebra finch beak). We found no significant Z-linked genetic variance in great tit plumage coloration, while zebra finch beak coloration exhibited significant W linkage and cytoplasmic inheritance. Our results support cytoplasmic inheritance of color in the zebra finch, a trait based on endogenously metabolized carotenoids, and demonstrate the potential for nonautosomal sources to account for a considerable share of genetic variation in coloration. Although often overlooked, such nonautosomal genetic variation exhibits sex-dependent patterns of inheritance and potentially influences the evolution of sexual dichromatism.