Social interactions among individuals are abundant, both in natural and domestic populations, and may affect phenotypes of individuals. Recent research has demonstrated that the social effect of an individual on the phenotype of its social partners may have a genetic component, known as an Indirect Genetic Effect (IGE). Little is known, however, of non-genetic factors underlying such social effects. Early life environments often have large effects on phenotypes of the individuals themselves later in life. Offspring development in many mammalian species, for example, depends on interactions with the mother and siblings. In domestic pigs, individuals sharing the same juvenile environment develop similar body weight later in life. We, therefore, hypothesized that offspring originating from the same early-life environment also develop common social skills, which generate Early-Life Social Effects (ELSE) that affect the phenotypes of their social partners later in life. We, therefore, quantified IGEs and ELSEs on growth in domestic pigs. Results show that individuals from the same early-life environment express similar social effects on the growth of their social partners, and that such ELSE shape the growth rate of social partners more than IGE. Thus, the social skills that individuals develop in early-life have a long-lasting impact on the phenotypes of social partners. Early-life and genetic social effects were independent of the corresponding direct effects of offspring on their own growth, indicating that individuals may enhance the growth of their social partners without a personal cost. Our findings also illustrate how research devoted to quantifying IGEs may miss non-genetic and potentially confounded social mechanisms, which may bias the estimates of IGEs.