Many organisms have evolved phenotypic plasticity but examples of a heritable genetic basis or genetic constraints for plasticity across environments remain scarce. Tadpoles of the Neotropical treefrog Dendropsophus ebraccatus alter tail coloration and shape differently in response to fish or aquatic insect predators. To assess the genetic basis of plasticity we raised 1020 tadpoles from 17 maternal half-sib pairs (34 unique families) individually with chemical cues of fish or aquatic insects, or with cue-free control water. We used Bayesian animal models to estimate narrow sense heritability of morphology and cross-trait genetic correlations in all three treatments, heritability of plasticity in response to each predator, and genetic correlations between responses to fish and insects. Families showed remarkably different responses to predators and heritability was often high (0.45–0.75), as was heritability of plasticity itself (0.42–0.62). We detected strong negative genetic correlations for responses to each predator (-0.45 and -0.59), providing clear evidence of a limit to plasticity. Most importantly, we show that prey genotypes are constrained in their capacity to respond to different types of predators, which likely maintains genetic variation for plasticity in a temporally and spatially variable landscape where there is no single adaptive peak.