Understanding how the natural world will be impacted by environmental change over the coming decades is one of the most pressing challenges facing humanity. Addressing this challenge is difficult because environmental change can generate both population level plastic and evolutionary responses, with plastic responses being either adaptive or non-adaptive. We develop an approach that links quantitative genetic theory with data-driven structured models to allow prediction of population responses to environmental change via plasticity and adaptive evolution. After introducing general new theory, we construct a number of example models to demonstrate that evolutionary responses to environmental change over the short-term will be considerably slower than plastic responses, and that the rate of adaptive evolution to a new environment depends upon whether plastic responses are adaptive or non-adaptive. Parameterization of the models we develop requires information on genetic and phenotypic variation and demography that will not always be available, meaning that simpler models will often be required to predict responses to environmental change. We consequently develop a method to examine whether the full machinery of the evolutionarily explicit models we develop will be needed to predict responses to environmental change, or whether simpler non-evolutionary models that are now widely constructed may be sufficient.