1. Climate change is expected to alter not only year-to-year variation in climate but also aspects of within-year variation, such as the length of the intervals between rainfall events and the duration of heat waves. Yet we still have a poor understanding of how intra-annual climate variability and individual weather events alter key vital rates (e.g. individual growth, reproduction and survival) that in part determine population dynamics. 2. Traditionally, ecologists have accounted for this variability across long time periods by attempting to correlate annual vital rates with measures of within-year variability (e.g., the coefficient of variation) to match the scale at which demographic data are collected. An alternative to this aggregate approach is to use within-season yet still relatively infrequent censuses in a probabilistic framework to determine the most likely way that vital rates respond to shorter-term variation in the environment, and how these short-term changes cumulatively lead to the observed yearly vital rates. 3. Here, I present an approach for inferring daily responses of vital rates to short-term weather variability, and apply it to understand how five species of summer annual plants in the Chihuahuan Desert will respond to climate change. 4. Vital rate models reveal that species differ in their responses to above- and below-average conditions, but generally fall into two life histories: 1) species that have fast growth on favorable days, but also experience higher mortality on less favorable days, and 2) species that have slower growth in the same conditions but lower mortality. Results show that the expected rainfall changes in the Chihuahuan desert (more late, cool season rainfall and less frequent, more intense rainfall events) could reduce growth and increase mortality of all summer annual plants. 5. Synthesis. My study shows that vital rates can change in response to short-term variability even when the total amount of rainfall and average temperature, common covariates in demographic models, remain constant. Accounting for changes in short-term environmental variation in climate change predictions will likely be important in systems with considerable environmental variation between censuses. The approach I present here can be widely applied to understand how short-term variability and individual weather events will alter organism responses to climate change.