Prospective contingency explains behavior and dopamine signals during associative learning

Burrell, Mark 1 1 ; Qian, Lechen 1 ; Hennig, Jay A.1; Matias, Sara 1 ; Murthy, Venkatesh N.1 ; Gershman, Samuel J.1 ; Uchida, Naoshige 1

Published Feb 15, 2026 on Dryad. https://doi.org/10.5061/dryad.mkkwh71f6

Data files

Feb 15, 2026 version files 2.92 GB

Code.zip

104.56 KB
Data.zip

2.92 GB
README.md

15.38 KB

Abstract

Associative learning depends on contingency, the degree to which a stimulus predicts an outcome. Despite its importance, the neural mechanisms linking contingency to behavior remain elusive. In the present study, we examined the dopamine activity in the ventral striatum—a signal implicated in associative learning—in a Pavlovian contingency degradation task in mice. We show that both anticipatory licking and dopamine responses to a conditioned stimulus decreased when additional rewards were delivered uncued, but remained unchanged if additional rewards were cued. These results conflict with contingency-based accounts using a traditional definition of contingency or a new causal learning model (ANCCR), but can be explained by temporal difference (TD) learning models equipped with an appropriate intertrial interval state representation. Recurrent neural networks trained within a TD framework develop state representations akin to our best ‘handcrafted’ model. Our findings suggest that the TD error can be a measure that describes both contingency and dopaminergic activity.

Directory Structure

.
├── README.md
├── Code/
│   └── Code/
│       ├── checked_figure_code/     # R code for generating figures
│       ├── matlab_simulation_code/  # MATLAB code for TD and ANCCR simulations
│       ├── photometry_data_analysis/# Python code for photometry analysis
│       ├── python_simulation_code/  # Python code for TD and RNN simulations
│       ├── r_analysis_code/         # R code for statistical analysis
│       └── value_rnn/               # Python package for value-RNN models
└── Data/
    ├── photo_data.parquet           # Processed photometry data
    ├── lick_data.parquet            # Licking behavior data
    ├── trial_data.parquet           # Trial-level behavioral data
    ├── mouse_table.csv              # Mouse metadata (group, sex, photometry)
    ├── Photometry Mice/             # Photometry and behavioral data
    │   ├── Raw_data/                # Raw photometry data (zipped)
    │   └── Preprocessing/           # Code for preprocessing of raw data
    └── Pre_processing_code/         # Data preprocessing scripts

Data Files

File: Data.zip

Main Data Files

File	Description	Format
`mouse_table.csv`	Mouse metadata including group assignment (conditioning/degradation/cuedrew), sex, and photometry status	CSV
`lick_data.parquet`	Trial-by-trial licking timestamps for all mice - variables are: lick (timestamp in seconds, relative to start of trial of lick), trialtype, trialnumber, day (session), mouse (name)	Apache Parquet
`trial_data.parquet`	Trial-level information including trial type, timing, and outcome	Apache Parquet
`photo_data.parquet`	Processed fiber photometry data from ventral striatum: variables are: trial, time (in seconds, relative to start of trial), day (session), data (z-scored photometry signal), mouse (name), site (recording location, see abbreviations below), trial_type	Apache Parquet

Raw Data

Raw photometry data files are provided as zip archives in Data/Photometry Mice/Raw_data/:

CuedRew.zip - Raw data from the cued reward group
Deg.zip - Raw data from degradation group

The organization of these files is as follows:

Photometry Mice\Raw_data\[Group: CuedRew or Deg].zip\[Group: CuedRew or Deg].zip\[animal_name]\Selina_C5D5R3E5R3

Within this, there is one folder called "Session Settings" containing a mat file with behavioral parameters include odor identity.

In the folder called "Session Data" are three folders:

Bpod - containing bpod raw data (files named Animal_Task_Date_session.mat). See bpod wiki for further information about this format: https://sanworks.github.io/Bpod_Wiki/user-guide/

Doric - containing the raw output of the photometry recordings as csv files, following the same naming convention as the bpod folder. See the Doric website for further details on the system: https://neuro.doriclenses.com/collections/software/products/doric-neuroscience-studio

Processed - Mat files, processed using the code provided in Code\photometry_data_analysis\preprocessing. Naming convention as above. Note that this data is compiled into the .parquet files provide as main data files for ease of use and legibility.

Mouse Groups

Mice were assigned to three experimental groups:

Conditioning (Cond): Continued standard conditioning (n=6 behavioral only)
Degradation (Deg): Received uncued background rewards (n=11 total, n=8 with photometry)
Cued Reward (CuedRew): Received cued background rewards (n=13 total, n=5 with photometry)

Photometry Recording Sites

Fiber photometry recordings were made from six locations in the ventral striatum:

Lateral nucleus accumbens (lNAc) - primary analysis site
Medial nucleus accumbens (mNAc)
Anterior lateral olfactory tubercle (alOT)
Anterior medial olfactory tubercle (amOT)
Posterior lateral olfactory tubercle (plOT)
Posterior medial olfactory tubercle (pmOT)

Code Files

File: Code.zip

Figure Generation (`Code/Code/checked_figure_code/`)

R scripts for reproducing manuscript figures:

File	Description
`00_data_import.R`	Data loading and preprocessing
`01_figure1.R` - `07_figure7.R`	Main text figure generation
`S1_figure1.R` - `S6_figure6.R`	Extended data figure generation
`flm_effect_plotter.R`	Functional linear model plotting utilities, called by various figure plotting code

Figure Descriptions

Figure	Brief Description of Associated Figure Code
Figure 1	Summarizes the behavioral (lick timing) data in all conditions
Figure 2	Plots example data and summarizes the fiber photometry data, in particular the response to odor A across all conditions
Figure 3	Plots Temporal Difference (TD) models explaining the pattern of results seen. See the associated MATLAB and Python code for model data generation.
Figure 4	Summarizes the behavioral and photometry data that supports the Belief-State TD model
Figure 5	Summarizes the non-odor A responses in the photometry data and compares against the Belief-State TD model
Figure 6	Summarizes and compares the Value-Recurrent Neural Network (Value RNN) model of the photometry data and visualizes the state space. See the value RNN python code description below for generation of data. Associated files (dataprep, run_cca.py) perform data loading and Canonical Correlation Analysis (CCA), respectively.
Figure 7	Summarizes the ANCCR model data. See the Matlab code section below for details on running the ANCCR models.
Supplementary Figure 1	Summarizes other behavioral data across all groups
Supplementary Figure 2	Summarizes photometry data across all areas recorded and performs pairwise correlation
Supplementary Figure 3	Explores the effect of lick timing on reward response
Supplementary Figure 4	Summarizes the effect of discounting (gamma) on the model predictions of Figure 4
Supplementary Figure 5	Summarizes the predictions of a microstimuli TD model (see python code below)
Supplementary Figure 6	Summarizes other predictions of the value RNN and Belief-State TD model and compares to actual photometry data

Computational Modeling

MATLAB Simulation Code (`Code/Code/matlab_simulation_code/`)

generate_simulated_experiments.m - Generate simulated behavioral experiments
run_belief_state_model.m - Belief-State TD model simulations (Figure 3)
Variations are run_belief_state_model_extinction.m and run_belief_state_model_changetransition. m, which models the Belief-State model of extinction (Figure 5) and the effect of changing transition probabilities on model predictions (Extended Data Fig 4D and E)
run_anccr_model.m - ANCCR model simulations
compute_belief_r2.m - computes the R2 values used to compare the Belief-State model to the value RNN data used in Figure 6
td_belief.m - TD learning with belief state representation - code called by run_belief_state_model to do the TD belief-state modelling. Variation td_belief_garr.m considers outcome-specific value (Extended Data Figure 8)
useful_functions/ - Helper functions called by simulation code
anccr_simulation_cluster_code/ - Versions of ANCCR simulation (see above) designed to be run on Cluster Computing

Python Simulation Code (`Code/Code/python_simulation_code/`)

train_rnns.py - Train value-RNN models (Figure 6)
generate_rnn_test_data.py - Generates simulated data from trained value-RNN models (Figure 6)
main_garr.py - Simulations for outcome-specific degradation (Extended Data Figure 8)
do_td_sim.py - TD model simulations (Figure 3)
do_td_sim_ms_fullrun.py - TD model microstimuli simulations (Extended Data Figure 5)
td_sim/ - TD simulation module - imported by do_td_sim.py.
rnn_analysis/ - helper functions imported by train_rnns.py.

Value-RNN Package (`Code/Code/value_rnn/`)

Python package for training recurrent neural networks to predict value:

valuernn/model.py - RNN model architecture
valuernn/train.py - Training routines
valuernn/tasks/ - Task implementations (contingency, blocking, etc.)

See install instructions below - code is called by train_rnns.py (see above)

Photometry Analysis (`Code/Code/photometry_data_analysis/`)

preprocessing/ - Raw data processing scripts involving the following steps
- a: process bpod and doric files to align behavior and photometry (Matlab code)
- b: combine sessions into a single file per mouse (Python code)

Software Requirements

R (version 4.4+)

Required packages:

tidyverse
arrow
lme4
lmerTest

Python (version 3.9+)

Required packages:

numpy
scipy
matplotlib
pytorch
scikit-learn
pyarrow
pyrcca (for CCA analysis)

Install value-RNN package:

conda create --name valuernn python=3.9 pytorch matplotlib numpy scipy scikit-learn
conda activate valuernn
cd Code/Code/value_rnn
pip install -e .

MATLAB (version 2022b+)

Required toolboxes:

Statistics and Machine Learning Toolbox
Parallel Computing Toolbox (optional, for faster simulations)

Reproducing Results

Behavioral and Photometry Analysis

Set working directory to Code/Code/checked_figure_code/
Ensure data files are in ./data/ subdirectory (or adjust paths in 00_data_import.R)
Run figure scripts in order:

source("00_data_import.R")
source("01_figure1.R")  # etc.

TD Model Simulations

Navigate to Code/Code/matlab_simulation_code/
Run generate_simulated_experiments.m to create simulated experiments
Run run_belief_state_model.m for Belief-State TD simulations
Run run_anccr_model.m for ANCCR comparisons

RNN Model Training

Navigate to Code/Code/python_simulation_code/
Run:

python train_rnns.py

Experimental Methods Summary

Animals

31 mice total (18 wild-type, 13 DAT-Cre x Ai148 transgenic)
Age: 3-6 months
Both sexes (balanced across groups)

Phases

Conditioning (Phase 1): 5 sessions, standard odor-reward pairing
Manipulation (Phase 2): 5 sessions
- Cond: continued conditioning
- Deg: uncued background rewards added
- CuedRew: cued background rewards added
Recovery/Extinction (Phases 3-5): Deg group only

Photometry

GCaMP6f expressed in dopamine neurons (DAT-Cre x Ai148)
Multi-fiber recording from 6 ventral striatum locations
473nm excitation, 405nm isosbestic control
20 Hz effective sampling rate