Chimpanzees Rationally Revise Their Beliefs

This repository contains data, code, and output for the paper "Chimpanzees Rationally Revise Their Beliefs," published in Science.

For questions or comments, please contact Emily M. Sanford at esanford@berkeley.edu.

Analysis Files

• [All_Analyses.Rmd]: Contains all analyses for all experiments.
  • The first chunk in the file specifies the packages that must be installed in R to run the script.
  • To run the full analysis, you will need to download [Functions.zip] and place the unzipped folder in your working directory.
  • You must also download [Stan_Models.zip] and place the unzipped folder in your working directory to fit the Bayesian computational models.
• [R_Data_Part_1.RData] and [R_Data_Part_2.RData]: These two files contain the results of running the [All_Analyses.Rmd] file, which is time consuming to run. These objects can be loaded in the first chunk of [All_Analyses.Rmd] to view the outputs of the analysis without running the entire script.

This repository contains four zip folders:

1. [Data.zip]: .csv files containing all data for all experiments.
2. [Functions.zip]: .R files containing auxiliary functions required to run the [All_Analyses.Rmd] file.
3. [Stan_Models.zip]: .stan files containing the computational models tested for each experiment.
4. [Unintentional_Cueing_Analysis.zip]: Contains sub-folders with box position coding data for each experiment, and the [Unintentional_Cueing_Analysis.Rmd] file to run this control analysis.

The contents of each folder are explained in more detail below.

1. [Data]

• [experiment_1_data.csv]: Data from Experiment 1
  • Date: Date on which the data were collected
  • Chimpanzee: Subject name
  • Counterbalance: Order in which the evidence conditions were presented to this subject across trials. There are two possible counterbalancing orders:
    • strong_first_weak_second: This subject was tested in the "strong first" condition (where strong evidence was presented prior to weak evidence) before they were tested in the "weak first" condition (where weak evidence was presented prior to strong evidence)
    • weak_first_strong_second: This subject was tested in the "weak first" condition (where weak evidence was presented prior to strong evidence) before they were tested in the "strong first" condition (where strong evidence was presented prior to weak evidence)
  • Trial: Trial number for this subject overall (integer, ranges from 1-11)
  • Condition: Indicates the order evidence was presented in on this trial
    • strong_first: Strong evidence was presented before weak evidence (Trial structure: Strong evidence, Choice 1; Weak evidence, Choice 2)
    • weak_first: Weak evidence was presented before strong evidence (Trial structure: Weak evidence, Choice 1; Strong evidence, Choice 2)
    • filler_strong: Filler trial where the subject was rewarded for their first choice; only strong evidence was presented (Trial structure: Strong evidence, Choice 1)
    • filler_weak: Filler trial where the subject was rewarded for their first choice; only weak evidence was presented (Trial structure: Weak evidence, Choice 1)
  • First_Choice: Indicates which box (the one supported by strong or weak evidence) the subject chose on their first choice in that trial
    • On filler trials, the subject chose between the evidence box (either strong or weak, depending on the condition) and a no evidence box (empty)
  • Second_Choice: Indicates which box (the one supported by strong or weak evidence) the subject chose on their second choice in that trial
    • Note: On filler trials, subjects only made one choice, so there was no Second_Choice; this is denoted by a /
  • Belief_Revision: Indicates whether the subject revised their beliefs (i.e., switched from one option to the other) between their First_Choice and Second_Choice
    • If First_Choice = Second_Choice, Belief_Revision = No
    • If First_Choice =/= Second_Choice, Belief_Revision = Yes
    • If there was no Second_Choice (as on filler trials), Belief_Revision = /
  • trialType: Binary variable indicating whether a trial was a filler trial (0) or a test trial (1)
  • cond: Binary variable indicating whether a trial was in the strong_first condition (1) or the weak_first condition (0)
  • choice1: Binary variable indicating whether the subject chose the strong evidence box (1) or the weak evidence box (0) on their first choice
  • choice2: Binary variable indicating whether the subject chose the strong evidence box (1) or the weak evidence box (0) on their second choice; this is NA for filler trials
  • belief_r: Binary variable indicating whether the subject changed their response from choice 1 to choice 2 (1) or gave the same response for both choices (0); this is NA for filler trials
  • evidence1: Categorical variable indicating what kind of evidence was presented first
    • visual_strong: experimenter turned box around, subject could see food inside through glass pane
    • auditory_weak: experimenter picked up and shook box, subject could hear something moving inside
  • choice1_ev1: Binary variable indicating whether, at their first choice, the subject chose the box that was supported by the first piece of evidence (1) or they chose the box that had not yet received any evidence (0)
  • trial.per.evidence: Integer variable indicating how many times the subject received that evidence type (strong or weak) as the first piece of evidence (ranges from 1-6)
  • subjID: Numeric subject ID
  • trial.per.condition: Integer variable indicating trial number for this subject for this condition (ranges from 1-4)
    • Note: trial.per.evidence and trial.per.condition are distinct from one another because of the filler trials. E.g., trial.per.condition differentiates between the strong_first condition (where subjects received two pieces of evidence and made two choices) from the filler_strong condition, where subjects received one piece of evidence and made only one choice. trial.per.evidence only corresponds to the first evidence received, so strong_first and filler_strong are categorized together in that variable.
• [experiment_2_data.csv]: Data from Experiment 2. The structure of the data is identical to Experiment 1 (see above), with the following exception:
  • evidence1: Categorical variable indicating what kind of evidence was presented first
    • auditory_strong: experimenter picked up and shook box, subject could hear something moving inside (equivalent to auditory_weak from Experiment 1)
    • visual_weak: experimenter moved boxes to the sides of the table, subject could see a trail of food leading towards the box
• [experiment_3_data.csv]: Data from both versions of Experiment 3 (Version 1 is included in the main text of the paper, while Version 2 is included in the supplemental materials)
  • Chimpanzee: Subject name
  • Date: Date on which the data were collected
  • Trial: Trial number for this subject in this version of Experiment 3 (integer, ranges from 1-8)
  • First.choice: Categorical variable indicating which box the subject chose for their first choice in version 2 only
    • NA: Version 1, does not apply
    • S: Box supported by strong evidence
    • W: Box supported by weak evidence
    • N: Box supported by no evidence
  • Second.choice: Categorical variable indicating which box the subject chose for their second choice in version 2 only
    • NA: Version 1, does not apply
    • S: Box supported by strong evidence
    • W: Box supported by weak evidence
    • N: Box supported by no evidence
  • choice1: Numeric variable corresponding to the subject's first choice in both versions 1 and 2
    • Version 1:
      • 1: Box supported by weak evidence
      • 0: Box supported by no evidence
    • Version 2:
      • 2: Box supported by weak evidence
      • 1: Box supported by strong evidence
      • 0: Box supported by no evidence
  • choice2: Numeric variable corresponding to the subject's second choice in version 2 only
    • 1: Subject chose the remaining box supported by the most evidence. If subject chose the strong box (S) at Choice 1, this corresponds to choosing the weak box (W); if they chose the weak (W) or no evidence (N) box at Choice 1, this corresponds to choosing the strong (S) box
    • 2: Subject chose the remaining box supported by the least evidence. This indicates that the subject either chose the no evidence (N) box at Choice 2, or that they chose the no evidence box at Choice 1 and the weak box (W) at Choice 2
  • target: Categorical variable indicating whether the subject chose the two boxes supported by evidence (S and W) in any order for choices 1 and 2
    • yes: Subject chose S for Choice 1 or 2 and W for Choice 1 or 2
    • no: Subject chose N for either Choice 1 or 2
  • targetN: Binary variable converted from target; yes = 1, no = 0
  • z.Trial: Continuous variable; z-scaled Trial variable
  • Version: Numeric variable with two levels, indicates which version of the experiment this trial belongs to
    • 1: Version 1 (included in the main text of the paper) (Trial structure: Subject received strong evidence for one box, weak evidence for a second box, and no evidence for a third box; The strong evidence box was removed; Subject made a single choice)
    • 2: Version 2 (included in the supplementary materials of the paper) (Trial structure: Subject received strong evidence for one box, weak evidence for a second box, and no evidence for a third box; Subject made a choice; The box they chose was removed; Subject made a second choice)
  • subjID: Numeric subject ID
• [experiment_4_data.csv]: Data from Experiment 4
  • Chimpanzee: Subject name
  • date: Date on which the data were collected
  • session: Each chimpanzee was tested for two days, this variable indicates which of those two days the given trial was collected on
  • counterbalance: Order in which the evidence conditions were presented to this subject across trials. There are two possible counterbalancing orders:
    • independent_first_then_dependent: This subject was tested in the "independent evidence" condition before they were tested in the "dependent evidence" condition
    • dependent_first_then_independent: This subject was tested in the "dependent evidence" condition before they were tested in the "independent evidence" condition
  • trial: Integer, ranging from 1-10. Trial number for this subject for this evidence type, with missing numbers corresponding to filler trials that were not analyzed
  • trial.2: Integer, ranging from 1-8. Trial number for this subject for this condition (re-numbered to exclude filler trials)
  • condition_name: Indicates what type of evidence was presented on this trial
    • independent: In the independent evidence condition, subjects heard the sound of food dropping into the box (Trial structure: Auditory dropping evidence, Visual evidence, Choice 1; Auditory dropping evidence, Choice 2)
    • dependent: In the dependent evidence condition, subjects heard the sound of food rattling around in the box when it was picked up and shaken (Trial structure: Auditory shaking evidence, Visual evidence, Choice 1; Auditory shaking evidence, Choice 2)
  • First.evidence: Indicates which box (the box on the left or right side of the table) was given the first evidence
  • Second.evidence: Indicates which box (the box on the left or right side of the table) was given the second evidence (will always be opposite from First.evidence)
  • First.choice: Indicates which box the chimpanzee chose at Choice 1 (the box on the left or right side of the table)
  • Third.evidence: Indicates which box (the box on the left or right side of the table) was given the third evidence
  • Second.choice: Indicates which box the chimpanzee chose at Choice 2 (the box on the left or right side of the table)
  • Belief.revision: Indicates whether or not the chimpanzee switched choices between Choice 1 and Choice 2 (yes or no)
  • cond: Binary variable indicating whether a trial was in the independent condition (0) or the dependent condition (1)
  • strongSide: Indicates which side (left or right) received strong (visual) evidence
  • choice1: Binary variable indicating whether the subject chose the visual evidence box (1) or the auditory (shaking or dropping) evidence box (0) on their first choice
  • choice2: Binary variable indicating whether the subject chose the visual evidence box (1) or the auditory (shaking or dropping) evidence box (0) on their second choice
  • belief_r: Binary variable indicating whether the subject changed their response from choice 1 to choice 2 (1) or gave the same response for both choices (0)
  • conditionF: Repeat of condition_name variable
  • subjID: Numeric subject ID
  • trial.per.condition: Repeat of trial.2
• [experiment_5_data.csv]: Data from Experiment 5
  • Chimpanzee: Subject name
  • date: Date on which the data were collected
  • session: Each chimpanzee was tested for two days, this variable indicates which of those two days the given trial was collected on (with NAs for "defeater alone" trials, which were tested after the main test trials)
  • counterbalance: Order in which the evidence conditions were presented to this subject across trials. There are two possible counterbalancing orders:
    • defeater_first: This subject was tested in the "defeater" condition before they were tested in the "non-defeater" condition
    • non_defeater_first: This subject was tested in the "non-defeater" condition before they were tested in the "defeater" condition
    • NA: Defeater alone trials
  • all.trials: Integer, ranging from 1-18. Trial number for this subject for this evidence type. Includes filler and failed trials.
  • trial: Character variable indicating the trial number for that condition. Filler trials are indicated by "F_" and "extra_filler." Trials that were cancelled and repeated (because they did not follow the strong evidence initially) are indicated by "failed." Otherwise, trials are numbered from 1 to 8 for the defeater and non_defeater conditions, and 1-10 for the defeater_alone condition.
  • domain: Indicates whether the defeated evidence is in the visual or auditory domain
  • condition: Variable indicating trial condition (numeric for test trials)
    • filler or extra_filler: Filler trial (Trial structure: Strong evidence, Choice 1)
    • failed: Trial failed (terminated and repeated) because subject did not follow strong evidence at choice 1
    • 1: Defeater condition (Trial structure: Weak evidence, Strong evidence, Choice 1; Defeater for Strong box, Choice 2)
    • 0: Non-defeater condition (Trial structure: Weak evidence, Strong evidence, Choice 1; Non-defeater for Strong box, Choice 2)
    • 2: Defeater alone condition (Trial structure: Defeater, Choice 1)
  • first.evidence: Categorical variable indicating what type of evidence was presented first
    • For test trials in the visual domain, this is auditory_shaking evidence
    • For filler trials in the visual domain, this is blurry_visual evidence
    • For defeater_alone trials in the visual domain, this is the visual_defeater (a pane of glass with a sticker of the food on it)
    • For test trials in the auditory domain, this is visual_trace evidence
    • For filler trials in the auditory domain, this is auditory_shaking evidence
    • For For defeater_alone trials in the auditory domain, this is the auditory_defeater (a rock that is removed from the box and tapped on its side)
  • first.evidence.side: Indicates which box (the box on the left or right side of the table) was given the first evidence
  • second.evidence: Categorical variable indicating what type of evidence was presented second
    • For filler and defeater_alone trials, this is NA
    • For test trials in the visual domain, this is blurry_visual evidence
    • For test trials in the auditory domain, this is auditory_shaking evidence
  • second.evidence.side: Indicates which box (on the left or right side of the table) was given the second evidence
    • This is NA for filler & defeater_alone trials
  • first.choice: Categorical variable indicating which side the chimpanzee chose in Choice 1 (left or right)
  • third.evidence: Categorical variable indicating what type of evidence was presented third
    • For filler & defeater_alone trials, this is NA
    • For test trials in the visual domain, this is the visual_defeater (defeater condition) or visual_non_defeater (non_defeater condition)
    • For test trials in the auditory domain, this is the auditory_defeater (defeater condition) or auditory_non_defeater (non_defeater condition)
  • third.evidence.side: Indicates which box (left or right) was given the third evidence
    • This is NA for filler & defeater_alone trials
  • second.choice: Categorical variable indicating which side the chimpanzee chose in Choice 2 (left or right)
    • This is NA for filler & defeater_alone trials
  • food.received: Categorical variable indicating whether the chimpanzee was rewarded (yes or no) at the end of this trial
  • belief.revision: Indicates whether or not the chimpanzee switched choices between Choice 1 and Choice 2 (yes, no, or NA if there were not two choices on this trial)
  • alone: Indicates whether the trial was in the defeater_alone condition (yes or no)
  • condition_name: Categorical variable indicating condition for test trials & defaeter_alone trials only (defeater or non-defeater)
  • defeatedSide: Categorical variable indicating which side (left or right) received defeater or non-defeater evidence
  • defeatedTime: Categorical variable indicating whether the first (defeater_alone trials only) or second (defeater and non-defeater conditions) evidence presented on this trial was defeated
  • choice1: Binary variable indicating whether the subject chose the strong evidence box (0) or the weak evidence box (1) on their first choice
    • For the defeater_alone condition, this is 1 for the side that did not receive the defeater and 0 for the side that did receive the defeater
  • choice2: Binary variable indicating whether the subject chose the strong evidence box (0) or the weak evidence box (1) on their second choice
  • belief_r: Binary variable indicating whether the subject changed their response from choice 1 to choice 2 (1) or gave the same response for both choices (0)
  • subjID: Numeric subject ID
  • conditionF: Categorical variable indicating whether the trial was in the defeater, non-defeater, or defeater_alone condition (blank for trials that were not included in the main analyses)
  • trial.per.condition: Numerical variable indicating trial # (within condition) for that chimpanzee

2. [Functions]

• [fitting_and_plotting_functions.R]: Contains organizing functions to fit all GLMs and Stan models, and to generate all plots
  • inv_logit: Defines inverse logit function
  • glm_prep: Takes inputs of data, formula for GLM, list of dummy variable(s), and name of "trial" variable; Returns list containing summary of fixed and random effects, and data in correct format for the specified GLM
  • glm_assumptions: Takes input of GLM model; Returns list containing results of model assumption checks (overdispersion, variance inflation factor if vif=T, and stability)
  • glm_bootstraps: Takes inputs of model, desired number of bootstraps, and variables to include; Returns bootstrapped results
  • glm_many_models: Executes the '100 altered models' analysis used for models with complete separation issues (Experiment 1 - 2nd choice GLMs, Full Analysis GLMs; Experiment 2 - 2nd Choice GLMs); Fits N models, perturbing the problematic data by a small amount each time, then compiles the average results from these models to estimate the true model values
    • Inputs: N = # models (default = 100), n_boots = # bootstraps (default = 10), model_data = data to be used in models, cells_to_change = list of problematic conditions, response_var = name of response variable, store_df = dataframe object to store analysis results
    • Returns a list containing:
      • Number of models that converged (out of N)
      • Dataframe containing model information for all perturbed models
      • Dataframe of model comparisons (for all N perturbations)
      • Dataframe of averaged coefficients across all N full models
      • List of dataframes containing bootstraps for all models
  • fit_stan_models: Fits specified computational models for each experiment
    • Inputs: data and experiment number (to determine which models to run)
    • Converts data to correct form for the specified models
    • Fits all Bayesian models for that experiment:
      • Experiments 1 & 2: Rational Choice, Belief Perseveration, Recency Bias, Cue Saliency, Null
      • Experiments 3 & 4: Rational Choice, Cue Saliency, Null
      • Experiment 5: Rational Choice, D Alone, Cue Saliency, Null
    • Outputs: List containing Stan fit objects and log likelihoods for all fitted models
  • med_post_lik: Takes log likelihood object from Stan fit, returns median posterior log likelihood for that model
  • collate_med_post_lik: Takes list of LLs for multiple models, returns DF containing median posterior likelihoods for those models
  • extract_params: Takes inputs of Stan model fit objects, subject names, model names, and a list containing lists of parameters in each model; Returns dataframe containing best-fitting parameters for all subjects for all models
  • plot_halfViolin: Returns a half-violin plot with subject points and CIs
    • Inputs: data, xvar = variable for x-axis, yvar = variable for y-axis, colvar = variable for coloring, model_ci = dataframe containing model CIs, levels = levels of x variable, plotlabels = labels to be displayed for each level of x variable, beginCol = value to start color scale, endCol = value to end color scale, ylim = y-axis limits (default = c(0,1)), ylab = label for y-axis (default = "% following evidence"), jitterWidth = amount to jitter x-values (default = 0), adjust.width = binary value indicating whether half-violin needs to be moved to the side to make room for all subject points (default = F)
  • plot_sigmoid: Returns a plot containing a sigmoid (cumulative gaussian) with overlaid points representing subject and average fitted evidence strengths for each evidence type
    • Inputs: dataframe containing parameter fits, dataframe containing CIs from parameter fits
    • Note: Only used for Experiments 1 & 2, so the parameters are fixed to eStrong & eWeak
• [storage_objects.R]: Contains code that generates empty dataframes to store the results of the '100 altered models' analyses for Experiment 1 (2nd choice GLMs, Full Analysis GLMs) and Experiment 2 (2nd Choice GLMs)

The following functions in the [Functions] folder were written by Roger Mundry (RMundry@dpz.eu).

• [boot_glm.r]: Contains function [boot.glmm.pred] to run bootstraps to fit confidence intervals for model estimates and predicted values
• [diagnostic_fcns.r]: Contains functions to perform diagnostics for evaluating GLM assumptions and stability
  • ranef.diagn.plot: Generates a random effect diagnostic plot
  • overdisp.test: Overdispersion test
  • fe.re.tab: Fixed/random effects table, which helps to determine which random slopes are needed
  • m.stab.plot: Plot model stability or CIs
• [glmm_stability.r]: Contains function [glmm.model.stab] to calculate model stability by removing one individual at a time and fitting the model again; The output shows the range of estimates

3. [Stan_Models]

• Experiments 1 and 2:
  • [rational_model_exp1_2.stan]: Rational Choice Model. Assumes that b1 (i.e., the subject's belief at Choice 1) is a function of the first evidence, b2 is a function of the sum of the first evidence + second evidence
  • [cue_model_exp1_2.stan]: Cue Saliency Model. Assumes that b1 is a function of the first evidence, b2 is a function of only the stronger of the first and second evidence
  • [persev_model_exp1_2.stan]: Belief Perseveration Model. Assumes that b1 and b2 are both functions of the first evidence only
  • [recency_model_exp1_2.stan]: Recency Bias Model. Assumes that b1 is a function of the first evidence, b2 is a function of the second evidence only
  • [null_model_exp1_2.stan]: Null Model. Assumes that neither choice is driven by evidence
• Experiment 3:
  • Experiment 3.1:
    • [rational_model_exp3_v1.stan]: Rational Choice Model. Assumes that choice is driven by weak evidence alone
    • [null_model_exp3_v1.stan]: Null Model. Assumes that choice is not based on evidence
  • Experiment 3.2:
    • [rational_model_exp3_v2.stan]: Rational Choice Model. Assumes that b1 is based on the sum of evidence for 3 options, b2 is based on the sum of the evidence for the remaining 2 options
    • [cue_model_exp3_v2.stan]: Cue Saliency Model. Assumes that b1 is based on the strong evidence, and b2 is a coin flip (if the strong box was chosen first) or only on the strong evidence (if the strong box was not chosen first)
    • [null_model_exp3_v2.stan]: Null Model. Assumes that choices are not based on evidence
• Experiment 4:
  • [rational_model_exp4.stan]: Rational Choice Model. Assumes that b1 is a function of the sum of the first two evidences, and that b2 is a function of b1 + the third evidence, but only in the New (independent) evidence condition
  • [cue_model_exp4.stan]: Cue Saliency Model. Assumes that b1 and b2 are both functions of the stronger of the first two evidences
  • [null_model_exp4.stan]: Null Model. Assumes that choices are not based on evidence
• Experiment 5:
  • [rational_model_exp5.stan]: Rational Choice Model. Assumes that b1 is a function of the sum of the first two evidences, and that b2 is a function of b1 + the defeater (which will negate the strong evidence in the defeating condition only)
  • [d_alone_model_exp5.stan]: Defeater Alone Model. Fit to trials from the defeater alone condition only; Assumes that the b1 is a function of the defeater when presented alone (i.e., without first-order evidence)
  • [cue_model_exp5.stan]: Cue Saliency Model. Assumes that b1 and b2 are both functions of the stronger of the first two evidences
  • [null_model_exp5.stan]: Null Model. Assumes that choices are not based on evidence

4. [Unintentional_Cueing_Analysis]

• [Experiment 1] through [Experiment 5]: Folders containing box coding data for each experiment
  • [..._Coding.csv]: Coder #1's judgment for which box is closer - Left (L), Right (R), or Same distance (S) - for every choice in the given experiment
  • [..._Reliability.csv]: Coder #2's judgments on a random 20% of choices for each experiment
  • [..._Image_Key.csv]: File used to link blind coding with the experimental data
• [Unintentional_Cueing_Analysis.Rmd]: File to run Unintentional Cueing Analysis (Did chimpanzees pick the box that was closest to them, rather than the box that had received more evidence?)
  • Lines 6-616: Import and clean data, add condition information to determine which box had received more evidence at the time of each choice, and compiles all experiments into a single data frame for meta-analysis
  • Lines 618-741: Unintentional cueing analysis, collapsed across all experiments
  • Lines 743-1293: Unintentional cueing analysis, separately for each experiment
  • Lines 1295-1439: Compiled results and plots for reliability and unintentional cueing analyses. You can run the entire script, then view all results together in these three final chunks.