Reinforcement learning models in reversal learning
Comparing fixed-rate and Bayesian learning models against behavioural data from a two-armed bandit task.
Jan 2025 – Mar 2025
I conducted a computational modeling project to understand how individuals learn and adapt in a dynamic decision-making environment, specifically a two-armed bandit task with reward contingency reversals. I aimed to model trial-by-trial learning by comparing two reinforcement learning models: the Rescorla-Wagner Model (RWM), with its fixed learning rate, and the Bayesian Reinforcement Learning Model (BRM), incorporating adaptive, uncertainty-driven learning. Our primary hypothesis was to determine which model best explained participants’ learning curves in this reversal paradigm.
I fitted these models to human behavioural data from 11 participants across 100 trials. The task involved a reversal at trial 50, shifting reward probabilities. My specific contribution included the full implementation and analysis of the Bayesian RL model, covering model simulation, parameter recovery, model fitting to human data, and comparison. I used maximum likelihood estimation for fitting and compared model fits using the Bayesian Information Criterion (BIC).
Based on BIC, the RWM provided a better fit to the observed behaviour (BIC = 958.6) compared to the BRM (BIC = 1046.6). This supports the hypothesis that a fixed learning rate mechanism was sufficient to describe participant behavior in this task, despite the BRM’s capacity for adaptive learning. The winning RWM model parameters indicated moderate learning and decision sensitivity to values.