Models of Learning

• Introduction

  1. Task description

• Reinforcement Learning

  1. Theory
     • Rescorla-Wagner
     • SoftMax
  2. Data plotting
  3. Simulation
     • Example participants
     • More participants
     • Explore Parameters
     • Grid Search
     • Marginal distributions
     • Summary values
  4. Real data
     • Fit the data
     • Statistical inference
  5. Review
     • Modelling how-to

• Bayesian Learning

  1. Coin tossing
     • Probability
     • Likelihood
     • Uncertainty
     • Learning
     • Review
  2. Bandit task
     • Updating
     • Transition function
     • Learning rate

Developed by
Jill O'Reilly
Hanneke den Ouden
-August 2015

Reversal learning task

Let's look at how Bayes can be used to model the reversal learning task you ran earlier.

As we saw in the previous section of this tutorial, we can model learning using repeated applications of Bayes' theorem.

Actually, we can think of the observations in the reversal learning task as analogous to the coin tosses in the previous section of the tutorial:

• heads <-> orange rewarded
• tails <-> blue rewarded
• q is p(heads) <-> q is p(orange rewarded)

At the end of that tutorial, you saw how the posterior probability distribution over candidate values of q evolved with each new observation (coin toss).

Imagine that instead of coin tosses, out observer was playing the reversal learning task.
Now our probability distributions over q tell us something quite interesting:

The probability distribution tells us what our participant believes about which option (orange/blue) is rewarded on each trial

Run the model

The Matlab script BayesNoVolatility.m is a Bayesian learning model for the one-armed-bandit reversal learning task.

• Open up the script BayesNoVolatility.m and run it.
• It will produce various figures, this tutorial will talk you through them

►►►