Models of Learning

• Introduction

  1. Task description

• Reinforcement Learning

  1. Theory
     • Rescorla-Wagner
     • SoftMax
  2. Data plotting
  3. Simulation
     • Example participants
     • More participants
     • Explore Parameters
     • Grid Search
     • Marginal distributions
     • Summary values
  4. Real data
     • Fit the data
     • Statistical inference
  5. Review
     • Modelling how-to

• Bayesian Learning

  1. Coin tossing
     • Probability
     • Likelihood
     • Uncertainty
     • Learning
     • Review
  2. Bandit task
     • Updating
     • Transition function
     • Learning rate

Developed by
Jill O'Reilly
Hanneke den Ouden
-August 2015

Reversal Learning task

You have just completed 135 trials of a reversal learning task.

In this task, 2 slot machines were presented on every trial, each associated with a certain probability of reward.

• Although you were not told this, the payout probabilities for the blue and orange machines were coupled.
  • There was always one ''good'' option and one ''bad'' option
  • The probability of reward when you picked the high reward machine was always 70%
  • The probability of reward when you picked the low reward machine was only 30%.

• At various points during the task we reversed the identity of the high and low reward machine
  • Subjects have to continuously keep track which machine was currently best.

• As a crucial manipulation, there were 2 versions of the task:
  • One version in which the reward probability changed quickly, every 10-20 trials
  • One version where they changed slowly, every 25-35 trials.
• Your subject number determined which version of the task you played.
  • odd subject numbers were assigned the high volatility (volatile) condition
  • even subject numbers the low volatility (stable) condition.

Below are the instructions given to the subjects:

''On each go, you will see a blue and an orange slot machine, one on each side of the screen.
You choose which one to play by pressing left or right.
The slot machines differ in how often they pay out.
You should choose the slot machine that pays out most often.
The pay-out of both machines can change, so you have to keep track of which machine is paying out more, and pick that one.
It is important to realize that the feedback you get ONLY depends on the colour of the slot machine.
So it does not depend on the location, nor on your previous choices.''

Ready to start modelling?

• Start Section A on standard reinforcement learning
• Start section B on Bayesian learning

If you want to do both, we recommend that you complete them in this order.