Models of Learning

• Introduction

  1. Task description

• Reinforcement Learning

  1. Theory
     • Rescorla-Wagner
     • SoftMax
  2. Data plotting
  3. Simulation
     • Example participants
     • More participants
     • Explore Parameters
     • Grid Search
     • Marginal distributions
     • Summary values
  4. Real data
     • Fit the data
     • Statistical inference
  5. Review
     • Modelling how-to

• Bayesian Learning

  1. Coin tossing
     • Probability
     • Likelihood
     • Uncertainty
     • Learning
     • Review
  2. Bandit task
     • Updating
     • Transition function
     • Learning rate

Developed by
Jill O'Reilly
Hanneke den Ouden
-August 2015

Simulating data

We often think about modelling in terms of fitting a model to a participant's behaviour.

However, we can also use the model to simulate what an 'ideal' participant would do, given certain values of the parameters such as learning rate.

This can help us get an intuitive understanding of:

• how these parameters may describe behaviour
• what sort of behaviour our model may capture

Therefore, before we are going to fit the behavioural data, we will first actually generate or simulate some data. This means that we will use our model, which is simply the 2 equations above, to 'play' the slot machine game.

The equations

The codes implement the update and observation equations.

• Open the file RLtutorial_simulate.m
• Can you find the observation (softmax) equation?

  ?

  ev = exp(beta*v); % exponentiate the value
  sev = sum(ev); % compute the sum of the values
  p = ev/sev; % probability each choice
  % i.e. ratio of each of the values and their sum. % This code effectively does:
  % p(1) = ev(1)/sev;
  % p(2) = ev(2)/sev;
  

• Can you find the update (Rescorla-Wagner) equation?

  ?

  dv = r-v(c); % compute prediction error
  v(c) = v(c) + alpha*dv; % update value
  

►►►