Models of Learning

• Introduction

  1. Task description

• Reinforcement Learning

  1. Theory
     • Rescorla-Wagner
     • SoftMax
  2. Data plotting
  3. Simulation
     • Example participants
     • More participants
     • Explore Parameters
     • Grid Search
     • Marginal distributions
     • Summary values
  4. Real data
     • Fit the data
     • Statistical inference
  5. Review
     • Modelling how-to

• Bayesian Learning

  1. Coin tossing
     • Probability
     • Likelihood
     • Uncertainty
     • Learning
     • Review
  2. Bandit task
     • Updating
     • Transition function
     • Learning rate

Developed by
Jill O'Reilly
Hanneke den Ouden
-August 2015

Likelihood of the cause, given the observations

As we have just seen, if we assume some value of q, where q is the p(heads), we can calculate the probability of the observations HHTHT given q.

We can use this fact to help us work out the value of q when this is unknown.

Say I work out the probability of the observations, HHTHT, for every possible value of q from 0 to 1 in steps of 0.01.

Let's try this using the pre-prepared MATLAB code in script 'UncertaintyTutorial1.m'.

• Open the file 'UncertaintyTutorial1.m' in Matlab
• Run section 1 only

  HELP!

  

You should have a plot a bit like this

?

From this plot, we can clearly see that the the observations, HHTHT, were most probable given a certain value of q - what is that value?

?

0.6 - why should this not surprise you?

?

Because if the true value fo q was 0.6, the coin should come up heads 3/5 of the time -
the exact ratio observed in the sequence HHTHT.

You may recall from the lecture that there is a mathematical relationship between

• The probability of the data given the parameters of the environment,
  in this case, p(HHTHT | q)
• The likelihood of the parameters, given the data we observed,
  in this case p(q | HHTHT)

Namely: p(q | data) = p(data | q)

►►►