Models of Learning

• Introduction

  1. Task description

• Reinforcement Learning

  1. Theory
     • Rescorla-Wagner
     • SoftMax
  2. Data plotting
  3. Simulation
     • Example participants
     • More participants
     • Explore Parameters
     • Grid Search
     • Marginal distributions
     • Summary values
  4. Real data
     • Fit the data
     • Statistical inference
  5. Review
     • Modelling how-to

• Bayesian Learning

  1. Coin tossing
     • Probability
     • Likelihood
     • Uncertainty
     • Learning
     • Review
  2. Bandit task
     • Updating
     • Transition function
     • Learning rate

Developed by
Jill O'Reilly
Hanneke den Ouden
-August 2015

Probability of the observations, given the cause

Let the probability that out coin comes up "heads" be q.

If we knew the value of q, then we could work out the probability of the outcomes above as follows:

Outcome 1: H			Probability: q
Outcome 2: H			Probability: q
Outcome 3: T			Probability: 1-q
Outcome 4: H			Probability: q
Outcome 5: T			Probability: 1-q

Because each coin toss is independent, we can get the overall probability of the sequence HHTHT by multiplying probabilities:

p(HHTHT) = q x q x (1-q) x q x (1-q)

So, for any given value of q, you should be able to tell me the probability of obtaining the sequence HHTHT

• For example, say q=0.6, what is the probability of the sequence HHTHT?

  ?

  p(HHTHT) = 0.6 x 0.6 x 0.4 x 0.6 x 0.4
  p(HHTHT) = 0.0346
  

• And what about if q=0.4, what is the probability of the sequence HHTHT then?

  ?

  p(HHTHT) = 0.4 x 0.4 x 0.6 x 0.4 x 0.6
  p(HHTHT) = 0.0230
  

• Was the sequence HHTHT more probable given that q, the probability the coin comes up heads, is 0.4, or 0.6?

  ?

  0.6
  

• Why should this not surprise you?

  ?

  In the observed sequence HHTHT there are more heads than tails.
  So this sequence would be more probable given a coin which favours heads (q=0.6) than a coin that favours tails (q=0.4).

►►►