learning & decision-making

Reinforcement learning tutorial

Want to use learning models to look at your data, but don't know where to start? Together with Jill O'Reilly we developed this on-line tutorial, to give beginning modellers a taste of how the use of computational models may help us understand behaviour. We aim to give you a flavour of the mechanics and different steps involved. We focus on learning to implement and fit very basic learning models from two classes of models: one reinforcement learning model, and one normative Bayesian learning model, and show you how you can start to use these models understand your subjects' behaviour. We first developed this tutorial for the Donders Institute summerschool  on Computational Approaches to Decision Making, August 10-14 in Nijmegen, the Netherlands, and by popular demand decided to put it online. We hope that it will help you to take your first steps in the wonderful world of computationals models. 

Dynamic Causal Modelling for fMRI data - getting started

I often get questions about where and how to get started using DCM, so what I will present here is a short summary of what DCM is, and some pointers as to where to go next. 

The aim of dynamic causal modeling (DCM) is to infer the causal architecture of coupled or distributed dynamical systems. It is a Bayesian model comparison procedure that rests on comparing models of how data were generated. DCM was developed for estimating coupling among brain regions and how that coupling is influenced by experimental changes (e.g., time, context or drug). The basic idea is to construct reasonably realistic models of interacting (cortical) regions or nodes. These models are then supplemented with a forward model of how the hidden states of each node (e.g., neuronal activity) map to measured responses (e.g. BOLD resonse). This enables the best model and its parameters (i.e., effective connectivity) to be identified from observed data. Bayesian model comparison is used to select the best model in terms of its evidence (inference on model-space), which can then be characterised in terms of its parameters (inference on parameter-space). DCM allows us to test hypotheses about how nodes communicate; e.g., whether activity in a given neuronal population modulates the coupling between other populations, in a task-specific fashion. (edited from DCM Scholarpedia page)

Where to learn more? 

If you want start using DCM, I would recommend to attend one of the SPM courses, for example

• Computational Approaches to NeuroImaging Toolkit, Donders Institute for Brain, Cognition and Behaviour
• SPM course, Translational Neuromodelling Unit, ETH Zurich. Great speakers, and good chocolate of course
• SPM short course, Wellcome Trust Centre for Cognitive Neuroimaging, UCL. Return to the mothership
• Also, the SPM mailing list is a good source answering many of the questions, including ones that you may not even have asked yourself yet!
• If you cannot afford to go on a course, you can watch the online lectures

Literature

Over the years I have compiled a short (and certainly non-exhaustive) reading list that I always present at the end of my lectures, and thought it would be nice to share with you here. When I have time, I will complete the links below. Until then, pubmed is your friend ;-). 

To get you started 

• 10 Simple Rules for DCM (2010). Stephan et al. NeuroImage. - This paper focusses mainly on model comparison
• Understanding DCM: Ten simple rules for the clinician (2013). Kahan & Foltynie NeuroImage. Excellent, also for non-clinicians!
• DCM scholarpedia page
• Understanding DCM, 10 simple rules for the clinician (2013). Kahan & Foltynie. NeuroImage 
• The first DCM paper: Dynamic Causal Modelling (2003).  Friston et al. NeuroImage.
• Bayesian model comparison: Comparing families of dynamic causal models (2010). Penny et al. PLoS Comput Biol.