My Academic Summer - Randomisation, Confidence Intervals and Forest Plots

Today I visited the Psychology Departement at Cardiff University for a crash course in statistics with Craig Hedge, PdD in experimental psychology. Craig is currently working on a Meta-analysis on reliability studies in cognitive control and is the go to guy when you are struggling with statistics. Liz and I had a few specific requests; we wanted to know how Craig carries out randomisation procedures using MatLab, we wanted him to explain confidence intervals and when to use them and we wanted to learn us about forest plotts. 

Randomisation: Craig has previously carried out the randomisation procedures for the DSL+-project. When carrying out a Randomised Controlled Trial (RCT) the participants are randomly placed in either a experimental group or a control group. The process of group placement is called randomisation and should always be done by a third party not involved in the study of question. Neither me nor Liz knew how this procedure was actually carried out, and therefore asked Craig to show us. Using MathLab this turns out to be a relatively straight forward procedure. Knowledge that might come in handy one day!

Before it got complicated...

Confidence Intervals: Next we wanted to understand confidence intervals - what do they tell us and when should they be used? Confidence intervals are used to describe the amount of uncertainty associated with a sample estimate of a population parameter. People interpret them as a measure of precision - in principle if you were to run an experiment 100 times a 95% confidence interval would contain the "true" population value in 95 of those experiments.

If you use IQ as an example the average of the population is 100. If you tested the IQ of a sample of 30 people you might get a confidence interval of 93 - 103. If you repeated the procedure with another sample of 30 people you might get a confidence interval of 98 - 110. If you did this 100 times, you would expect the confidence interval to contain the true value (100, population mean) 95 times. The remaining five times would then have a confidence interval not containing the true value (100, population mean), e.g. 101 - 111.

You can read about confidence intervals in the article; The fallacy of placing confidence in confidence intervals (Morey, Hoekstra, Rouder, Lee and Wagemakers, 2016).

The face of confusion....

So how can confidence intervals be used? One example is when presenting results from a meta-analysis using a forest plot. Two things to take from this is a) if the confidence interval goes through the vertical "0" line the results of the studie is non significant and as such b) if the confidence interval is on the right side of the vertical "0" line the results of the studie is significant. The diamond in the bottom is the average mean of all the included studies. 

Illustration image, Wikipedia commons

Thanks to Craig Hedge for having us visit!

- Silje and Liz -