Poster C64, Friday, August 17, 10:30 am – 12:15 pm, Room 2000AB
The impact of sample size on the reproducibility of voxel-based lesion-deficit mappings
Diego Lorca-Puls1, Andrea Gajardo-Vidal1, Jitrachote White1, Mohamed Seghier1,2, Alexander Leff1, David Green1, Jenny Crinion1, Philipp Ludersdorfer1, Thomas Hope1, Howard Bowman3,4, Cathy Price1;1University College London, 2Emirates College for Advanced Education, 3University of Kent, 4University of Birmingham
This study investigated how sample size affects the reproducibility of findings from univariate voxel-based lesion-deficit analyses (e.g., voxel-based lesion-symptom mapping and voxel-based morphometry). Our effect of interest was the strength of the mapping between brain damage and speech articulation difficulties, as measured in terms of the proportion of variance explained. First, we identified a region of interest by searching on a voxel-by-voxel basis for brain areas where greater lesion load was associated with poorer speech articulation using a large sample of 360 right-handed English-speaking stroke survivors. We then randomly drew thousands of bootstrap samples from this data set that included either 30, 60, 90, 120, 180, or 360 patients. For each resample, we recorded effect size estimates and p values after conducting exactly the same lesion-deficit analysis within the previously identified region of interest and holding all procedures constant. The results show (i) how often small effect sizes in a heterogeneous population fail to be detected; (ii) how effect size and its statistical significance varies with sample size; (iii) how low-powered studies (due to small sample sizes) can greatly over-estimate as well as under-estimate effect sizes; and (iv) how large sample sizes (N ≥ 120) can yield highly significant p values even when effect sizes are so small that they become trivial in practical terms. Recognised solutions to the issues highlighted here include using: (a) larger sample sizes; (b) multivariate methods; (c) informed sampling strategies; and (d) independent replications.
Topic Area: Methods