Presentation
Search Abstracts | Symposia | Slide Sessions | Poster Sessions | Lightning Talks
Decoding brain states during language processing: deep learning applied on fMRI temporal sequences
There is a Poster PDF for this presentation, but you must be a current member or registered to attend SNL 2023 to view it. Please go to your Account Home page to register.
Poster A93 in Poster Session A, Tuesday, October 24, 10:15 am - 12:00 pm CEST, Espace Vieux-Port
Tiphaine Caudrelier1, Adrien Boustié1, Monica Baciu1, Martial Mermillod1; 1Univ. Grenoble Alpes, Univ. Savoie Mont Blanc, CNRS, LPNC, 38000 Grenoble, France
Language is a cognitive function which relies on various brain regions and functional networks depending on the specific task performed by the brain. It is possible to isolate brain regions involved in specific aspects of language (e.g. phonological processing) using fMRI contrasts. It is also possible to identify specific networks based on functional connectivity analysis. However, some spatiotemporal patterns may not be visible through these types of fMRI analyses. Is there a specific algorithm that could learn to classify brain states related to different languages tasks from temporal sequences of fMRI signal? And what if the system could tell us which voxels are important to differentiate these tasks? Deep learning is a supervised learning technique which can classify objects (e.g. sequences of images) without any a priori about the features to take into account to perform the classification. It only needs a large amount of data to be trained on. Deep learning is increasingly used to decode brain states from fMRI signals thanks to the emergence of large-scale open-access fMRI databases like the dataset from the Human Connectome Project – hereafter HCP (Van Essen et al., 2013). HCP database includes fMRI signals recorded during multiple tasks relying on various cognitive functions (e.g motor, short-term memory, language, social) in more than thousand participants. Convolutional Neural Networks (hereafter CNN) have been shown to be able to classify those various cognitive functions/tasks from HCP with up to 97% accuracy (Jiang et al., 2022). Recently, transfer learning of CNN from big datasets to smaller datasets has turned out successful. Would deep neural networks be able to distinguish brain states at a more fine-grained level and differentiate various language tasks? In the present study we trained a CNN on the classification of seven cognitive tasks from the HCP. Using a transfer learning approach, we then fine-tuned this CNN on the InLang dataset which includes 13 different language tasks structured into five groups (Roger et al., 2022). We used a backpropagation algorithm to detect which voxel were critical to perform the classification. The obtained CNN was able to classify fMRI temporal sequences in those five groups with a validation accuracy of 67% while typical machine learning methods (e.g. Support Vector Machine, Linear Discriminant Analysis) were at chance level (20%). Results and perspectives are discussed. References: Jiang, Z., Wang, Y., Shi, C. W., Wu, Y., Hu, R., Chen, S., Hu, S., Wang, X., & Qiu, B. (2022). Attention module improves both performance and interpretability of four-dimensional functional magnetic resonance imaging decoding neural network. Human Brain Mapping, 43(8), 2683‑2692. https://doi.org/10.1002/hbm.25813 Roger, E., Rodrigues De Almeida, L., Loevenbruck, H., Perrone-Bertolotti, M., Cousin, E., Schwarz, J., Perrier, P., Dohen, M., Vilain, A., Baraduc, P., Achard, S., & Baciu, M. (2022). Unraveling the functional attributes of the language connectome : Crucial subnetworks, flexibility and variability. 1‑54. Van Essen, D. C., Smith, S. M., Barch, D. M., Behrens, T. E. J., Yacoub, E., & Ugurbil, K. (2013). The WU-Minn Human Connectome Project : An overview. NeuroImage, 80, 62‑79. https://doi.org/10.1016/j.neuroimage.2013.05.041
Topic Areas: Computational Approaches, Methods