Recently, deep learning models have been successfully applied in fMRI modeling and associated applications. However, there still exist at least two challenges. Firstly, due to the lack of sufficient data, deep learning models tend to suffer from overfitting in the training process. Secondly, it is still challenging to model the temporal dynamics from fMRI, due to that the brain state is continuously changing over scan time. In addition, existing methods rarely studied and applied fMRI data augmentation. In this work, we construct a deep recurrent variational auto-encoder (DRVAE) that combined variational auto-encoder and recurrent neural network, aiming to address all of the above mentioned challenges. The encoder of DRVAE can extract more generalized temporal features from assumed Gaussian distribution of input data, and the decoder of DRVAE can generate new data to increase training samples and thus partially relieve the overfitting issue. The recurrent layers in DRVAE are designed to effectively model the temporal dynamics of functional brain activities. LASSO (least absolute shrinkage and selection operator) regression is applied on the temporal features and input fMRI data to estimate the corresponding spatial networks. Extensive experimental results on 7 tasks from HCP dataset showed that the DRVAE and LASSO framework can learn meaningful temporal patterns and spatial networks from both real data and generated data. The results on group-wise data and single subject suggest that the brain activities may follow certain distribution. Moreover, we applied DRVAE on 4 resting state fMRI datasets from ADHD-200 for data augmentation, and the results showed that the classification performances on augmented datasets have been considerably improved, further demonstrating the effectiveness of the proposed DRVAE on fMRI data modeling and augmentation.
© 2021 IOP Publishing Ltd.