Abstract:

In this study, we developed machine learning models to predict the valence class and valence rating of emotions felt by participants based on their brain activity and physiological responses. The ICBHI 2024 Scientific Challenge involves using a rich dataset comprising pre-processed fMRI, photoplethysmography (PPG), and respiratory data from 20 participants. Each participant watched emotion-provoking video clips categorized into three valence classes (positive, negative, neutral) and rated them on a nine-level scale.

Our approach integrates Convolutional Neural Networks (CNN) for analyzing fMRI data and Long Short-Term Memory (LSTM) networks for handling PPG and respiratory data. The model is trained to classify the valence class and predict the valence level, using a combination of categorical cross-entropy and mean squared error as loss functions.

Initial results show promising trends, indicating the model's potential for accurate emotion prediction. However, further fine-tuning and architectural adjustments are necessary to enhance performance. Our work aims to contribute to the understanding of how brain activity and physiological responses can be used to decode emotional states, with potential applications in psychological assessment and therapeutic interventions.

Results: Link to submission file: https://github.com/alexsalman/ICBHI2024/blob/main/submission.csv

Contact: Ali Salman a.salman@student.unisi.it "University of Siena" Team