Depression severity detection from facial expressions in videos using recurrent neural networks

Abstract

Major Depressive Disorder (MDD), commonly known as depression, impacts over 300 million people worldwide. Diagnosing this condition often relies on subjective judgment, emphasizing the need for an objective approach. Deep learning, particularly Recurrent Neural Networks (RNNs), offers a promising solution, especially for analyzing multivariate time series data. Researchers have explored the use of RNNs and their variants to detect depression severity. Therefore, this study conducts experimental testing to identify depression severity using first derivative techniques and feature engineering on the RNN model and its variations. The first derivative is calculated for each frame during the subject’s interview, and feature engineering techniques focus on the eyes and lips, known to be associated with depression, including calculations of upper and lower lip distances, eye openness, and more. The RNN model, incorporating feature engineering, achieved the best results among the proposed methods, with a Mean Absolute Error (MAE) of 5.04 and Root Mean Squared Error (RMSE) of 6.03. However, further performance enhancement is possible by increasing the number of layers and neurons, considering the current model’s relative simplicity due to limited resources.