A Neural-network-based Investigation of Eye-related Movements for Accurate Drowsiness Estimation

Mingfei Sun, Masanori Tsujikawa, Yoshifumi Onishi, Xiaojuan Ma, Atsushi Nishino, Satoshi Hashimoto

Abstract: Many studies reported that eye-related movements, e.g., blank stares, blinking and drooping eyelids, are highly indicative symptoms of drowsiness. However, few researchers have investigated the computational efficacy accounted for drowsiness estimation by these eye-related movements. This paper thus analyzes two typical eye-related movements, i.e., eyelid movements $X_{el}(t)$ and eyeball movements $X_{eb}(t)$, and investigates neural-network-based approaches to model temporal correlations. Specifically, we compare the effectiveness of three combinations of eye-related movements, i.e., $[X_{el}(t)]$, $[X_{eb}(t)]$, and $[X_{el}(t),X_{eb}(t)]$, for drowsiness estimation. Furthermore, we investigate the usefulness of two typical types of neural networks, i.e., CNN-Net and CNN-LSTM-Net, for better drowsiness modeling. The experimental results show that $[X_{el}(t), X_{eb}(t)]$ can achieve a better performance than $[X_{el}(t)]$ for short time drowsiness estimation while $[X_{eb}(t)]$ alone performs worse even than the baseline method (PERCLOS). In addition, we found that CNN-Net are more effective for accurate drowsiness level modeling than CNN-LSTM-Net.