Fatigue Detection Based on Facial and Vocal Features Using Dynamic Fuzzy Neural Network In air traffic controller

Abstract

Fatigue among air traffic controllers (ATCs) has become a critical issue for flight safety, particularly with the increasing volume of global air traffic. Accurately detecting fatigue is essential, as it directly influences the safety and operational efficiency of air traffic control. In this study, we propose a non-invasive approach to fatigue detection by analyzing both facial and vocal characteristics of ATCs. We first developed efficient methods for facial feature extraction, enabling us to track indicators such as "percentage of eyelid closures" and yawning frequency from video footage. Additionally, we extracted a range of vocal features from audio data, including average fundamental frequency, short-time average magnitude, short-time zero-crossing rate, harmonic-to-noise ratio, jitter, shimmer, loudness, and Mel-frequency cepstral coefficients. These facial and vocal features were transformed into temporal sequences and fed into a dynamic fuzzy neural network (DFNN). By combining these data with the Stanford Sleepiness Scale, we were able to accurately assess and predict ATC fatigue levels