MULTIMODAL INSIGHTS: FORECASTING BEHAVIOR CHANGE IN SPECIAL EDUCATION NEEDS STUDENTS

Abstract

The availability of educational data in creative ways and formats has given students with special education needs (SEN), whose behaviour and learning are especially sensitive to their physical circumstances and surrounding environments, new possibilities. Multimodal learning analytics (MMLA) gathers and analyses student and learning environment data in a number of ways to clarify the underlying educational ideas. In this work, we used MMLA to forecast the behaviour of special education needs (SEN) children undergoing applied behaviour analysis (ABA) therapy. One kind of special education intervention that aims to address behavioural problems and encourage constructive behaviour changes is ABA treatment. Here, we show that by integrating multimodal educational data, our deep neural network and machine learning models can best predict the behaviour change of SEN children with 98% accuracy and 97% precision. We also demonstrate how adding environmental, psychological, and motion sensor data may statistically improve the performance of prediction models using just traditional educational data. Our efforts have enhanced intensive ABA treatment for over 500 SEN children in Singapore and Hong Kong since 2020 using the Integrated Intelligent Intervention Learning (3I Learning) System.