Multi-Modal Machine Learning Framework for Automated Seizure Detection in Laboratory Rats

A multi-modal machine learning system uses multiple unique data sources and types to improve its performance. We have developed a system that combines results from several types of models, all of which are trained on different data signals. We have applied this system towards an experiment in which multiple types of data are collected from rats suffering from seizures. This data includes electrocorticography readings, piezoelectric motion sensor data, and video recordings. Each of these data sources can be used to independently build machine learning models, the results of which can be combined into a final classification to automatically determine when seizures occur throughout the specimen monitoring process. This can reduce or prevent the need for manual review, which is a lengthy and tedious process, and allows researchers to quickly reach important conclusions about the frequency and severity of seizures.

The advantage of a multi-modal approach is the ability to weigh the results of each model to reach the most accurate conclusion. Individual models struggle with overall accuracy due to the rarity of true seizure events. False positives are typically abundant across all data sources, due to both noisy data and imbalanced class labels. By combining the results of each model, many of these false positives can be filtered out by limiting the final seizure classifications to only time frames in which all models agree that a seizure took place. However, stricter thresholds on seizure classifications can lead to increased false negatives, so a balance is important to maximize both precision and recall.

In practice, the models trained on each of the three data sources typically have high recall, meaning they correctly identify when a seizure takes place. However, they also have very low precision, meaning they produce a large number of false positives. Our research has found that when combining the results from these three models and performing postprocessing techniques, the proportion of false positives can be significantly reduced while maintaining high recall.

Link to Full Paper: https://arxiv.org/abs/2402.00965#