Multi-Modal Machine Learning Framework for Automated Seizure Detection in Laboratory Rats
A multi-modal machine learning system uses multiple unique data sources and types to improve its performance. We have developed a system that combines results from several types of models, all of which are trained on different data signals. We have applied this system towards an experiment in which multiple types of data are collected from rats suffering from seizures. This data includes both electrocorticography readings and video recordings. For each timestamp, we need to be able to predict whether a seizure is occurring. There are two main ways these kinds of data can be combined: build two models independently and combine the results into a final classification, or build a model that can take in both ECoG and video data and use them simultaneously to create predictions. We are currently experimenting with both of these methods to achieve the best results.
Automating this process is a huge benefit to this research study. Currently, researchers have to manually watch weeks worth of footage for each laboratory rat and identify when seizures are occurring to get an accurate count. By building a system that can take in this data and produce an output listing every predicted seizure and the corresponding timestamps, most of this lengthy and tedious manual review can be prevented, allowing researchers to quickly reach important conclusions about the frequency and severity of seizures.
Due to the rarity of seizure events in the data, false positives are a common problem. Combining multiple data sources helps to alleviate this, allowing for customized weighing of data sources to determine the optimal combination of predictions that catches the most seizures while reducing the number of false positives. For example, many false positives can be filtered out by limiting the final seizure classifications to only time frames in which both models agree that a seizure took place. However, stricter thresholds on seizure classifications can lead to increased false negatives, so a balance is important to maximize both precision and recall.
In practice, the models trained on each of these data sources typically have high recall, meaning they correctly identify when a seizure takes place. However, they also have very low precision, meaning they produce a large number of false positives. Our research has found that when combining the results from these models and performing postprocessing techniques, the proportion of false positives can be significantly reduced while maintaining high recall.
A full paper covering this subject can be found here.
