A Florida Tech team led by computer scientist Siddhartha Bhattacharyya has received funding from the renowned Defense Advanced Research Project Agency (DARPA) to help develop ways to prevent soldiers from experiencing cybersickness in a cognitive attack when using mixed reality systems.
This DARPA research effort, “Modeling and Analysis Toolkit for Realizable Intrinsic Cognitive Security (MATRICS),” is a collaboration led by Collins Aerospace and involving Smart Information Flow Technologies and the RTX Technology Research Center from the private sector and Florida Tech and Iowa State University on the academic side.
MATRICS is an $8 million project overall, and Bhattacharyya and his team will receive $304,000 for their work.
This work falls under the Intrinsic Cognitive Security (ICS) program at DARPA, an effort the agency is tackling before military personnel rely heavily on mixed reality (MR) systems. The program aims to explore and validate the use of mathematical and computational theories, known as formal methods, to provide guarantees that MR system designs mitigate potential cognitive attacks.”
Mixed reality is “a live direct or indirect view of a physical, real-world environment whose elements are augmented by computer-generated sensory input, such as sound, graphics, labels or 3D (animated) models,” according to Science Direct.
Bhattacharyya and his two Ph.D. students, Parth Ganeriwala and Candice Chambers, will investigate modeling and analysis with formal methods and cognitive architectures for MR systems to help detect and then prevent cybersickness.
“It is expected that with the application of a variety of formal methods, a diverse range of cybersickness issues can be captured to prevent the harm that can be inflicted on humans as they are working with MR,” said Bhattacharyya, associate professor and program chair, computer Science and software engineering.
But how can such a system detect if a user is feeling sick?
“If we know the range of values for mixed reality systems such as frame rate, latency and optical flow that work well for normal people, we will be able to detect when the values are out of range and may cause cybersickness,” Bhattacharyya said. “Florida Tech is modeling and verifying the detection and then prevention of physiology-based attacks.”
Cybersickness can cause dizziness, nausea, headache and fatigue.
The research Bhattacharyya, Ganeriwala and Chambers will conduct will use existing physiological and operation-context data to model parameters and equations and then use that to build formal models of mixed-reality user cognition.
The models will then help formally verify the detection of and guarantee the prevention against cognitive attacks that lead to physiology-based cybersickness.
