What if a game could read your mind? What if a game could feel your emotions? What if a game changed along with the players themselves? Does that sound absolutely awesome to you, or does it sound like something straight out of a nightmare?
We cannot undermine the potential EEG (Electroencephalogram) brainwave detection devices have in the future of gaming, especially VR. To prove this, I have created a sound-based horror experience in UE5, where the sonic environment changes based on the players individual brainwaves. A research paper is available where the methodology and the findings of the project have been documented (attached document).
Even though this project mainly focuses on the sound aspect of EEG-driven games, I believe that the methodology can be translated to pretty much every aspect of game development. With technology like this, creating interesting features and gameplay mechanics seems to be only limited by how far the mind can reach (literally 😉).
Struggles, Conclusion & the future
Due to a lack of experience in actual neuroscience, I found it quite hard to interpret and validate the values from the brainwave detection device. However, because of heavy experimentation, I believe it was still possible to connect the data to sound parameters in meaningful ways.
After extensive experimentation with my current brainwave device, I must conclude that there are periods of instability and unreliablility in the returned data. However, I believe that in the future, more advanced, stable and reliable hardware will become cheaper and thus more accessible to regular consumers and game developers. VR is definitely a target for this technology as it could easily integrate brainwave sensors into the bands of the headset itself.
The result of this research is a prototype which proves the concept of brainwave integration in video games. I believe that if more people get involved in this, and more experts in different fields work together on a single project. We could end up with an amazing, never before seen experience which could cause a paradigm shift in gaming.
JUCE
To have a completely different view on how brainwaves could manipulate sounds and its implementation, I have created a custom c++ synthesizer plugin with JUCE which has the possibility to connect its parameters to a brainwave device via OSC messaging. This project was made more for fun than to actual retrieve scientific findings from it. Below you can see the result!
