Connecting a low-cost off-the-shelf neuroheadset directly to a mobile phone or a tablet and performing real-time signal analysis and visualization would be pretty much science fiction 5 years ago. Yes, 5 years ago we had Nokia N800, Nokia N95, and iPhone. And tablets were simply a different thing than what we have now.
Today we can do it. The technology is there in the form of affordable EEG headsets (think emotiv or neurosky), powerful mobile devices (mobile phones, tablets, and everything in between, from 3 to 10 inches). The software is there: hackable mobile OSes, multiplatform framework Qt (so we don’t have to rewrite everything for every single device). And knowledge is there, sophisticated and fast methods of analyzing EEG signals. All pieces seem to be in place?
There is something more: motivation. Why do we feel it is worth to spend months of work of not-so-dumb people to develop mobile solution instead of simply using standard high-quality (and quite pricy) setups?
There are several reasons for that. Some of them are even pretty good.
Portable in Greek means ‘better’. Well, no, not really. Portability for us is about setup that can be easily deployed pretty much everywhere and is self-contained. No need for power, network connection or furniture (naturally, up to some sane limits).
Mobile setup allows the user to move around without spaghetti of cables dangling around him. Standing, talking, walking around is very difficult with classical EEG setup (although of course possible).
Cheap setup with off-the-shelf components lowers the entry level for researchers, enthusiast, and eventually regular users. One doesn’t need to be an EEG expert to start playing with such setup just for fun. Or if one is a researcher, buying 30+ EEG headsets and the same amount of mobile phones or tablets suddenly doesn’t sound so crazy.
Real time approach is necessary for many end-user oriented techniques, such as neurofeedback or brain-computer interfaces (bci). There is no time to analyze the data offline for minutes or hours: the results must be calculated and delivered right on the spot. This itself creates many unique challenges.
Sophisticated analysis of the EEG signal means going beyond looking at single electrodes in time or frequency domain. As I have mentioned in my previous post, one of the approaches we are using is source reconstruction, where we try to work with actual brain activity, not just the measured signal.
Hack-away approach to the whole system comes from the belief that proper end-user applications can already be created using the setup. We do not make plugins to MATLAB or dependency-heavy pipelines: everything is written in Qt, can be compiled for any platform and will run as long as we can deliver raw packets from the USB dongle (or when the dongle disappears in the future, deliver packets directly via Bluetooth). SmartphoneBrainScanner2 exposes both raw data but also higher level extracted features (e.g. reconstructed sources) so you can plug into the data stream wherever you find it suitable.
Using low-cost mobile setup naturally poses many challenges. There are few electrodes and those that are there are not placed in an optimal way for many applications. Signal gets much more noisy once we allow users to move around. We need to estimate certain parameters (e.g. noise) in real time, instead of simply doing estimation on the whole signal in the post analysis. And so on.
But on the other hand, we currently have several BSc and MSc students working with the system, designing and implementing simple interfaces for neurofeedback, conducting simple experiments confirming established (and some not-so-well-established) paradigms. Folks who never worked with EEG systems before, received a short crush course and could start hacking away. This is a great experience.
There is no denying that what we are doing is somewhere down there inspired by the story of Kinect: sensor developed using really serious research, that was supposed to be a gaming controller but instead is best known for enabling researchers and hobbyist to do this amazing human-computer interaction systems. Kinect didn’t really bring anything technologically new to the table: systems of accurate skeleton tracking, voice recognition or depth sensors were available on the market for a long time. But what Kinect did bring was the ridiculously low entry point: buy one and start playing. Buy 10 if you feel like that. Work with raw data or let the software do tracking for you.
This lowering of the entry barrier is the real value we are trying to create in this project. This includes software, algorithms and definition of novel approaches to EEG experiments. From the researchers perspective, SmartphoneBrainScanner2 is a lab in a pocket: a self-contained inexpensive mobile solution that can deliver stimuli, collect responses and provide framework for real-time analysis of the data. On multiple subjects. Hobbyists and all-around hackers will be able to use it to quickly create brain-computer interfaces, that will work on different time scales: starting from game-like control in the window of milliseconds (e.g. mu suppression) all the way up to slow changes in the user state (e.g. relaxation) taking minutes or hours. And finally users, hopefully sooner than later will be able to use the applications to interface with the machines or as a mean of self-improvement (neurofeedback training). Not in a lab, hour a day for a week but at home, as much and for as long as needed. Current work in neurofeedback looks very promising [Zoefel2011], but the really interesting question is: what will happen after weeks or months of training ourselves?
In the next part I will write about the software we are creating and hopefully will be able to give some idea when everything will go fully open. The fun is just beginning so stay tuned.
[Zoefel2011] Zoefel, B., Huster, R. J., & Herrmann, C. S. (2011). Neurofeedback training of the upper alpha frequency band in EEG improves cognitive performance. NeuroImage, 54(2), 1427-31. Elsevier Inc. doi:10.1016/j.neuroimage.2010.08.078