Brain Scanner - Computerphile

Key Takeaways

my work involves using a brain scanner the one you can see here to measure people's mental workload while they perform various tasks so this means that we can tell when software sucks as well as when we made it better humans have limited resources like a computer processors if you want so let's draw a good line here and people can allocate less or more resources to the task in hand there's two interesting points here one is close to the maximum resources and one is close to when we don't allocate enough resources or we're doing something repetitive over a period of time the best example I can think about is an air traffic controller he's working very intense for long periods of times therefore as soon as he's reaching a close Point next to maximum resources he has he might be prone to mistakes on on the other hand train drivers do a repetitive task maybe driving hours waiting for the next train station therefore they are very close to boredom obviously the air traffic controller is doing lots of mental processing keeping planes in the air or Landing planes and that and that space there is being taken up underneath the graph and then the available resources for I don't know responding to people talking to him and things are getting less and less and less that's right and it's always good to have a balance so don't work too hard don't work too little I saw a video recently and I'm going a bit off topic of a racing driver driving along and then trying to do some text messages and crashing the car exactly so thinking about I don't know designing a satnav uh that is dedicated to a driver you need to think about that the driver is already doing a task what we can do with this technique is to measure where on the graph the user is we kind of want to find out more about the user during interaction and and we had couple of experiments where we actually used it in I don't know for measuring usability of of a new system or comparing two two version of a website and the layout and and see what impact has on on on on users's brain but what we are looking to create here is a system to tell users their mental workload in real time so kind of alerting you for instance if you're reaching the the high workload level we were talking about before so maybe on the graph so maybe you can reallocate your resources take a break ask for help we're trying to measure this effect that it might have on you as well as if you're not working very hard then we're trying to communicate this to you maybe don't so you're not getting bored and maybe doing something else so so both ends of that graph are a problem both both hands are a problem and keeping the right balance would be ideal um and now we should invite Max I think we're joined by Max from our wonderful uh search engine videos who is involved in this project as well so tell me what you're going to do then what we're going to do is connect his brain activity data in real time to the machine on the right this machine is going to classify the workload in real time is going to sense when Max is reaching a high workload or low workload and then we're going to communicate the workload we're going to give Max some feedback using the lamps so when the workload increases to a high point the lights will become red depending on what Max is doing he's getting busier and busier or if he's getting towards relax or bored them they're going to change back to White so we can kind of monitor Max as well as give him feedback during his interaction with the computer now I'm going to ask Max to play a game an air traffic controller game the game is called airport mads 4 and your job as a player is to play the role of an air traffic controller manage The Landings and departures of planes we're ready we can see Max's brain activity increasing and decreasing this is his forehead or what's happening behind his forehead and the red dots here mean increased activity so if we ask Max to pause and calm down and relax you can see that it kind of goes away or something happens so now if we start again the experiment I can connect it to the lights and what do those lines all mean that you're looking at there then this is the raw signal this is um normally this technology measures how um much oxygen you have behind your forehead how much oxygenated the blood behind your forehead is however this is just the road the raw signals and you control the intensities of each light depending on the skin color of the participant you need to have the right level so it reaches the right depth in your in your brain we're taking a baseline now that means we're asking Max to relax while we take an average of his brain activity right now we connected the lights that means we can control them from this machine they will change color when Max is getting very very busy or his brain becomes really really active so now can you start playing the game please okay so max already feels that he made a mistake and something is getting stressed stressed and more stressed and more stressed because it's getting busier and busier so if I ask Max some questions now it's going to make it even worse hello can you remind me what Pythagoras Theorem is um has to do with triangles the lights are about 5 seconds behind what um max is going through this much it takes from the the heart to pump up the blood the brain and observe the change so let's say Max is getting busy 5 seconds ago the lights will turn on red now hello his phone's going as well the wife thing is they they were already red they've gone white since you picked the phone up now try to go back to your task Max hello he's back he's back on the traffic controller job he's man maning his own workload apparently [Laughter] no so now what you have to do is to control the little aircraft that come from the left side because you became better at the task they give you more more work to do like in real life jobs so now you need to control the aircraft on the small terminal I'm trying to make Max um distracted from the task a little bit I'm trying to distract him so his de oxygenation values reach minus 5 which is just crazy are you trying to relax now okay do that going to crush let it go let it go can you pause you cannot relax you cannot relax until you POS the game is this typical then typical Behavior yeah yeah yeah you can't because your mind still stays in the game you can see he's trying to to stay calm but as soon as he can see a plane approaching a crack is going back in the game thanks Max for your time I'm going to just take it off you now thank you very much you're welcome have fun what exactly do you take away from an experiment like that well analyzing brain data is not an easy step it involves um a lot of signal processing we need to clear up the movement artifacts that Max potentially did transforming the raw data into meaningful oxygenation information uh and brain activity information um and you can have a look at this scen this is the data after it's been cleared from any kind of movement artifact and you can see that there is some interesting moments the signal going up and down but you can also have a more understandable if you want picture at a brain and see which parts were active during which events uh using markers we track interesting events or um with the limit between conditions and then having something like a visual picture of the brain can give us some meaningful information so the next step could be moving towards U more wearable user friendly devices we're try we're trying to use uh things like smart watches like this bit of Kit to understand whether we need or we don't need the brain scanner Really we want something that people can wear and if we can infer brain activity from something like this then we might just give up the brain scanner cuz everybody's using a smartwatch these days and fitbits and other some sort of equipments so what we want is a measure that will say this result set is better than that result set and so for this what we use is an average Precision score you have to do a lot of Matrix solving to work that one out and that's the difference in the problem that gets given to us