Correlates of EEG and agency in a single arm reaching visual manipulation

Background 

Experiments on the distortion of movement (mismatch between seen and felt position/orientation of a limb) have shown that humans may be misled by vision regarding the actual position of his/her limb. We proposed a protocol to track these thresholds of perception by manipulating single reaching moments in Virtual Reality (VR), which are assessed through an awareness of distortion question. This project will expand from that protocol, in order to also record electroencephalography (EEG). The goal is to explore Error Related Potential (ErrP) in the EEG signal to verify if it correlates with the responses provided to the awareness of distortion question, which in turn will be used to prevent the usage of distortions above a certain threshold by the application.

Project Idea

In the present experiment, we classify the distortions we apply to a virtual avatar as positive or negative, meaning that the virtual avatar will deviate in order to either facilitate or constrict the task. According to this factor as well as the magnitude of distortion, the discrepancy will eventually become apparent to the subject, and is reported by her on an awareness of distortion question.

 

Nevertheless, it has been shown that ErrP present in the EEG signal can be used to indicate when a system provides erroneous feedback, and has been widely investigated in the context of Brain Computer Interface (BCI). For instance, to improve precision of EEG classifiers when ErrP point to incorrect choices made by the classifier.

 

The student will analyze the related components of the ErrP in order to verify the minimum threshold of distortion that can be detected through them, and whether it is possible to correlate the signal with the subjective response to the agency question.

The student will:

1. Integrate the EEG equipment (g.tec usbamp and active electrodes), which will collect EEG signal in synchrony with displayed Unity events (samples of similar implementation will be provided).

2. Adapt the Unity scene used to detect the threshold of perception for distorted movements.

3. Design of experiment with the help of the project proponent.

4. Implement the experiment and data recording scripts. (datalog scripts for Unity are provided, but it is likely that they will have to be adapted to accommodate the proposed measurements)

5. Conduct an experiment to collect data.

6. Analyze data and look for correlates with distortion awareness question.

7. Propose a model that retrieves whether the distortion might have been noticed by the subject through the EEG signal (ErrP)

8. Test the proposed model in an adapted application that will avoid presenting distortions that elicits strong ErrP components.

Assuming that steps 1 through 6 will yield results that allow the implementation of steps 7 and 8 is an optimistic foresight. Thus, if step 6 shows that the data is insufficient or uncorrelated with awareness of distortion, steps 7 and 8 are replaced by deepening the methods and approaches of analysis of the collected data, as well as exploring what we can assume about the distortion from the EEG alone, even if uncorrelated with the subject awareness of distortion.

 
Development framework:

The development of this project is based on the interactive game engine Unity 3D.

Hardware:

Head mounted display – Oculus Rift DK2;

Motion tracking – Phasespace / playstation move controllers

EEG – g.tech usbamp with active electrodes.

Requirement:

Programming (C# or Javascript for Unity3D).

References:

Chavarriaga, Ricardo, Aleksander Sobolewski, and José del R. Millán. “Errare machinale est: the use of error-related potentials in brain-machine interfaces.” Frontiers in neuroscience 8 (2014).

Knoblich, Günther, and Tilo TJ Kircher. “Deceiving oneself about being in control: conscious detection of changes in visuomotor coupling.” Journal of Experimental Psychology: Human Perception and Performance 30.4 (2004): 657.

Kohli, Luv, Mary C. Whitton, and F. P. Brooks. “Redirected touching: training and adaptation in warped virtual spaces.” 3D User Interfaces (3DUI), 2013 IEEE Symposium on. IEEE, 2013.

The Disappearing Hand Trick https://www.youtube.com/watch?v=Qcm0_QPhLe4

Contact:

 

Henrique GALVAN DEBARBA (henrique.galvandebarba at epfl.ch INJ139 )
Ronan BOULIC (Ronan.boulic at epfl.ch   INJ141 )