Physiologic monitoring of immersive interaction

Physiologic monitoring of immersive interaction



3-health sensor

e-Health Sensor Platform


In this project, we would like to track mental states by monitoring physiological changes during an immersive interaction task or game. One of the possible solutions is using Biofeedback, for example some monitoring equipment to monitor certain physiological activities during the immersive playing. In our lab, we bought e-Health Sensor Shield that allows Arduino and Raspberry Pi users to perform biometric and medical applications where body monitoring is needed by using 10 different sensors: pulse, oxygen in blood (SPO2), airflow (breathing), body temperature, electrocardiogram (ECG), glucometer, galvanic skin response (GSR – sweating), blood pressure (sphygmomanometer), patient position (accelerometer) and muscle/eletromyography sensor (EMG).

Project idea:

We want to exploit the usage of this e-Health Sensor Shield and integrate it in our 3D immersive interaction task to monitor physiological changes during the task. The outputs of this project should contain 2 parts:

  • Numerical data of physiological changes over time captured by some biometric sensors (e.g.  pulse, electrocardiogram)
  • Events happened in the immersive interaction task and behaviors performed by the subject.

If necessary, movement detection sensor, like accelerometer and depth camera, like Kinect will be used for human-body movement data recording.

Depending on the type of project (semester/ master diploma), there are 2 possible achievements:

  • Semester project: exploit only the e-health Sensor Shield and integrate it with our platform.
  • Master diploma: +events synchronization in an immersive interaction task and recording dataset.

Developments and Requirement:

The development of this project contains several parts:

  • Arduino language is merely a set of C/C++ function that can be called from application
  • Wireless communication: Bluetooth or WiFi data transfer will be achieved by using Arduino C-like functions
  • An API (or DLL) of Unity3D engine should be developed as a Plug-in that can be used in the future research
  • Medium level of Unity3D programming (C# and DLL)
  • Data analysis and visualization tools (R, Matlab or Matplotlib)


Nan WANG ( INJ138 )

Ronan BOULIC (   INJ141 )