Conveying body pose errors and collisions to the sense of vision through the use of additional viewports in VR

Context

In this project we are interested on the relation between the user body and the virtual environment. More specifically in two related topics:

  • Embodied learning on virtual architectural prototypes and dangerous environments people may need to deal in real life;
  • Posture learning by mimicking pre-recorded whole body poses.

Problem with Haptic feedback

The lack of materiality in virtual environments is a major limitation of virtual reality techniques. Realistic approaches that account for the user sense of touch relies on the usage of active mechanical devices or passive real objects (active and passive haptics respectively), and are unsuitable for malleable full body interactions. The active approach is generally expensive, limits interaction space, and can deliver realistic haptic rendering to a limited region of the body (e.g. hands). Passive haptic is cheaper but not malleable, requiring the match between virtual and real environment shapes.

Project

This semester project proposes conveying collision and body pose mismatch information to the sense of vision instead of touch. Additional cameras are going to be used to highlight these undesired/incorrect interactions. Whenever these happen, a viewport will be generated and shown to the user so she can be aware of the problem and account on this information to successfully correct her pose. Two sample cases are going to be explored:

  • Limb – obstacle interpenetration and risk of interpenetration;
  • Unsuccessful reproduction of a pose instructed by a virtual avatar teacher.

Questions to explore are:

  • How many virtual cameras and virtual screens are necessary and/or convenient;
  • Where to position these virtual cameras and screens;
  • Behavior of the cameras and screens.

Expected Outcome

In order to highlight collisions and position mismatches the student will develop and evaluate automatic camera behaviors. The ideal view should clearly demonstrate where the error happened, and allows for easy understanding of how the user may correct her posture to respect the virtual environment constraints. Additionally, an evaluation may be conducted to evaluate the proposed solution.

 

  Distinction between master thesis/semester project workload

  • A semester project student will receive a well-defined application for which he is expected to explore and propose optimal configurations of cameras/screens position and behavior.
  • A master thesis student is expected to approach the problem in a more general way,  exploring and proposing optimal configurations of cameras/screens position and behavior that will accommodate a wider range of application.

 

Expected Background

3D Character Animation
Programming in C++, familiarity with C# or Javascript (Unity3D)
Previous knowledge of Unity3D and VR displays is a plus.


Contact

  • Henrique Galvan Debarba (email: @epfl.ch, office INJ 139)