Movies

Computer Animation

Intuitive software for 3D character posing from 2D strokes (WALT CTI project with Moka Studio 2015-17)

The CTI WALT project aimed to provide an intuitive tool suitable for 2D artists using touch-enabled pen tablets. An artist-oriented tool should be easy-to-use, real-time, versatile, and locally refinable .

M. Mahmudi, P. Harish, B. Le Callennec, and R. Boulic “Artist-Oriented 3D Character Posing from 2D Strokes”, Computer & Graphics, Vol 57, p. 81-91, June 2016, Elsevier, in Open Access, DOI:10.1016/j.cag.2016.03.008

Scenes and characters are designed by Mireille Clavien

The second core contribution from the CTI WALT project is a fast IK solver taking advantage of GPU and/or multi-core architectures, achieving 10 to 150 times speedup over the state of the art.

Open Access to the ACM-TOG paper through the authorizer link located at the bottom of this page.

YouTube links: ACM TOG demo, SIGGRAPH16 teaser

Scenes and characters are designed by Mireille Clavien

Full-Body Interaction

Performance Animation Retargeting

  Demo of our Open Access IEEE TVCG paper to appear in 2017, yet visible online: http://ieeexplore.ieee.org/document/7942001/

Our approach allows to transfer the performer movement on a target character while preserving the relative location of body parts. This allows to prevent interpenetrations and to preserve the meaning of the performer action.

This is achieved in real-time while the performer is moving to allow producing 3D animation content more efficiently.

Subjective experience of embodied interaction

The topic of our submission deals with the subjective experience of Embodied Interaction. This goes much further than ensuring a good immersive experience in a virtual environment to the user. In fact we want also that the user, who is equipped with a mocap sensors, feels as if he were a TARGET SUBJECT with a very different body height. We retained the example of a child as an extreme case that can make sense for the design of a large range of consumer’s equipments and dedicated facilities (home, schools, hospital, car, etc…). We advocate that a better subjective immersion is achieved by scaling the whole space and virtual environment by the ratio : user height / target subject height. This makes sense because it has been demonstrated in neurosciences that an individual evaluates distances egocentrically, in body size units. In our video it results in scaling up the whole space when the user adopts this approach.

R. Boulic, D Maupu, D Thalmann, “On Scaling Strategies for the Full Body Interaction with Virtual Mannequins”, Journal Interacting with Computers, Special Issue on Enactive Interfaces, Elsevier, 21(1-2), January 2009. 11-25.
Motion capture: Eray Molla
Video edit: Nan Wang

Sense of Embodiment: Self-location, Body Ownership, Agency

Characterizing First and Third Person Viewpoints and their Alternation for Embodied Interaction in Virtual Reality

experimental setup  

Video of the experimental protocol when studying the alternation of first and third person viewpoint, associated to the following paper:

H. Debarba, S. Bovet, R. Solomon, O. Blanke, B. Herbelin & R. Boulic “Characterizing First and Third Person Viewpoints and their Alternation for Embodied Interaction in Virtual Reality” ,PLOSONE December 2017, in Open Access

influence of the point of view: first persion (1PP) vs third person (3PP)

This is a small sample illustrating the motion capture used to evaluate performance and the relation the subject creates with the avatar in first and third person perspective:

Debarba, H. G., Molla, E., Herbelin, B., & Boulic, R. (2015, March). Characterizing embodied interaction in First and Third Person Perspective viewpoints. In 3D User Interfaces (3DUI), 2015 IEEE Symposium on (pp. 67-72). IEEE.

influence of the projection: planar vs non-planar

In this paper we evaluate the use of non-planar projections as a means to increase the Field of View (FoV) in embodied Virtual Reality (VR). Our main goal is to bring the virtual body into the user’s FoV and to understand how this affects the virtual body/environment relation and quality of interaction.

Debarba, H. G., Perrin, S., Herbelin, B., & Boulic, R. (2015, November). Embodied interaction using non-planar projections in immersive virtual reality. In Proceedings of the 21st ACM Symposium on Virtual Reality Software and Technology (pp. 125-128). ACM.

Virtual Reality

Within-crowd immersive evaluation of collision avoidance behaviors – ACM VRCAI 2012

 We present our crowd simulation method, Trajectory Variant Shift (TVS), based on real pedestrian trajectories re-use. We detail how to re-use and shift these trajectories to avoid collisions while retaining the liveliness of captured data.
  Second, we conducted a user study in a four-screen CAVE to compare our approach with three others when the subject is standing within the crowd to perform a visual search task (waiting for a specific person).
  Results confirm that our approach is considered as good as the state of the art regarding subject’s spatial awareness within the crowd, and better regarding not only the perceived liveliness of the crowd, but also the comfort in the CAVE.

J. Ahn, N. Wang, D. Thalmann, R. Boulic, Within-Crowd Immersive Evaluation  of Collision Avoidance Behaviors, to appear in Proc. of ACM SIGGRAPH VRCAI 2012, Singapore, December 2012

3D Virtual calligraphy

 ICDVRAT 2006

We have explored a new type of light painting interface in the framework of the European Union Network of Excellence “Enactive Interfaces”. Full body movements are exploited to created 3D entities with either ribbon, tube or spray. The entities are visualized on a transparent canvas.

Damien Maupu, José Rosales, Ronan Boulic and the painter Muma

Alter body

 We have collaborated with the University of Geneva (anthropometric body meshes) and the CEIT in Spain (real-time motion capture) to assess the importance of scaling the virtual environment when controlling an avatar that may have a different body height. The study shows that the avatar control is much more effective when working with a third person viewpoint in front of a large immersive screen rather than with a head-Mounted Display. This work has been achieved in the framework of the European Union Network of Excellence “Enactive Interfaces”.

EPFL VRLAB: Damien Maupu, Ronan Boulic
University of Geneva: Mustafa Kasap
CEIT: Luis Unzueta

Motion

Long term real trajectory reuse – 2011

 Our goal is to improve  the realism of real-time simulated crowds by reducing short term collision avoidance through long term anticipation of pedestrian trajectories. For this aim, we choose to reuse outdoor pedestrian trajectories obtained with non-invasive means.

The concept of region goal is exploited to enforce the principle of “sufficient satisfaction”: it allows the pedestrians to relax the prescribed trajectory to the traversal of successive region goals.

J. Ahn, S. Gobron, Q. Silvestre, B. Shitrit and H. Beny et al. Long Term Real Trajectory Reuse Through Region Goal Satisfaction . The Fourth International Conference on Motion in Games, Edinburgh, UK, 2011.

Encapsulating motion continuity for constraint-based motion editing

 We introduce a novel method for interactive human motion editing. Our main contribution is the development of a Low-dimensional Prioritized Inverse Kinematics (LPIK) technique that handles user constraints within a low-dimensional motion space – also known as the latent space.

Our technique is based on the mathematical connections between linear motion models such as Principal Component Analysis (PCA) and Prioritized Inverse Kinematics (PIK). Furthermore, two strategies to impose motion continuity based on PCA are introduced.
S. Carvalho , R.Boulic, C. Vidal, D. Thalmann. “Latent motion spaces for full-body motion editing”, The Visual Computer, to appear in 2013.

Motion constraint

 We propose a hybrid postural control approach taking advantage of data-driven and goal-oriented methods while overcoming their limitations.s. We take advantage of the latent space characterizing a given motion database. We introduce a motion constraint operating in the latent space to benefit from its much smaller dimension compared to the joint space. This allows its transparent integration into a Prioritized Inverse Kinematics (PIK) framework. The motion constraint benefits from the natural flow of movement provided by the motion database to channel the convergence of the PIK while retaining the spatio-temporal coherence of the captured motions.

D. Raunhardt and R. Boulic . Motion constraint , in Visual Computer, vol. 25, p. 509-518, 2009.

D. Raunhardt and R. Boulic . Immersive singularity-free full-body interactions with reduced marker set, in Computer Animation And Virtual Worlds, vol. 22, p. 407-419, 2011.

 

Interactive low-dimensional motion synthesis by combining motion models and PIK

 We introduce a constraint-based motion editing technique enforcing the intrinsic motion flow of a given motion pattern (e.g., golf swing). 
Its major characteristic is to operate in the motion Principal Coefficients (PCs) space instead of the pose PCs space. By construction, it is sufficient to constrain a single frame with Inverse Kinematics (e.g., the hitting position of the golf club head) to obtain a motion solution preserving the motion pattern style.

S. Carvalho, R. Boulic and D. Thalmann . Interactive Low-Dimensional Human Motion Synthesis by Combining Motion Models and PIK , in Computer Animation & Virtual Worlds, vol. 18, 2007.

Progressive clamping

Interactive Control of Kinematic Chain

We propose the progressive clamping method to better handle the kinematic anistropy of joint limits for virtual mannequins or robots. Our method damps only the joints’ variation component heading towards the limits. In addition we propose to dynamically express the corrective joint variation as a highest priority constraint that naturally extends the management of inequality constraints.

D. Raunhardt, R. Boulic, “Progressive Clamping”, IEEE  ICRA07, Roma

Robust kinematic constraint detection for motion data

  We developped a method for detecting kinematic constraints for motion data, which is an important step to ease further operations such as blending or motion editing..
It detects when an object (or an end-effector) is stationary in space or is rotating around an axis or a point.
Our method is fast, generic and may be used on any kind of objects in the scene. Furthermore, it is robust to highly noisy data as we detect and reject aberrant data by using a least median of squares (LMedS) method.

B. Le Callennec, R. Boulic  “Robust Kinematic Constraint Detection for Motion Data” , Proc. of  EG-SIGGRAPH SCA06, Vienna Sept. 2006

 

Locomotion engine

 We propose an on-line reactive animation method generalizing a biped locomotion pattern combining standing, walking, and running. The resulting engine is able to animate human-like characters of any size and proportions.

For that purpose, several motion capture data from several persons have been organized into a hierarchical PCA (Principal Component Analysis) structure to perform not only interpolation, but also extrapolation.

P. Glardon, R. Boulic and D. Thalmann . Dynamic obstacle avoidance for real-time character animation, in Visual Computer, vol. 22, num. 6, p. 399-414, Vis. Comput. (Germany), 2006

P. Glardon, R. Boulic and D. Thalmann . Robust on-line adaptive footplant detection and enforcement for locomotion, in Visual Computer, vol. 22, num. 3, p. 194-209, Vis. Comput. (Germany), 2006.

Motion deformation with prioritized constraints

 We present an interactive motion deformation method to modify animations so that they satisfy a set of prioritized constraints. Our approach successfully handles the problem of retargetting, adjusting a motion, as well as adding significant changes to preexisting animations.

We introduce the concept of prioritized constraints to avoid tweaking issues for competing constraints. Each frame is individually and smoothly adjusted to enforce a set of prioritized constraints. The iterative construction of the solution channels the convergence through intermediate solutions, enforcing the highest prioritized constraints first.
In addition, we propose a new, simple formulation to control the position of the center of mass so that the resulting motions are physically plausible.

B. Le Callennec and R. Boulic . Interactive motion deformation with prioritized constraints, in Graphical Models, vol. 68, num. 2, p. 175-93, Graph. Models (USA), 2006

P. Baerlocher and R. Boulic . An inverse kinematics architecture enforcing an arbitrary number of strict priority levels, in Visual Computer, vol. 20, num. 6, p. 402-17, Vis. Comput. (Germany), 2004.

Emotion

Conveying ambivalent feelings through asymmetric facial expressions – 2012

  Achieving effective facial emotional expressivity within a realtime rendering constraint requests to leverage on all possible inspiration sources and especially from the observations of real individuals. One of them is the frequent asymmetry of facial expressions of emotions, which allows to express complex emotional feelings such as suspicion, smirk, and hidden emotion due to social conventions. To achieve such a higher degree of facial expression, we propose a new model for mapping emotions onto a small set of 1D Facial Part Actions (FPA)s that act on antagonist muscle groups or on individual head orientation degree of freedoms. The proposed linear model can automatically drive a large number of autonomous virtual humans or support the interactive design of complex facial expressions over time.

Junghyun Ahn, Stephane Gobron, Daniel Thalmann, and Ronan Boulic

An NVC emotional model for conversational VHs – 2012

movie

 This research proposes a new emotional model for Virtual Humans (VHs) in a conversational environment. As a part of a multi-users emotional 3D-chatting system, the research focus on how to formulate and visualize the flow of emotional state defined by the Valence-Arousal-Dominance (VAD) parameters.
From this flow of emotion over time, we successfully visualized the change of VHs’ emotional state through the proposed emoFaces and emoMotions. The notion of Non-Verbal Communication (NVC) was exploited for driving plausible emotional expressions during conversation. With the help of a proposed interface, where a user can parameterize emotional flow, we succeeded to vary the emotion expressions and reactions of VHs in a 3D conversation scene.

Junghyun Ahn, Stephane Gobron, David Garcia, Quentin Silvestre, Daniel Thalmann, and Ronan Boulic

An event-based 3D NVC chatting architecture – 2012

 Non-verbal communication (NVC) such as gesture, posture, and facial expression makes up about two-thirds of all communication. However, this fundamental aspect of communicating is often omitted in 3D social forums or virtual world oriented games.
This research proposes an answer to this issue by presenting a multi-user 3D-chatting system enriched with NVC relative to motion. Basically, this event-based architecture tries to recreate a context by extracting emotional cues from dialogs and derives virtual human potential body expressions from that event triggered context model.
We structured by expounding the system architecture enabling the modeling NVC in a multi-user 3D-chatting environment. There, we present the transition from dialog-based emotional cues to body language, and the management of NVC events in the context of a virtual reality client-server system.

Stephane Gobron, Junghyun Ahn, David Garcia, Quentin Silvestre, Daniel Thalmann, and Ronan Boulic

Oldies

A generic model for geometric constraints detection – 2004

 Constraints are widely used by motion editing techniques as a means to apply changes to existing motions. During this process, we may want to preserve some important features of the original motion. we proposed a generic method to achieve automatic detection of geometric constraints on existing animations. At each frame, elementary geometric constraints are computed with respect to a frame of reference. We then use constraint-related concepts of union and intersection to merge the elementary constraints together and to generate new ones. Finally, our algorithm provides an exhaustive list of geometric constraints on the motion itself as well as with moving objects of the scene.

B. Le Callennec, R. Boulic  “Robust Kinematic Constraint Detection for Motion Data” , Proc. of  EG-SIGGRAPH SCA06, Vienna Sept. 2006

Motion editing with prioritized constraints – 2003

 Our goal is to allow the end-user to transform an original motion by applying position constraints on freely selected locations of the character body. More importantly, it is possible to assign a priority level to each constraint. As a consequence the end-user can enforce a larger range of natural behaviors where conflicting constraints compete to control a common set of joints.

B. Le Callennec, R. Boulic  Interactive Motion Deformation with Prioritized Constraints , Selected among EG-SIGGRAPH SCA04 best papers, extended and revised. Graphical Models, 68(2), 175-193, March 2006

Gait style editor – 2003

 The Gait style editor provides an H-ANIM compliant interface for the
interactive design of real-time gait styles. The user can select both a desired linear speed and a desired angular speed and tune the current gait style for numerous postural parameters. An integrated step frequency adjustement always ensures the correct realization of the desired linear speed.

The editor is built on top of the VHD++ platform which offers a flexible
initialization of the environment and numerous navigation metaphors.

based on:

R. Boulic, Thalmann D, Magnenat-Thalmann N, A global human walking model with real time kinematic personification, The Visual Computer, 6 (6), December 1990

Versatile walk engine – 2002

 We propose an H-Anim compliant walk engine producing a realistic walking pattern with continuously evolving velocity. Apart from an intrinsic real-time requirement the described walking model addresses three issues: generalization (animation of a wide population of virtual humans for a wide range of the walking parameters), openness (user-defined personification of the gait style) and reactivity (changing the user defined context at any time while maintaining the coherence of the model).

R. Boulic, B. Ulicny, D. Thalmann «Versatile Walk  Engine», Journal of Game Development, 1(1), pp 29-52, Michael van Lent Editor, Charles River Media. www.jogd.com, 2004

R. Boulic, Thalmann D, Magnenat-Thalmann N, A global human walking model with real time kinematic personification, The Visual Computer, 6 (6), December 1990

Motion capture – 1998

 Demonstration of a Motion Capture technique that relies on the position/orientation measurement provided by magnetic sensors strapped on body segments. The real-time posture reconstruction algorithm exploits the position measurement of one sensor (at the spine base) and only the orientation measurement for the others. This guarantees the high robustness of the method.

T. Molet, Boulic R., Thalmann, D., “Human Motion Capture Driven by Orientation Measurements”, Presence, 8(2), pp 187-203,  MIT Press, April 1999

T. Molet, Boulic R., Rezzonico S., Thalmann, D.,”An architecture for immersive evaluation of complex human tasks”, IEEE Transaction in Robotics and Automation, Special Section on Virtual Reality, 15 (3), pp 475-485, June 1999

Virtual fighting – 1997

 A hierarchical model of human actions is used to capture the human body posture in real-time, via sensors attached to the user. The demo shows a life participant with ten sensors used to animate the avatar in the virtual scene. The participant performs fight gestures which are recognized by the virtual opponent. The latter responds by playing back a pre-recorded keyframe sequence.

L. Emering, Boulic R., Thalmann D., “Conferring human action recognition skills to life-like agents “, Journal of Applied Artificial Intelligence, Special Issue on Animated Interface Agents, Volume 13 (4-5), pp 539-565, June-August 1999

L. Emering, Boulic R., Thalmann, D. “Interacting with Virtual Humans Through Body Actions”, IEEE Journal of Computer Graphics and Application, “Projects in VR” pp 8 – 11, January 1998