The main goal is the tracking of human body limbs for markerless motion capture. When agents come closer to camera sensors and the active camera zooms in on these agents, their body posture can be evaluated to check for compatibility with behaviour hypotheses generated.

Detection of human body motion

Feature-based human detection becomes extremely challenging when the agent is being observed from different viewpoints. Besides, similar actions, such as walking and jogging, are hardly distinguishable by considering the human body as a whole.

Get Flash to see this player.

Download:

We have developed a system which detects human body parts under different views and recognize similar actions by learning temporal changes of detected body part components. Firstly, human body part detection is achieved to find separately three components of the human body, namely the head, legs and arms. We incorporate a number of sub-classifiers, each for a specific range of view-point, to detect those body parts. Subsequently, we have extended this approach to distinguish and recognise actions like walking and jogging based on component-wise HMM learning.

Get Flash to see this player.

Download:

Tracking of human body motion

Full body 3D tracking from a monocular image sequence has become an intense studied area over the past years. The main challenges involved arise from 3 main issues. First, we must deal with 2D-3D projection ambiguities between the real world and the projected images. Then, most of the times the 2D position of the body joints are not observable in the images due to self-occlusions and occlusions with other objects. Finally, the shape and appearance of the human body may change drastically over time due to illumination changes, rotations in-depth of limbs, and loosely fitting clothing.

To overcome these issues, one of the most used techniques for full-body human tracking consists of estimating the probability of the parameters of a human body model over time by means of a particle filter. However, given the high-dimensionality of the models to be tracked, the number of required particles to properly populate the space of solutions makes the problem computationally expensive.

Get Flash to see this player.

Download:

We have designed an action-specific dynamic model of human motion to avoid particle wastage within the prediction step of the Particle Filter. Hence, particles are propagated taking into account their motion history, and previously learnt motion directions from real training data. Next, the state space is constraint by filtering out those body configurations which are not likely according to our motion model. As a result, as long as the truly performed motion lies within the bounds of our motion model, robustness is added to the whole tracker against non-reliable measurements from the image sequence, i.e. in case of occlusions and/or background clutter.

Get Flash to see this player.

Download:

In fact, experimental results show that the tracker allows the reconstruction of the 3D motion parameters of a full body stick figure model using only the 2D positions of a very reduced set of observable joints, namely the head, one hand, and one foot.

Get Flash to see this player.

Download:

Get Flash to see this player.

Download:

Currently, we are exploring solutions that learn a mapping between body silhouettes obtained by background subtraction techniques and the viewpoint at a given height of the camera w.r.t. the subject. Finally, the overall tracking approach will be trained for other kinds of actions, and add multiple action support by selecting appropriate training sets and dealing with transitions between actions.