The interest on 3D reconstruction and motion analysis devices is growing due to the wide application of these systems in different industrial and scientific fields. The recent advancements in Computer Vision have impacted highly in the movie and advertisement industries (Boujou, 2009), in the medical analysis area, in video-surveillance applications and in biomechanical studies of the human body (Corazza et al., 2007; Fayad et al., 2009). However, the strongest limitation for several systems is their restriction to deal with rigid bodies only. A shape which is deforming introduces strong challenges, the object can vary arbitrary and the observed shape may have different articulations not known a priori. How to model and identify such variations is still an open issue.
The vision system here presented, developed in the framework of the VERITAS project, is tuned to tackle such problems. It consists of a set of twelve cameras each in stereo pair and a suit with a special pattern of distinctive markers to overlay over the subject (see upper Figure). Our system is able to capture simultaneously human shape and its motion.
Algorithm
The algorithm is composed of the following steps:
- images segmentation and features extraction;
- features association and 3D shape estimation (De Cecco, 2010), see 2nd figure from top;
- 3D features tracking, next figure;
- features segmentation (Yan, 2008), next figure;
- relative segments kinematics extraction and joint angles estimation (Setti, 2010), next two figures.
Images segmentation and features extraction;features association and 3D shape estimation (De Cecco, 2010), see 2nd figure from top;3D features tracking, next figure;features segmentation (Yan, 2008), next figure;relative segments kinematics extraction and joint angles estimation (Setti, 2010), next two figures.
Images segmentation is achieved by means of edge detection algorithms.Clustering the markers according to their chromatic properties performs features extraction.
For each stereo pairs features are associated and, buy triangulation, the 3D position for each corresponding marker is estimated. The point’s cloud is an estimation of the body shape.
Information about joint angles is achieved by clustering the markers in different subsets according to trajectory similarities.
To achieve the goal first the features tracking is performed, than the segmentation according to each point trajectory.
Once the different limbs are segmented their relative kinematics is estimated.
Information about joint angles is achieved by clustering the markers in different subsets according to trajectory similarities. To achieve this goal first the features tracking is performed, than the segmentation according to each point trajectory. Once the different limbs are segmented their relative kinematics is estimated. The method employed is called in computer vision “structure from motion”. SfM refers to the process of finding the three-dimensional structure of the scene by acquiring and estimating its motion over time. One advantage of the approach is that it works without the need to guess an a-priori model. Practically this means that, also if some limbs are not visible, only the parts that move are extracted, together with their relative motions.
With respect to commercial systems already available on the market our system has several advantages:
- reconstruction of the human body form in high density and accuracy – many systems on the market only detect motion parameters
- possibility to measure the chest breathing volume, even during subject motion
- possibility to measure in natural conditions in scenarios typical of a home environment – the suit has more than 3000 markers that allow a robust detection of the motion of each body in the field of view even if partially covered or in the presence of other bodies
- from the shape it is possible a better estimate of the body inertial parameters and therefore a better estimate of the internal and external forces and torques through the joints
- ability to measure even rapid movements using the adherence of the suit. Traditional systems that employ markers or inertial instruments bound by bracelets to the body segments suffer the high accelerations
- it is not needed to place a certain number of markers at fiducial points with a cumbersome procedure (for which is needed a technician) that causes an unavoidable interference with natural motion of the subject. It is enough to wear a suit
- the density of the reconstruction is much higher than VICON (some thousand of points wrt less than one hundred with VICON)
- the lighting system causes less fatigue than the IR one used for other systems (eg VICON or other)
- compared to Kinect system is more accurate (for every kind of gestures but particularly for motions in the direction of the kinec)
- the measure is not affected by drift as in the case of inertial systems
- with proper cameras the system allows outdoor motion measurement with high accuracy and no drift
- the garment can be integrated with wearable sensors to monitor temperature, breath, ECG, electrodes and other fisiological parameters
- from the shape it is possible to model the non-rigid part of the body in motion