Measurements by Vision Systems

The vision system here presented, developed in the framework of the VERITAS project, is tuned to tackle such problems. It consists of a set of twelve cameras each in stereo pair and a suit with a special pattern of distinctive markers to overlay over the subject (see upper Figure). Our system is able to capture simultaneously human shape and its motion.

The algorithm is composed of the following steps:

1. images segmentation and features extraction;
2. features association and 3D shape estimation (De Cecco, 2010), see 2nd figure from top;
3. 3D features tracking, next figure;
4. features segmentation (Yan, 2008), next figure;
5. relative segments kinematics extraction and joint angles estimation (Setti, 2010), next two figures.

Images segmentation is achieved by means of edge detection algorithms.Clustering the markers according to their chromatic properties performs features extraction.

For each stereo pairs features are associated and, buy triangulation, the 3D position for each corresponding marker is estimated. The point’s cloud is an estimation of the body shape.

Information about joint angles is achieved by clustering the markers in different subsets according to trajectory similarities. To achieve this goal first the features tracking is performed, than the segmentation according to each point trajectory. Once the different limbs are segmented their relative kinematics is estimated.
The method employed is called in computer vision “structure from motion”. SfM refers to the process of finding the three-dimensional structure of the scene by acquiring and estimating its motion over time. One advantage of the approach is that it works without the need to guess an a-priori model. Practically this means that, also if some limbs are not visible, only the parts that move are extracted, together with their relative motions.

With respect to commercial systems already available on the market our system has some advantages:

-it is not needed to place a certain number of markers at fiducial points with a cumbersome procedure that causes an unavoidable interference with natural motion of the subject. It is enough to wear a suit.

-the density of the reconstruction is much higher (some thousand of points wrt less than one hundred with VICON)

– the garment can be integrated with wearable sensors like temperature, breath and electrodes to monitor the fisiological parameters during exercise

– from the shape it is possibleto model the non-rigid part of the body motion

– from the shape it is possible a better estimate of the body inertial parameters and therefore a better estimate of the internal and external joints forces and torques

To see the corresponding video:

http://www.youtube.com/watch?v=AjFSto-zyPk

We are developing an instrument based upon several stereo-cameras synchronized together by means of a properly designed calibration system. A proper method for uncertainty associated to each of the 3D point of the cloud was developed in order to evaluate the compatibility between the corresponding points and therefore discard double points while enhancing the final accuracy of the reconstructed mesh.

In the foto a proper 3D shape is inserted in order to synchronise the whole setup by means of BA.

The use of three cameras makes the system more robust to outliers while enlarging its field of view.

The steps of the algorithm we are developing are:
– image acquisition
– features extraction invariant to scale and rotation
– encoding of a vector of properties for each feature
– search for correspondence between images
– validation of correspondence by means of epipolar geometry
– triangulation by means of the three cameras and building of the points cloud

In the upper picture the camera setup, in the middle one the images of a glove inflated acquired by the three cameras, in the last an example of reconstruction. The surface is reconstructed by means of a certain number of selected features for each image. Those features represent each pixel whose local information is higher than a certain threshold level.

A self-calibration method, based on planar homography and a reference solid, was developed in order to evaluate and compensate for the calibration parameters change due to temperature cycles.

This research is going to be carried out with the contribute of Enrico Marcuzzi in collaboration with Spacelight http://www.spacelight.it/spacelight/

The leather has a matrix of markers whose position is estimated before and after the application of a 2D deformation by means of a special traction machine.

The accuracy achieved in this application was of about ±0.02 mm.

for more details see the website of my old position: http://cisas.unipd.it/

News on this topic to come in the next future …