Faculty: A. Goshtasby
Student: Lyubomir Zagorchev
Sponsor: Ohio Board of Regents
Step 1. Determining
the geometry of a scene from structured light.
Two video frames showing a stereo pair obtained
by a stereo camera setup while sweeping the laser over the object.
Detection of the laser stripe in the two
images. The horizontal displacement between laser points on the same
scanline in the images determines the 3-D coordinates of the laser
point.
Initial 3-D model of the object in different
views. Since there are missing image points in the model, we first
approximate the missing points and obtain a new 3-D model as shown
below. This process will also reduce noise
in data.
There are still some holes in the data because
mapping points in an image to a 3-D model produces some gaps. We fit a
NURBS surface to 3-D data estimated from a stereo image pair to
represent a continuous model
3-D model constructed with a NURBS surface. This further reduces noise and produces a smooth model.
The NURBS surface with mapped texture.
The following images show the same process but using a human model.
Initial 3-D model.
Approximated 3-D model by initial filling in process.
NURBS surface representation of the model
The NURBS surface with mapped texture.
Step 2: Rigid
tracking from stereo correspondence.
Having a complete 3D model of the face, obtained
as a result of Step 1, we then proceed with tracking of the rigid
motion of the head in the stereo video sequence. For every pair of
frames, we track a number of MPEG-4 facial features . The tracking
process is initiated by selecting the features interactively. Template
matching in a small neighborhood is conducted in the tracking.
From the tracked points, we pick four among them that move the least with respect to other points because they represent the rigid (global) movement of the head. Having the 3D coordinates of these four points in consecutive frames, we determine the new rotation and the translation of the head by the least squares approach. The rotation and translation are then applied to the 3D model and the process is repeated for all pairs of frames in a video sequence.
At this point we have determined the rotation and translation of the head for all frames, and by applying them to the previously created model we track the head in 3D.
Between some frames, due to inaccurate template
matches, we may not obtain a smooth transition in 3D. Therefore, we
introduce a smoothing step. For every
feature point we build a 3D Rational Gaussian curve (RaG) that
represents the trajectory of the point in 3-D alog x, y, and z axes.
Smoothing the initial RaG curves, we obtain the
new 3D trajectories for all feature points. By running the initial algorithm for determining
rotation and translation over the smoothed trajectories, we obtain the
smooth rigid motion of the head.
Left Camera Stereo Video Sequence |
Initial Motion |
RaG Smoothed Motion (sigma=10) |
Step3: Non-rigid tracking from stereo correspondence (in progress).
[Intelligent Systems Laboratory] [WSU Home Page] [CSE Department Home Page]
For more information contact A. Goshtasby (agoshtas@cs.wright.edu).
Last modified: 7/28/03