三维人脸建模与识别

1 .3D Face Modelling and Recognition Survey Timur Aksoy Biometrics Course Fall 2011 Sabanci University

2 .Outline Background Approaches Average Face Model Iterative Closest Point Algorithm Regional Models Hybrid 2D/3D Methods Conclusions

3 .Background Gallery Images are of persons enrolled in the system Probes are the images to identify or authenticate . One to one match for veriification , one to many for identification . 3D model can be rendered either as depth image , shaded model or mesh of points . 2D image can be thought of texture map of 3D image . Range images are 2.5D as they only give depth values of x-y coordinates from one view point . 3D images scan the head completely and usually represented by polygonal mesh or cloud of points

4 .3D vs 2D 2D detection performance degrades under pose changes , nonrigid motions , changes in illumination occlusion aging . 3D detection is considered to be effected less by these factors .

5 .Approaches Point Clouds Depth based from Range Images Geometric Methods Differential Global Local Shape Descriptors

6 .Point Clouds (ICP) Point Set: Points with x,y,z coordinates represent the surface . Probe face is registered to gallery faces using Iterative Closest Point Algorithm and dense match between features and points is obtained . Most systems use a reference or template face for alignment with ICP (e.g. Average Face ) so that alignment is performed only once . ICP only accounts for rigid motion . It ultimately finds point to point correspondence .

7 .Point Clouds

8 .Point Clouds ( Nonrigid ) Landmarks are used for pre - alignment ex : Procrustes Analysis . After rigid alignment , face may be deformed by splines to fit the reference face . B- Splines and Thin - Plate Splines are common . Nonrigid registration is needed for robustness against expressions . Deformation can be performed on both local and global scales . Subregions of face may be matched independently .

9 .Point Clouds ( Similarity ) After fitting , points may be sampled from both surfaces according to average face model Point set differences may be used for dissimilarity measure after registration . Alternatively , Hausdorff distances may be used for distance function between 2 point sets . Hausdorff distance measures the maximum of the smallest distances between the points in both sets . PCA can also be applied on Point Cloud and new axes with maximum variance are determined .

10 .Depth Image Based Depth image intensities indicate the depth at that coordinate . 2D image matching techniques can be used such as PCA, LDA and ICA. Poses are first normalized using features or ICP.

11 .Range Images

12 .Geometric Methods ( Profiles ) Curves are extracted from surface and used in 2D analysis Central and Lateral profiles may be used for recognition . Iterative Conditional Mode (ICM) optimization is used for matching profiles . Curvatures along profiles used as features . Vertical Symmetry Profile curve seen as most discriminative .

13 .Seven vertical profiles

14 .Geometric Methods ( Curvature ) Curvature of surfaces are computed . Extract landmarks using curvature (i.e. Gaussian curvature ) and segment the face Gaussian curvature is the product of two principle curvatures at one point . Maximum and minimum principle curvatures may be represented by two Enhanced Gaussian Images . Extended Gaussian Image (EGI), constructed by mapping principal curvatures and their directions at each surface points, onto two unit spheres . Their values are mapped to lengths of vectors like histogram . Similarity match is performed by Fisher’s spherical approximation on EGI’s

15 .Principle Curvatures of Saddle

16 .Extended Gaussian Image

17 .Geometric Methods ( others ) Normals on the surface may also be computed as 3 component vectors . Difference between normals of 2 images may be computed by angle difference histograms . Landmark distances and angles can also be computed as features SVM based classifiers can classify faces according to the geometric features . Expression resistant regions such as nose and eyes have high recognition rates and their scores may be fused . Forehead , jaw line , eye corner cavities and cheeks of same person remains same in different range images . Local curvatures are also resistant to non - rigid motion since expression changes the face globally .

18 .Surface Normals

19 .Shape Descriptor - based Local and global facial features are extracted Point Signatures are popular and used in coarse registration and region detection . Point Signatures may be computed on rigid regions and compared for expression resistance . A mesh is fitted to the point cloud and shape descriptors are extracted and their moments are computed . Sphere Spin Images describe the surface locally .

20 .Point Signatures Draw a sphere around point p of surface . Intersect sphere with surface and get a space curve C which may not be planar . Fit a plane P to this curve approximately . Call its normal n1 and translate the plane along n1 to include point p. Call the plane P2 and project C to P2 to form a new curve C2. Distance of points on C2 form signed distance profile Direction from p to point on C2 with largest positive distance is unit vector n2. Theta is the cw angle of point on C2 about n1 from n2. We discretize theta and record signed distances which produces the point signature .

21 .Point Signature

22 .Sphere Spin Images (SSI) An SSI, associated with a point on the surface, is a 2D histogram constructed from distances of other points in its neighborhood surface falling in the sphere centered at that point It captures the characteristic of local shape. Usually small set of points are selected for SSI analysis by minimum principle curvature . Similarity between series of SSI for each subject is measured by a simple correlation coefficient .

23 .Feature Extraction Most approaches use coarse to fine localization . Simplest methods locate valleys such as eye sockets and the peak as the nose however they assume normalized faces . Appereance based models project image into subspace with PCA, ICA, DCT, Gaussian Derivative Filters or Gabor Wavelets Geometric-based methods use angles, distances and areas between landmarks I n the structure-based methods, the ensemble of candidate landmarks is fitted to a model of feature locations and the likelihood is considered.

24 .The Work of Bosphorus Group Bosphorus University Computer Engineering

25 .Average Face Model Instead of aligning each face separately create Average Face Model from gallery . Instead of registering probe to every gallery image , register it to the AFM. Multiple AFM’s are also possible . Few landmarks are extracted . Then probe is first coarsely registered , rigidly registered (ICP) then non - rigidly (TPS). The probe image is resampled after registering .

26 .AFM and 7 Landmarks

27 .ICP Dense correspondence is established and the points without correspondence or with high distance are removed . The objective function to be minimized is: where is quaternion rotation matrix , is translation vector , is measured data set and is the model data set. Points with the same indices correspond to each other and point sets have same size.

28 .

29 .Estimation of Rotation and Translation The cross variance of sets P and X is given by