07_Face_detection

1 .Ch. 7 : Face detection KH Wong Face detection v8a 1

2 .Introduction Face interface Face detection Face recognition Face detection v8a 2 Face detection Face recognition Mr.Chan Prof..Cheng Face database Output:

3 .Face detection [1] To detect faces in an image (Not recognize it yet) Challenges A picture has 0,1 or many faces. Faces are not the same: with spectacles, mustache etc. Sizes of faces vary a lot. Available in most digital cameras nowadays The simple method Slide a window across the window and detect faces. Too slow, pictures have too many pixels. (1280x1024=1.3M pixels) Face detection v8a 3

4 .Evaluation of face detection Detection rate Total number of faces that are correctly detected/total number of faces actually exist in the picture Should be high > 95%. False positive rate The detector output is positive but it is false (there is actually no face). Definition of False positive: A result that is erroneously positive when a situation is normal. An example of a false positive: a particular test designed to detect cancer of the toenail is positive but the person does not have toenail cancer. (http://www.medterms.com/script/main/art.asp?articlekey=3377) Should be low <10 -6 A good system has High detection rate, Low false positive rate. Face detection v8a 4 False positive result

5 .Example What are the detection rate and false positive rate here? Answer detection rate=(6/9)*100% false positive rate=(1/7)*100% Face detection v8a 5 False positive result 6 faces correctly detected in the picture, 9 actually faces exist in the image 7 windows reported to have faces , but in 1 window it is not a face

6 .The Viola and Jones method [ 1 ] The most famous method Training may need weeks Recognition is very fast, e.g. real-time for digital cameras. Techniques Integral image for feature extraction Ada-Boost for face detection Attentional cascade for fast rejection of non-face sub-windows Face detection v8a 6

7 .Class exercise 8.1 Detected results are in red frames What are the detection rate and false positive rate here? Answer detection rate=? false positive rate=? Face detection v8a 7

8 .The Viola and Jones method Technique 1: Integral image for feature extraction Face detection v8a 8

9 .Face detection v8a 9 Image Features ref [ 3 ] A very simple feature calculation method “Rectangle filters” Rectangle_Feature_value f= ∑ (pixels values in white area) – ∑ (pixels values in shaded area)

10 .Example Find the Rectangle_Feature_value (f) of the box enclosed by the dotted line Rectangle_Feature_value f= ∑ (pixel value in white area) – ∑ (pixel value in shaded area) f=(8+7)-(0+1) =15-1= 14 1 2 3 3 3 0 1 3 5 8 7 1 0 2 3 6 Face detection v8a 10 An image with gray levels shown

11 .Class exercise 8.2 Find the Rectangle_Feature_value (f) of the box enclosed by the dotted line Rectangle_Feature_value f= ∑ (pixel value in white area) – ∑ (pixel value in shaded area) f= 2 7 5 8 2 6 2 9 5 4 8 2 8 2 5 10 Face detection v8a 11

12 .Face detection v8a 12 Example: A simple face detection method using one feature in the middle of image Result This is a face: The eye-area (shaded area)is dark, the nose-area(white area) is bright. So f is large, hence it is face This is not a face. Because f is small Rectangle_Feature_value f f= ∑(pixels in white area) – ∑ (pixels in shaded area) If (f) is large, then it is face ,i.e. if (f)>threshold, then face Else non-face

13 .Face detection v8a 13 How to find features faster? Integral images fast calculation method [Lazebnik09 ] The integral image = sum of all pixel values above and to the left of ( x , y ) Can be found very quickly by the hardware system (x,y)

14 .Examples The integral image = sum of all pixel values above and to the left of ( x , y ) Pixel P is at (x=3,y=2) integral image of P is =1+2+3+3+4+6 integral image of Q is =1+2+3+3+4+6+5+2+4+0+2+3 Face detection v8a 14 1 2 3 3 3 4 6 3 5 2 4 1 0 2 3 6 P Top-left corner(x,y)position=(1,1) Q x y

15 .Face detection v8a 15 Computing the integral image [Lazebnik09 ] Cumulative row sum: s(x, y) = s(x–1, y) + i(x, y) Integral image: ii(x, y) = ii(x, y−1) + s(x, y) MATLAB: ii = cumsum(cumsum(double(i)), 2); ii(x, y-1) s(x-1, y) i(x, y) (x=1,y=1)

16 .Face detection v8a 16 Calculate sum within a rectangle A,B,C,D are the values of the integral images at the corners of the rectangle R. The sum of image values inside R is: Area_R = A – B – C + D If A,B,C,D are found , only 3 additions are needed to find Area_R Calculations of areas can reused for other windows. D C A B For the Rectangle R the Area is Area_R

17 .Why do we need to find pixel sum of rectangles? Answer: We want to get face features You may consider these features as face features Left Eye: (Area_A-Area_B) Nose : (Area_C+Area_E-Area_D) Mouth: (Area_F+Area_H-Area_G) They can be different sizes, polarity and aspect ratios Face detection v8a 17 A B C D E F G H

18 .Face detection v8a 18 Face feature and example -1 +2 A face Shaded area White area F=Feat_val = pixel sum in white area - pixel sum in shaded area Example Pixel sum in white area= 216+102+78+129+210+111=846 Pixel sum in shared area= 10+20+4+7+45+7=93 Feat_val=F=846-93=753 If F>threshold, feature= +1 Else feature= -1 End if; If we can choose threshold =700 , so feature is +1. 10 20 4 7 45 7 216 102 78 129 210 111 Pixel values inside the areas

19 .Definition: Area_X = sum of pixels in the rectangular area from the left-top corner to pixel X (including the top left corner and pixel X). Find the feature output of this image. Area_D=1 Area_B=1+2+3=6 Area_C =1+3=4 Area_A=1+2+3+3+4+6=19 Area_E=? 1+3+5=9 Area_F=? 1+2+3+3+4+6+5+2+4=30 Pixel sum of the area inside the box enclosed by the dotted lines= Area_F - Area_B - Area_E +Area_D =? 30 -6-9+1=16 1 2 3 3 3 4 6 3 5 2 4 1 0 2 3 6 Face detection v8a 19 D C A B E F Top-left corner

20 .Class exercise 8. 3 Definition: Area at X =pixel sum of the area from top-left corner to X= Area_X Find the feature output of this image. Area_D=1 Area_B=1+2+3=6 Area_C =1+3=4 Area_A=1+2+3+3+4+6=19 Area_E=? 1+3+5=9 Area_F=? 1+2+3+3+4+6+5+2+4=30 Pixel sum of the area inside the box enclosed by the dotted lines= Area_F - Area_B - Area_E +Area_D =30 -6-9+1=16 WA=White area enclosed by the dotted line=? GA=Gray area enclosed by the dotted line=? (white area-shaded area)=WA-WG=? 1 2 3 3 3 4 6 3 5 2 4 1 0 2 3 6 Face detection v8a 20 D C A B E F Top-left corner

21 .4 basic types of Rectangular Features for (white_area)-(gray_area) Type) Rows x columns Type 1) 1x2 Type 2) 2x1 Type 3) 1x3 Type 4) 3x1 Type 5) 2x2 Each basic type can have difference sizes and aspect ratios. I.e. the following feature windows are of the same type (Type2) even they have different sizes, or aspect ratios Each rectangle inside is of the same dimension Face detection v8a 21

22 .Faces can be any sizes, Example: a face can be big or small , from to 24 x24 to 1024x1024, There are faces with different sizes So, we need feature windows with different sizes. As long as white/gray areas have the relations The followings are Type2 Rectangular Features The white rectangle is above the shaded rectangle White and shaded rectangle are of same dimension Face detection v8a 22

23 .Face detection v8a 23 Class exercise 8.4 Feature selection [Lazebnik09 ] For a 24x24 detection region, the number of possible rectangle features is ~160,000! Name the types (type 1,2,3,4,5) of the rectangular features in the figures. Some examples and their types Fill in the types for the 2nd, 3rd rows 2 5 1 4 3 Standard Rectangular Feature Types 1) 2) 3) 4) 5)

24 .Class exercise 8.5: Features in a 24x24 (pixel) window Exercise 8.5a : How many rectangular features of all-5-types can be found a 24x24 pixel window? Answer: 162,336 (explain) Exercise 8.5b : How many type 1 features are in a 24x24 (pixel) window? Answer:_43200 (explain) Face detection v8a 24 http://stackoverflow.com/questions/1707620/viola-jones-face-detection-claims-180k-features

25 .Standard Types 1) 2) 3) 4) 5) Class exercise 8.6? Still keeping the 5 basic rectangular features types (1,2,3,4,5) (5 types: 2x1,1x2,3x1,1x3,2x2) Find the number of rectangular features for a resolution of 36 x36 windows Answer: 816264 , explain your answer. Face detection v8a 25

26 .The Viola and Jones method Technique 2: AdaBoost for face detection Face detection v8a 26

27 .Class exercise 8.7: The detection challenge Use 24x24 base window For y=1;y<=1024;y++ {For x=1;x<=1024;x++{ Set (x,y) = the left top corner of the 24x24 sub-window, different scales are needed to be considered too. For the 24x24 sub-window, extract 162,336 features and see they combine to form a face or not.} } Exercise 7 : Discuss the number of operations required. Conclusion : too slow, solution use boosting Face detection v8a 27 X-axis Y-axis (x,y) 24x24Sub-window 1024 (1,1) 1280 Answer 7:

28 .Solution to make it efficient The whole 162,336 feature set is too large Solution: select good features to make it more efficient. Use: “Boosting” Boosting Combine many small weak classifiers to become a strong classifier. Training is needed. Face detection v8a 28

29 .Face detection v8a 29 Boosting for face detection Define weak learners based on rectangle features window value of rectangle feature P t = polarity{+1,-1} threshold