13_Image preprocessing and feature extraction

1 .Computer vision: models, learning and inference Chapter 13 Image preprocessing and feature extraction

2 .Preprocessing The goal of pre-processing is to try to reduce unwanted variation in image due to lighting, scale, deformation etc. to reduce data to a manageable size Give the subsequent model a chance Preprocessing definition: deterministic transformation of pixels p to create data vector x Usually heuristics based on experience 2 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

3 .Structure 3 3 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Per-pixel transformations Edges, corners, and interest points Descriptors Dimensionality reduction

4 .Normalization Fix first and second moments to standard values Remove contrast and constant additive luminance variations Before After 4 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

5 .Histogram Equalization Make all of the moments the same by forcing the histogram of intensities to be the same Before/ normalized/ Histogram Equalized 5 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

6 .Histogram Equalization 6 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

7 .Convolution Takes pixel image P and applies a filter F Computes weighted sum of pixel values, where weights given by filter. Easiest to see with a concrete example 7 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

8 .Blurring (convolve with Gaussian) 8 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

9 .Gradient Filters Rule of thumb: big response when image matches filter 9 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

10 .Gabor Filters 10 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

11 .Haar Filters 11 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

12 .Local binary patterns 12 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

13 .Textons An attempt to characterize texture Replace each pixel with integer representing the texture ‘type’ 13 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

14 .Computing Textons Take a bank of filters and apply to lots of images Cluster in filter space For new pixel, filter surrounding region with same bank, and assign to nearest cluster 14 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

15 .Structure 15 15 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Per-pixel transformations Edges, corners, and interest points Descriptors Dimensionality reduction

16 .Edges (from Elder and Goldberg 2000) 16 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

17 .Canny Edge Detector 17 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

18 .Canny Edge Detector Compute horizontal and vertical gradient images h and v 18 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

19 .Canny Edge Detector Quantize to 4 directions 19 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

20 .Canny Edge Detector Non-maximal suppression 20 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

21 .Canny Edge Detector Hysteresis Thresholding 21 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

22 .Harris Corner Detector Make decision based on image structure tensor 22 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

23 .SIFT Detector Filter with difference of Gaussian filters at increasing scales Build image stack (scale space) Find extrema in this 3D volume 23 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

24 .SIFT Detector Identified Corners Remove those on edges Remove those where contrast is low 24 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

25 .Assign Orientation Orientation assigned by looking at intensity gradients in region around point Form a histogram of these gradients by binning. Set orientation to peak of histogram. 25 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

26 .Structure 26 26 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Per-pixel transformations Edges, corners, and interest points Descriptors Dimensionality reduction

27 .Sift Descriptor Goal: produce a vector that describes the region around the interest point. All calculations are relative to the orientation and scale of the keypoint Makes descriptor invariant to rotation and scale 27 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

28 .Sift Descriptor 1. Compute image gradients 2. Pool into local histograms 3. Concatenate histograms 4. Normalize histograms 28 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

29 .HoG Descriptor 29 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince