Kadir算子的显著性、尺度和图像描述

1 . The Kadir Operator Saliency, Scale and Image Description Timor Kadir and Michael Brady University of Oxford 1 

2 . The issues… • salient – standing out from the rest, noticeable, conspicous, prominent • scale – find the best scale for a feature • image description – create a descriptor for use in object recognition 2 

3 . Early Vision Motivation • pre-attentive stage: features pop out • attentive stage: relationships between features and grouping 3 

4 .4 

5 .Detection of Salient Features for an Object Class 5 

6 . How do we do this? 1. fixed size windows (simple approach) 2. Harris detector, Lowe detector, etc. 3. Kadir’s approach 6 

7 . Kadir’s Approach • Scale is intimately related to the problem of determining saliency and extracting relevant descriptions. • Saliency is related to the local image complexity, ie. Shannon entropy. • entropy definition H = -∑ Pi log2 Pi i in set of interest 7 

8 . Specifically • x is a point on the image • Rx is its local neighborhood • D is a descriptor and has values {d1, ... dr}. • PD,Rx(di) is the probability of descriptor D taking the value di in the local region Rx. (The normalized histogram of the gray tones in a region estimates this probability distribution.) 8 

9 .Local Histograms of Intensity Neighborhoods with structure have flatter distributions which converts to higher entropy. 9 

10 .Problems Kadir wanted to solve 1. Scale should not be a global, preselected parameter 2. Highly textured regions can score high on entropy, but not be useful 3. The algorithm should not be sensitive to small changes in the image or noise. 10 

11 . Kadir’s Methodology • use a scale-space approach • features will exist over multiple scales – Berghoml (1986) regarded features (edges) that existed over multiple scales as best. • Kadir took the opposite approach. – He considers these too self-similar. – Instead he looks for peaks in (weighted) entropy over the scales. 11 

12 . The Algorithm 1. For each pixel location x a. For each scale s between smin and smax i. Measure the local descriptor values within a window of scale s ii. Estimate the local PDF (use a histogram) b. Select scales (set S) for which the entropy is peaked (S may be empty) c. Weight the entropy values in S by the sum of absolute difference of the PDFs of the local descriptor around S. 12 

13 . Finding salient points • the math for saliency discretized • saliency YD (s, x)  HD (s, x) WD (s, x) HD (s, x)   ps , x(d ) log 2 ps , x(d ) • entropy d D • weight s2 WD (s, x)   2 s  1 dD ps , x (d )  ps  1, x(d ) based on difference between x point scales s  s, r,    scale, eccentricity, orientation  s D low - level feature domain (gray tones) probability ofof ps , x(d ) histogram descriptor valuesDoftaking D invalue d ins, x region X the region centered at x with scale s = normalized histogram count for the bin representing gray tone d. 13 

14 .Picking salient points and their scales 14 

15 . Getting rid of texture • One goal was to not select highly textured regions such as grass or bushes, which are not the type of objects the Oxford group wanted to recognize • Such regions are highly salient with just entropy, because they contain a lot of gray tones in roughly equal proportions • But they are similar at different scales and thus the weights make them go away 15 

16 . Salient Regions • Instead of just selecting the most salient points (based on weighted entropy), select salient regions (more robust). • Regions are like volumes in scale space. • Kadir used clustering to group selected points into regions. • We found the clustering was a critical step. 16 

17 . Kadir’s clustering (VERY ad hoc) • Apply a global threshold on saliency. • Choose the highest salient points (50% works well). • Find the K nearest neighbors (K=8 preset) • Check variance at center points with these neighbors. • Accept if far enough away from existant clusters and variance small enough. • Represent with mean scale and spatial location of the K points 17 • Repeat with next highest salient point 

18 .More examples 18 

19 . Robustness Claims • scale invariant (chooses its scale) • rotation invariant (uses circular regions and histograms) • somewhat illumination invariant (why?) • not affine invariant (able to handle small changes in viewpoint) 19 

20 .More Examples 20 

21 .Temple 21 

22 .Capitol 22 

23 .Houses and Boats 23 

24 .Houses and Boats 24 

25 .Sky Scraper 25 

26 .Car 26 

27 .Trucks 27 

28 .Fish 28 

29 .Other … 29