HBase Practice at Ke.com

1 . hosted by Hbase Practice At ke.com Guoxian Zhao 

2 . hosted by Content 01 Architecture Evolution 02 Olap Scenario 03 Monitor Scenario 04 Data Mining Scenario 05 Reatime ETL Scenario 06 Hdic Scenario 

3 . hosted by 01 Architecture Evolution 

4 . hosted by bigdata architecture for ke Evolution 

5 . hosted by 02 Olap Scenario 

6 . hosted by Olap Scenario Requirement List Billion data set High concurrency subsecond response times sql interface 

7 . hosted by Olap Scenario kylin 

8 . hosted by Olap Scenario Olap data architecture map product Cube Metrics api redis query execute redis permission adhoc Meta Statistics manager queryengine Kylin middleware monitor Sparksql presto zk nginx Offline Spark cluster kylin meta build build query query Alluxio hadoop, warehouse + compute Hbase scheduler load … … 

9 . hosted by Data For Kylin  800+cube、16+business  200T storage、160000 million data、1 cube 6000 million  Query/day million， <500ms(95%), <1s (99%) 

10 . hosted by Data Product Metrics platform • roll-up drill • multidimensional analysis • self-service configuration report • standard indices 

11 . hosted by Performance Optimizations For Hbase SSD-FIRST Clien t OR H D S S D H D D • Read remote ssds first D DN1 DN2 DN3 

12 . hosted by Performance Optimizations For Hbase others  HDFS short-circuit  Data hedged ，multi thread read  Test stale datanodes, avoid the slow datanode  Shutdown balance  Multiwal， Improve write performance 

13 . hosted by 03 Monitor Scenario 

14 . hosted by Monitor senario grafana TeleGraf TSD Task metrics HAProxy HBASE Hadoopmetrics TSD 

15 . hosted by Monitor Scenario 

16 . hosted by 04 Data Mining Scenario 

17 . hosted by Data Mining Scenario 

18 . hosted by 05 RealTime ETL Scenario 

19 . hosted by RealTime ETL Scenario Subtitle Text RealTime DB ETL RealTime log ETL fast write bigdata storage HBASE 3 fast read 4 change record 

20 . hosted by 06 Hdic Scence 

21 . hosted by Hdic Scenario Subtitle Text 3 4 

22 . hosted by Data For Ke 

23 .hosted by Thanks