Flink Streaming SQL 2018 Part I

1 . Flink Streaming SQL 2018 ‫ݪل‬ғ data Artisans ᘳ֖ғ Software Engineer ᄍᦖᘏғ Piotr Nowojsk 

2 . About data Artisans ‫ى‬ԭ data Artisans PLATFORM Original Creators of Real-Time Stream Processing Apache Flink® Enterprise Ready Apache Flink ܻত֢ᘏ մӱᕆਫ෸ၞୗ॒ቘ 

3 .Flink Streaming SQL 2018 

4 . ፓ୯ $JHQGD ԅ֜ᭌೠ 64/Ҙ Why SQL? ᭌೠ 6WUHDPLQJ64/ਖ਼ᥝᶎ੒ጱ೴౴ Challenges in Streaming SQL ࣁ 6WUHDPLQJ64/Ӿᬳളᤒጱӧ‫ݶ‬ොୗ Various ways to join tables in Streaming SQL ཛྷୗᦩ‫ڦ‬ Pattern recognition ٌ՜ᬪ๗౮ຎ Other recent improvements. 

5 . ԅ֜ᭌೠ SQL Why SQL? ռಅޮᎣጱള‫ݗ‬ :HOONQRZQLQWHUIDFH ෫ᵱᖫᑕńńฃԭӤಋ 1RSURJUDPPLQJLVUHTXLUHG HDVLHUWROHDUQ ኩᦫୗ᧍᥺ᤒᬡ֦ጱࠟӱ᭦ᬋ 'HFODUDWLYHZD\WRH[SUHVV\RXUEXVLQHVVORJLF ٖୌս۸ 2XWRIWKHER[RSWLPL]DWLRQ 

6 .ᭌೠ Streaming SQL ਖ਼ᥝᶎ੒ጱ೴౴ Challenges in Streaming SQL 

7 . ಢེ॒ቘਫֺ Batch example SELECT a.id FROM A a, B b WHERE a.id = b.id 

8 . SELECT a.id FROM A a, B b WHERE a.id = b.id Table A Table B Result 1 42 1 42 7 42 2 3 3 3 1 6 

9 . ୭ଚᬳളᓒဩ Sort-Merge Join Table A Table B 1 42 42 7 2 3 3 1 6 

10 . ᒫӞྍ ń ‫ړ‬ᔄ First step - Sort Table A Table B 1 1 2 3 3 7 6 42 42 

11 . ᒫԫྍ ‫ݳ‬ଚ݊ᬳള Second step Merge and Join Table A Table B Result 1 1 1 2 3 3 3 7 42 6 42 42 

12 . ग़ᤒᬳളᓒဩ Hash Join Table A Table B Result 1 42 1 42 7 42 2 3 3 3 1 6 

13 . ᬳളᬳᖅັᧃ Join in continuous queries Table A Table B Result ... 42 ... ... 

14 . ᬳളᬳᖅັᧃ Join in continuous queries Table A Table B Result 1 42 ... ... ... 

15 . ᬳളᬳᖅັᧃ Join in continuous queries Table A Table B Result 1 42 42 42 ... ... ... 

16 . ᬳളᬳᖅັᧃ Join in continuous queries Table A Table B Result 1 42 42 42 7 ... ... ... 

17 . ᬳളᬳᖅັᧃ Join in continuous queries Table A Table B Result 1 42 42 42 7 ... ... 3 ... 

18 . ᬳളᬳᖅັᧃ Join in continuous queries Table A Table B Result 1 42 42 42 7 1 ... 3 ... 1 ... 

19 . ᬳളᬳᖅັᧃ Join in continuous queries Table A Table B Result 1 42 42 42 7 1 2 3 3 3 1 ... 6 ... ... 

20 . ෸ᳵᑻ‫ݗ‬ᬳള Time-windowed Join 

21 . ࿜֖ᕚ Watermarks Ԫկ෸ᳵ౿ Ԥၞ Event timestamp Stream (out of order) 21 19 20 17 22 12 14 12 9 15 11 7 ࿜֖ᕚ ࿜֖ᕚ Watermark 17 Watermark 11 Ԫկ Event 

22 . ෸ᳵᑻ‫ݗ‬ᬳള Time-windowed Join SELECT * FROM Orders o, Shipments s WHERE o.id = s.orderId AND s.shiptime BETWEEN o.ordertime AND o.ordertime + INTERVAL '4' HOUR 

23 .෸ᳵᑻ‫ݗ‬ᬳള Time-windowed Join 

24 . ܲ‫ݥ‬ᤒ Temporal Tables