ETL Made Easy with Azure Data Factory and Azure Databricks

1 .WIFI SSID:SparkAISummit | Password: UnifiedAnalytics 

2 .ETL Made Easy with Azure Data Factory & Azure Databricks Mark Kromer Sr. Azure Data Program Manager Microsoft #UnifiedAnalytics #SparkAISummit 

3 .Cloud ETL Patterns with ADF Azure Data Factory #UnifiedAnalytics #SparkAISummit 3 

4 .Nightly ETL Data Loads Code-free 

5 .Slowly Changing Dimension Scenario 

6 .Load Star Schema DW Scenario 

7 .Data Lake Data Science Scenario 

8 .Workflow Data Pipelines/Control Flow Azure Data Factory #UnifiedAnalytics #SparkAISummit 9 

9 .

10 .

11 .

12 .Mapping Data Flows Azure Data Factory #UnifiedAnalytics #SparkAISummit 13 

13 .What is ADF Mapping Data Flow? • Transform Data, At Scale, in the Cloud, Zero-Code – Cloud-first, scale-out ELT – Code-free dataflow pipelines • Serverless scale-out transformation execution engine • Maximum Productivity for Data Engineers – Does NOT require understanding of Spark / Scala / Python / Java • Resilient Data Transformation Flows – Built for big data scenarios with unstructured data requirements – Operationalize with Data Factory scheduling, control flow and monitoring 

14 .Code-free Data Transformation At Scale • Does not require understanding of Spark, Big Data Execution Engines, Clusters, Scala, Python … • Focus on building business logic and data transformation – Data cleansing – Aggregation – Data conversions – Data prep – Data exploration 

15 .Transformation Function Expression Language 

16 .Debug Data Flows with Data Preview and Data Sampling 

17 .Deep Monitoring Introspection of Data Transformations 

18 .DON’T FORGET TO RATE AND REVIEW THE SESSIONS SEARCH SPARK + AI SUMMIT