The Last Mile On Democratizing Ai

1 .Last Mile On Democratizing AI Howard Huang, Huawei Jianfeng Ding, Intel 

2 .Outline ➔ Diffuse The Hype ➔ Introduce Cyborg Project ➔ Intel’s Recent Effort in AI ➔ Look Into The Future 

3 .Everyone is talking about democratizing AI 

4 .But it can’t be truly done without an open cloud infrastructure ● Tensorflow, CNTK, Pytorch, Caffe, MXNET, … Basically everything you can Application find now about major AI related open source projects ● Same goes to majority of the research papers Infrastructure ??? 

5 .Interesting comparison on Blockchain and AI Most hyped AI technologies Blockchain Level of Good Sense Few Sense understanding on Infrastructure Few work Good work 

6 .Define a Cloud Infrastructure For AI PaaS Cloud Infrastructure IaaS Hardware Heterogeneous Physical Resource (CPU,GPU,FPGA,ASIC,...) 

7 .Asking the right question Can we have an AI cloud infrastructure software which (1) Provides nice abstraction and management of the heterogeneous resources (2) Is open source and driven by an open community (3) Fascilates the e2e AI development 

8 .Outline ➔ Diffuse The Hype ➔ Introduce Cyborg Project ➔ Intel’s Recent Effort in AI ➔ Look Into The Future 

9 .Cyborg is a general management framework for accelerators Proud OpenStack Official Project since 2017.09 (https://github.com/openstack/cyborg ) 

10 .Cyborg Project Overview ● Subteams: release, driver, doc ● Active Chinese Dev wechat group (48 members) from companies like Huawei, China Mobile, Intel, Lenovo, ZTE, Tencent, Nokia, Unionpay, 99Cloud, Xilinx, Inspur, iFlyTech, UC Berkeley, UIUC, CMU ● Lots of gifs ... 

11 .Cyborg Pike Release cyborg-api Legend cyborg-conductor cyborg-db Pike Finished Out-of-scope cyborg-agent vendor-a-driver vendor-b-driver vendor-a-acc vendor-b-acc 

12 .Cyborg Queens Release cyborg-api Legend cyborg-db cyborg-conductor Pike Finished (resource provider) Queens Finished Out-of-scope cyborg-agent report SPDK driver cyborg-generic-driver Intel FPGA driver NVMe SSD vendor-acc-test Intel FPGA 

13 . Cyborg Rocky Release Planning (OpenStack) pythonclient-cyborg cyborg-api os-acc Legend Progr cyborg-db cyborg-conductor Quota Pike Finished aming (resource provider) Queens Finished Rocky Planned cyborg-agent report Out-of-scope SPDK driver Intel FPGA driver NVIDIA GPU driver Xilinx FPGA driver NVMe SSD Intel FPGA NVIDIA GPU Xilinx FPGA 

14 .Cyborg Rocky Release Planning (Kubernetes) ● Align Cyborg data model with DPI before 1.13 release ● Cyborg DPI Plugin releady when DPI GA ● Consider the possibility of a CRD Acc controller ● Could be utilized by Kubeflow containerized 

15 .Cyborg is forming 

16 .Outline ➔ Diffuse The Hype ➔ Introduce Cyborg Project ➔ Intel’s Recent Effort in AI ➔ Look Into The Future 

17 .Intel AI Compute Continuum Cloud/Data Center Edge Devices Large scale data centers Small scale data centers, User-touch endpoint such as public cloud or small business IT devices with lower power comms service providers, infrastructure, to few requirements such as gov’t and academia, large on-premise server racks and laptops, tablets, smart enterprise IT workstations home devices, drones 

18 .

19 .Intel® Xeon® Scalable Processors 

20 .Intel® Deep Learning Inference Accelerators 

21 .Intel® Omni-Path Architecture 

22 .Intel® Next Generation High Performance Storage 

23 .Intel® Optimized AI Libraries 

24 .Intel® Optimized AI Libraries (continue) 

25 .Deep Learning Frameworks Many Popular DL Frameworks are now optimized for CPU 

26 .Intel® Open AI Cloud Reference Architecture 

27 .Outline ➔ Diffuse The Hype ➔ Introduce Cyborg Project ➔ Intel’s Recent Effort in AI ➔ Look Into The Future 

28 .Future #1 : AI Native Open Infrastructure Infrastructure As Code - Programmable Framework Infrastructure As Model - Learnable Framework 

29 .Future #1 : AI Native Open Infrastructure Software 1.0 : Write a complex fixed cyborg-api scheduler cyborg-conductor cyborg-db Where is the scheduler ? cyborg-agent