Bring Intelligence to the Edge with Intel® Movidius™ Neural Compute Stick

1 .Bring Intelligence to the Edge with Intel® Movidius™ Neural Compute Stick Darren Crews Principal Engineer, Lead System Architect, Intel NTG Intel® Movidius™ Neural Compute Stick Program 

2 .Legal Disclaimer INFORMATION IN THIS DOCUMENT IS PROVIDED IN CONNECTION WITH INTEL PRODUCTS. NO LICENSE, EXPRESS OR IMPLIED, BY ESTOPPEL OR OTHERWISE, TO ANY INTELLECTUAL PROPERTY RIGHTS IS GRANTED BY THIS DOCUMENT. EXCEPT AS PROVIDED IN INTEL'S TERMS AND CONDITIONS OF SALE FOR SUCH PRODUCTS, INTEL ASSUMES NO LIABILITY WHATSOEVER AND INTEL DISCLAIMS ANY EXPRESS OR IMPLIED WARRANTY, RELATING TO SALE AND/OR USE OF INTEL PRODUCTS INCLUDING LIABILITY OR WARRANTIES RELATING TO FITNESS FOR A PARTICULAR PURPOSE, MERCHANTABILITY, OR INFRINGEMENT OF ANY PATENT, COPYRIGHT OR OTHER INTELLECTUAL PROPERTY RIGHT. A "Mission Critical Application" is any application in which failure of the Intel Product could result, directly or indirectly, in personal injury or death. SHOULD YOU PURCHASE OR USE INTEL'S PRODUCTS FOR ANY SUCH MISSION CRITICAL APPLICATION, YOU SHALL INDEMNIFY AND HOLD INTEL AND ITS SUBSIDIARIES, SUBCONTRACTORS AND AFFILIATES, AND THE DIRECTORS, OFFICERS, AND EMPLOYEES OF EACH, HARMLESS AGAINST ALL CLAIMS COSTS, DAMAGES, AND EXPENSES AND REASONABLE ATTORNEYS' FEES ARISING OUT OF, DIRECTLY OR INDIRECTLY, ANY CLAIM OF PRODUCT LIABILITY, PERSONAL INJURY, OR DEATH ARISING IN ANY WAY OUT OF SUCH MISSION CRITICAL APPLICATION, WHETHER OR NOT INTEL OR ITS SUBCONTRACTOR WAS NEGLIGENT IN THE DESIGN, MANUFACTURE, OR WARNING OF THE INTEL PRODUCT OR ANY OF ITS PARTS. Intel may make changes to specifications and product descriptions at any time, without notice. Designers must not rely on the absence or characteristics of any features or instructions marked "reserved" or "undefined". Intel reserves these for future definition and shall have no responsibility whatsoever for conflicts or incompatibilities arising from future changes to them. The information here is subject to change without notice. Do not finalize a design with this information. The products described in this document may contain design defects or errors known as errata which may cause the product to deviate from published specifications. Current characterized errata are available on request. Contact your local Intel sales office or your distributor to obtain the latest specifications and before placing your product order. Code names featured are used internally within Intel to identify products that are in development and not yet publicly announced for release. Customers, licensees and other third parties are not authorized by Intel to use code names in advertising, promotion or marketing of any product or services and any such use of Intel's internal code names is at the sole risk of the user. *Other names and brands may be claimed as the property of others. Copyright © 2017, Intel Corporation. All rights reserved. Intel® Movidius™ Neural Compute Stick Program 2 

3 . Agenda • Motiviation to move intelligence to the edge • Edge compute use cases • Barriers to moving intelligence to the edge • Deep learning algorithms – can they run on an edge device? • Movidius Neural Compute Stick (arch,usage, etc) • Code walkthrough and demo Intel® Movidius™ Neural Compute Stick Program 

4 . Let’s look at a larger scale… 20 billion connected devices by 2020¹ ... generating billions of petabytes of data traffic between devices & the cloud 1 Source: http://www.gartner.com/newsroom/id/3598917 Intel® Movidius™ Neural Compute Stick Program 

5 . Why move intelligence from the cloud to the edge? What are you? I am asking the cloud if I should vacuum you, too. I'll scratch you down to your motors if you come any closer! Reduce BW Reduce Latency Privacy Intel® Movidius™ Neural Compute Stick Program 

6 . Computer vision and AI at the edge Intel® Movidius™ Neural Compute Stick Program 

7 . But what about Developers? Complete HW + SW solution for developing Deep Learning + application on the edge NC SDK Free download @ developer.movidius.com Intel® Movidius™ Neural Compute Stick Program 

8 .Use Case with the Neural Compute Stick Little Ripper Lifesaver* UAV https://newsroom.intel.com/editorials/data-most-important-force-society-today/ Intel® Movidius™ Neural Compute Stick Program https://www.youtube.com/watch?v=QJOMfDyhUyo 

9 . Key Capabilities Typical use cases could be: • Robot • Security camera • Smart-home assistant Key capabilities: • Object detection • Object classification • Facial recognition • Natural language processing Intel® Movidius™ Neural Compute Stick Program 

10 .Barriers to Moving Compute to the Edge Move compute from the cloud to the edge: • Accuracy • Available compute • Model efficiency • Model size Intel® Movidius™ Neural Compute Stick Program 

11 .Image Classification: Getting more accurate and every year Intel® Movidius™ Neural Compute Stick Program 

12 .Efficiency Key for Edge Devices Alfredo Canziani, Eugenio Culurciello, Adam Paszke “AN ANALYSIS OF DEEP NEURAL NETWORK MODELS FOR PRACTICAL APPLICATIONS” Intel® Movidius™ Neural Compute Stick Program 

13 . MobileNet performance on NCS Real time image classification on NCS https://github.com/tensorflow/models/blob/master/research/slim/nets/mobilenet_v1.md Intel® Movidius™ Neural Compute Stick Program 

14 .Object Detection Benchmarks Deep neural networks win in object detection in 2013. Ross Girshick, IEEE International Conference on Computer Vision (ICCV), 2015 Intel® Movidius™ Neural Compute Stick Program 

15 .Object Detectors – Speed matters! Significant improvements in Object Detectors in the last two years enable Object Detection on edge devices like Intel Movidius NCS. Pascal 2007 MaP Speed R-CNN 66.0 .05FPS Fast R-CNN 70.0 0.5FPS Faster R-CNN 73.2 7FPS GPU YOLO 69.0 45FPS SSD-Mobilenet 72.7 11FPS NCS Jonathan Huang, et al, Speed/accuracy trade-offs for modern convolutional object detectors Intel® Movidius™ Neural Compute Stick Program 

16 .Object Detectors – lots of choices, choose wisely Jonathan Huang, et al, Speed/accuracy trade-offs for modern convolutional object detectors Intel® Movidius™ Neural Compute Stick Program 

17 .Machine Translation Intel® Movidius™ Neural Compute Stick Program 

18 .Intel® Movidius™ Neural Compute Stick Redefining the AI developer kit • Neural network accelerator in USB stick form factor • TensorFlow™ and Caffe frameworks supported, along with many popular networks • Source is available for the SDK, which allows you to compile for other platforms • Features the same Intel Movidius vision processing unit (Intel Movidius VPU) used in drones, surveillance cameras, VR headsets, and other low- power intelligent and autonomous products Intel® Movidius™ Neural Compute Stick Program 

19 .Demo: Scaling inference performance with multiple sticks Intel® Movidius™ Neural Compute Stick Program 

20 .Object Detection and Classification AP Draw Downscale Crop box and class NCS1 NCS2 Object Bird detection classification https://github.com/movidius/ncappzoo/tree/master/apps/birds Intel® Movidius™ Neural Compute Stick Program 

21 . Application Processor Architecture NCSDK Compiler NCAPI Intel Movidius NCS contains the Intel® Movidius™ Myriad™ 2 vision processing unit, including 4 Gbit of LPDDR. USB Intel Movidius NCS is connected to an application NCS processor (AP), such as a Raspberry Pi or UP Leon Microprocessor Squared board. Programmable deep learning engines Shave Shave Shave Shave Execution is controlled by the LEON Shave Shave Shave Shave microprocessor, and the calculations are done on the SHAVE processors. Shave Shave Shave Shave DDR 4Gbit LPDDR Intel® Movidius™ Neural Compute Stick Program 

22 .NC SDK Workflow Intel® Movidius™ Neural Compute Stick Program 

23 .NC SDK Workflow Intel® Movidius™ Neural Compute Stick Program 

24 . Application Processor TensorFlow or Development Flow Caffemodel NCSDK Compiler NCAPI Step 1: Convert the model NCS graph USB NCS Leon Microprocessor Programmable deep learning engines Shave Shave Shave Shave Shave Shave Shave Shave Shave Shave Shave Shave DDR 4Gbit LPDDR Intel® Movidius™ Neural Compute Stick Program 

25 . Trained model can be profiled on NCS for performance: Intel® Movidius™ Neural Compute Stick Program 

26 .NC SDK Workflow Intel® Movidius™ Neural Compute Stick Program 

27 . Application Processor Development Flow NCS NCSDK graph NCS Compiler NCAPI FW Step 2: Load the model and the FW USB NCS Leon Microprocessor Programmable deep learning engines Shave Shave Shave Shave Shave Shave Shave Shave Shave Shave Shave Shave DDR 4Gbit LPDDR Intel® Movidius™ Neural Compute Stick Program 

28 . Application Processor Development Flow NCSDK Compiler NCAPI Step 3: Perform inference • Load the image USB • Run the model NCS • Return the results Leon Microprocessor Programmable deep learning engines Shave Shave Shave Shave Shave Shave Shave Shave Shave Shave Shave Shave DDR 4Gbit LPDDR Intel® Movidius™ Neural Compute Stick Program 

29 . Demo Intel® Movidius™ Neural Compute Stick Program