Page 21 - MSDN Magazine, July 2018

MACHINE LEARNING
Machine Learning with
IoT Devices on the Edge
James McCaffrey
Imagine that, in the not too distant future, you’re the designer of a smart traffic intersection. Your smart intersection has four video cameras connected to an Internet of things (IoT) device with a small CPU, similar to a Raspberry Pi. The cameras send video frames to the IoT device, where they’re analyzed using a machine learning (ML) image-recognition model and control instructions are then sent to the traffic signals. One of the small IoT devices is connected to Azure Cloud Services, where information is logged and analyzed offline.
This is an example of ML on an IoT device on the edge. I use the term edge device to mean anything connected to the cloud, where cloud refers to something like Microsoft Azure or a com- pany’s remote servers. In this article, I’ll explain two ways you can design ML on the edge. Specifically, I’ll describe how to write a custom model and IO function for a device, and how to use the Microsoft Embedded Learning Library (ELL) set of tools to deploy an optimized ML model to a device on the edge. The custom IO approach is currently, as I write this article, the most common way to deploy an ML model to an IoT device. The ELL approach is forward-looking.
Even if you’re not working with ML on IoT devices, there are at least three reasons why you might want to read this article. First, the design principles involved generalize to other software development scenarios. Second, it’s quite possible that you’ll be working with ML and IoT devices relatively soon. Third, you may just find the tech- niques described here interesting in their own right.
Why does ML need to be on the IoT edge? Why not just do all processing in the cloud? IoT devices on the edge can be very inex- pensive, but they often have limited memory, limited processing capability and a limited power supply. In many scenarios, trying to perform ML processing in the cloud has several drawbacks.
Latency is often a big problem. In the smart traffic intersection example, a delay of more than a fraction of a second could have disastrous consequences. Additional problems with trying to perform ML in the cloud include reliability (a dropped network connection is typically impossible to predict and difficult to deal with), network availability (for example, a ship at sea may have connectivity only when a satellite is overhead) and privacy/security (when, for exam- ple, you’re monitoring a patient in a hospital.)
This article doesn’t assume you have any particular background or skill set but does assume you have some general software devel- opment experience. The demo programs described in this article (a Python program that uses the CNTK library to create an ML model, a C program that simulates IoT code and a Python program that uses an ELL model) are too long to present here, but they’re available in the accompanying file download.
What Is a Machine Learning Model?
In order to understand the issues with deploying an ML model to an IoT device on the edge, you must understand exactly what an ML model is. Very loosely speaking, an ML model is all the infor- mation needed to accept input data, make a prediction and generate output data. Rather than try to explain in the abstract, I’ll illustrate the ideas using a concrete example.
Take a look at the screenshot in Figure 1 and the diagram in Figure 2. The two figures show a neural network with four input nodes, five hidden layer processing nodes and three output layer nodes. The
The ELL system is an open source project and is in the very early stages of development. Therefore, some of the information about ELL in this article may have changed by the time you read this article.
This article discusses:
• What a machine learning (ML) model consist of
• Deploying a standard ML model to an IoT device
• Writing a custom code solution
• Using the Microsoft Embedded Learning Library
Technologies discussed:
Visual Studio Code, Microsoft Embedded Learning Library (ELL), Python, Microsoft CNTK library
Code download available at:
msdn.com/magazine/0718magcode
July 2018 15