Page 42 - MSDN Magazine, March 2019

MACHINE LEARNING
Support Vector Machines
Using C#
James McCaffrey
A support vector machine (SVM) is a software system that can make predictions using data. The original type of SVM was designed to perform binary classification, for example pre- dicting whether a person is male or female, based on their height, weight, and annual income. There are also variations of SVMs that can perform multiclass classification (predicting one of three or more classes) and regression (predicting a numeric value).
In this article I present a complete working example of an SVM that’s implemented using only raw C# without any external libraries. A good way to see where this article is headed is to examine the demo program in Figure 1.
The demo begins by setting up eight dummy training data items. Each item has three predictor values, followed by the class to pre- dict encoded as -1 or +1. The dummy data doesn’t represent a real
problem, so the predictor values have no particular meaning. In a realistic SVM problem, it’s important to normalize the predic- tor values, usually by applying min-max normalization so that all values are scaled between 0.0 and 1.0.
The demo creates an SVM classifier that uses a polynomial kernel function. (I’ll explain kernel functions shortly.) Training the SVM model generated three support vectors: (4, 5 7), (7, 4, 2) and (9, 7, 5). These are training data items [0], [1] and [4]. Each sup- port vector has an associated weight value: (-0.000098, -0.000162, 0.000260). Additionally, an SVM model has a single value called a bias, which is -2.505727 for the demo data.
The support vectors, weights and biases define the trained SVM model. The demo program uses the model to generate predicted class values for each of the eight training items. A decision value that’s negative corresponds to a predicted class label of -1 and a positive decision value corresponds to a predicted class of +1. Therefore, the trained model correctly predicts all eight of the training items. This isn’t too surprising because the problem is so simple.
The demo concludes by predicting the class for a new, previously unseen item with predictor values (3, 5, 7). The computed deci- sion value is -1.274 and therefore the predicted class label is -1. The computation of the decision value is explained later in this article.
This article assumes you have intermediate or better program- ming skill with C#, but doesn’t assume you know anything about SVMs. The demo program is coded using C# and though it’s com- plicated, you should be able to refactor it to another language, such as Java or Python, if you wish.
This article discusses:
• The support vector machine (SVM) demo program • Understanding and using SVMs
• Kernel functions
• The sequential minimal optimization algorithm Technologies discussed:
C#, Visual Studio 2107
Code download available at:
msdn.com/magazine/0319magcode
36 msdn magazine