Page 56 - MSDN Magazine, April 2017

TesT Run JAMES MCCAFFREY Kernel Perceptrons Using C#
A kernel perceptron is a machine learning (ML) clas- sifier that can be used to make binary predictions. For example, a kernel perceptron could predict the sex of a person (male = -1, female = +1) based on age, income, height and weight. Kernel perceptrons are an advanced variation of ordinary perceptrons and can handle more complex data.
A good way to see where this article is headed is to take a look at the demo program in Figure 1 and the associated data in Figure 2. The goal of the demo program is to predict the class, -1 or +1, of dummy input data that has just two predictor variables, x0 and x1. For example, the first training data item is (2.0, 3.0, -1), which means that if the input values are x0 = 2.0 and x1 = 3.0, the correct class is -1.
The 21 training data points have a circular geom- etry, which means that simple linear classification techniques, such as ordinary perceptrons or logis- tic regression, are ineffective. Such data is called non-linearly separable.
The demo program uses the training data to
create a prediction model using a kernel perceptron.
Unlike many prediction models that consist of a set
of numeric values called weights, a kernel prediction
model creates a set of counter values, one for each training data item. The demo program displays the counter values for the first two data items (1 and 2) and the last two data items (0 and 1).
After training, the kernel perceptron model predicts all 21 data items correctly, which is to be expected. The trained model is applied to four new test data items that were not used during training. The first test item has inputs (2.0, 4.0) and a class of -1. The prediction model correctly predicts that item. Overall, the model correctly predicts three of the four test items for 0.75 accuracy.
Behind the scenes, the kernel perceptron uses a function called a radial basis function (RBF) kernel. The kernel function requires a parameter called sigma. The value of sigma must be determined by trial and error, and sigma = 1.5 in the demo. Note that by set- ting sigma to 0.5, the demo program would achieve 100 percent accuracy. I used sigma = 1.5 to point out that for most data sets you won’t achieve 100 percent accuracy.
Figure 1 Kernel Perceptron Demo
This article assumes you have intermediate or higher program- ming skills, but doesn’t assume you know anything about kernel perceptrons. The demo program is coded using C#, but you should have no trouble refactoring the code to another language such as Python or JavaScript if you wish. All the key demo code, except for the hardcoded data and some display statements, is presented in this article. The complete demo source code, including data, is available in the accompanying download.
Understanding Kernel Functions
In order to understand kernel perceptrons you must understand kernel functions in general. There are many kernel functions, but the most common one, and the type used by the demo program, is called the RBF kernel. The RBF kernel function accepts two vectors (that is, arrays) and a parameter named sigma, and returns a single value between 0.0 and 1.0 that’s a measure of similarity between the two arrays. A return value of exactly 1.0 means the two arrays are identical. The less similar the two arrays are, the smaller the value of RBF is, approaching but never quite reaching 0.0.
Code download available at msdn.com/magazine/0417magcode.
42 msdn magazine