Page 70 - MSDN Magazine, October 2017

Figure 3 NeuralTimeSeries Program Structure
Figure 4 Setting up the Training Data
using System;
namespace NeuralTimeSeries {
class NeuralTimeSeriesProgram {
static void Main(string[] args) {
Console.WriteLine("Begin times series demo"); Console.WriteLine("Predict airline passengers "); Console.WriteLine("January 1949 to December 1960 ");
double[][] trainData = GetAirlineData();
trainData = Normalize(trainData); Console.WriteLine("Normalized training data:"); ShowMatrix(trainData, 5, 2, true); // first 5 rows
int numInput = 4; // Number predictors int numHidden = 12;
int numOutput = 1; // Regression
Console.WriteLine("Creating a " + numInput + "-" + numHidden + "-" + numOutput + " neural network");
NeuralNetwork nn = new NeuralNetwork(numInput, numHidden, numOutput);
int maxEpochs = 10000;
double learnRate = 0.01;
double[] weights = nn.Train(trainData, maxEpochs, learnRate);
Console.WriteLine("Model weights and biases: "); ShowVector(weights, 2, 10, true);
double trainAcc = nn.Accuracy(trainData, 0.30); Console.WriteLine("\nModel accuracy (+/- 30) on training " +
"data = " + trainAcc.ToString("F4"));
double[] future = new double[] { 5.08, 4.61, 3.90, 4.32 }; double[] predicted = nn.ComputeOutputs(future); Console.WriteLine("January 1961 (t=145): "); Console.WriteLine((predicted[0] * 100).ToString("F0"));
Console.WriteLine("End time series demo ");
Console.ReadLine(); } // Main
static double[][] Normalize(double[][] data) { . . }
static double[][] GetAirlineData() {. . }
static void ShowMatrix(double[][] matrix, int numRows, int decimals, bool indices) { . . }
static void ShowVector(double[] vector, int decimals, int lineLen, bool newLine) { . . }
public class NeuralNetwork { . . } } // ns
double[][] trainData = GetAirlineData(); trainData = Normalize(trainData); Console.WriteLine("Normalized training data:"); ShowMatrix(trainData, 5, 2, true);
Method GetAirlineData is defined as:
static double[][] GetAirlineData() {
double[][] airData = new double[140][];
airData[0] = new double[] { 112, 118, 132, 129, 121 }; airData[1] = new double[] { 118, 132, 129, 121, 135 };
...
airData[139] = new double[] { 606, 508, 461, 390, 432 }; return airData;
}
The demo uses a simple single-hidden-layer neural network, implemented from scratch. Alternatively, you can use the tech- niques presented in this article along with a neural network library such as Microsoft Cognitive Toolkit (CNTK).
The demo begins by setting up the training data, as shown in Figure 4.
Here, the rolling-window data is hardcoded with a window size of 4. Before writing the time-series program, I wrote a short utility pro- gram to generate the rolling-window data from the raw data. In most non-demo scenarios you’d read raw data from a text file, and then programmatically generate rolling-window data, where the window size is parameterized so you could experiment with different sizes.
The Normalize method just divides all data values by a constant 100. I did this purely for practical reasons. My first attempts with non-normalized data led to very poor results, but after normaliza- tion, my results were much better. In theory, when working with
neural networks, your data doesn’t need to be normalized, but in practice normalization often makes a big difference.
The neural network is created like so:
int numInput = 4; int numHidden = 12; int numOutput = 1; NeuralNetwork nn =
new NeuralNetwork(numInput, numHidden, numOutput);
The number of input nodes is set to four because each rolling window has four predictor values. The number of output nodes is set to one because each set of window values is used to make a prediction for the next month. The number of hidden nodes is set to 12 and was determined by trial and error.
The neural network is trained and evaluated with these statements:
int maxEpochs = 10000;
double learnRate = 0.01;
double[] weights = nn.Train(trainData, maxEpochs, learnRate); ShowVector(weights, 2, 10, true);
The Train method uses basic back-propagation. There are many variations, including using momentum or adaptive learning rates to increase training speed, and using L1 or L2 regularization or drop- out to prevent model over-fitting. The helper method ShowVector displays a vector with real values formatted to 2 decimals places, 10 values per line.
My first attempts with non- normalized data led to very poor results, but after normalization, my results were much better.
After the neural network time-series model has been created, its prediction accuracy is evaluated:
double trainAcc = nn.Accuracy(trainData, 0.30); Console.WriteLine("\nModel accuracy (+/- 30) on " +
" training data = " + trainAcc.ToString("F4"));
For time-series regression, deciding whether a predicted value is correct or not depends on the problem being investigated. For the airline passenger data, method Accuracy marks a predicted passenger count as correct if the unnormalized predicted count is plus or minus 30 of the actual raw count. For the demo data, the first five predictions, for t = 5 to t = 9 are correct, but the prediction for t = 10 is incorrect:
66 msdn magazine
Test Run