Page 58 - MSDN Magazine, June 2018

The neural network has two hidden layers. The X object as acts the input to the first hidden layer; the first hidden layer acts as input to the second hidden layer; and the second hidden layer acts as input to the output layer.
The heart of CNTK functionality is the ability to train a neural network model.
The two hidden layers use tanh (hyperbolic tangent) activation. The two main alternatives are logistic sigmoid and rectified linear units (ReLU) activation. The output layer uses the “None” activation, which means the values of the output nodes aren’t modified. This is
Figure 3 Regression Demo Program
the design pattern to use for a regression problem. Using no activation is sometimes called using the identify activation function because the mathematical identity function is f(x) = x, which has no effect.
The demo program creates an alias named “model” for the out- put layer. This technique is optional and is a bit subtle. The idea here is that a neural network is essentially a complex math func- tion. The output nodes conceptually represent both a layer of the network and the network/model as a whole.
Training the Model
The heart of CNTK functionality is the ability to train a neural
network model. Training is prepared with these statements:
tr_loss = C.squared_error(model, Y) max_iter = 50000
batch_size = 11
learn_rate = 0.005
learner = C.adam(model.parameters, learn_rate, 0.99) trainer = C.Trainer(model, (tr_loss), [learner])
# hydro_reg.py
# CNTK 2.4 with Anaconda 4.1.1 (Python 3.5, NumPy 1.11.1) # Predict yacht hull resistance based on six predictors
import numpy as np import cntk as C
def create_reader(path, input_dim, output_dim, rnd_order, sweeps):
x_strm = C.io.StreamDef(field='predictors',
shape=input_dim, is_sparse=False)
y_strm = C.io.StreamDef(field='resistance',
shape=output_dim, is_sparse=False)
streams = C.io.StreamDefs(x_src=x_strm, y_src=y_strm) deserial = C.io.CTFDeserializer(path, streams) mb_src = C.io.MinibatchSource(deserial,
randomize=rnd_order, max_sweeps=sweeps) return mb_src
# ========================================================
def main():
print("\nBegin yacht hull regression \n") print("Using CNTK version = " + \
str(C.__version__) + "\n")
input_dim = 6 # center of buoyancy, etc. hidden_dim = 5
output_dim = 1 # residuary resistance
train_file = ".\\Data\\hydro_data_cntk.txt"
# data resembles:
# |predictors 0.540 0.542 . . |resistance 0.001 # |predictors 0.540 0.542 . . |resistance 0.004
# 1. create neural network model
X = C.ops.input_variable(input_dim, np.float32) Y = C.ops.input_variable(output_dim)
print("Creating a 6-(5-5)-1 tanh regression NN for \ yacht hull dataset ")
with C.layers.default_options(): hLayer1 = C.layers.Dense(hidden_dim,
activation=C.ops.tanh, name='hidLayer1')(X) hLayer2 = C.layers.Dense(hidden_dim,
activation=C.ops.tanh, name='hidLayer2')(hLayer1) oLayer = C.layers.Dense(output_dim,
activation=None, name='outLayer')(hLayer2) model = C.ops.alias(oLayer) # alias
# 2. create learner and trainer
print("Creating a squared error batch=11 Adam \ fixed LR=0.005 Trainer \n")
tr_loss = C.squared_error(model, Y)
max_iter = 50000 batch_size = 11 learn_rate = 0.005
learner = C.adam(model.parameters, learn_rate, 0.99) trainer = C.Trainer(model, (tr_loss), [learner])
# 3. create reader for train data
rdr = create_reader(train_file, input_dim, output_dim,
rnd_order=True, sweeps=C.io.INFINITELY_REPEAT) hydro_input_map = {
X : rdr.streams.x_src,
Y : rdr.streams.y_src }
# 4. train
print("Starting training \n") for i in range(0, max_iter):
curr_batch = rdr.next_minibatch(batch_size, input_map=hydro_input_map)
trainer.train_minibatch(curr_batch) if i % int(max_iter/10) == 0:
mcee = trainer.previous_minibatch_loss_average print("batch %6d: mean squared error = %8.4f" % \
(i, mcee)) print("\nTraining complete")
# (could save model to disk here)
# 5. use trained model to make some predictions np.set_printoptions(precision=2, suppress=True)
inpts = np.array(
[[0.520000, 0.785714, 0.550000, 0.405512, \
0.648352, 0.000000],
[1.000000, 1.000000, 0.550000, 0.562992, \
0.461538, 1.000000]], dtype=np.float32)
actuals = np.array([0.003044, 0.825028], dtype=np.float32)
for i in range(len(inpts)):
print("\nInput: ", inpts[i])
pred = model.eval(inpts[i])
print("predicted resistance: %0.4f" % pred[0][0]) print("actual resistance: %0.4f" % actuals[i])
print("\nEnd yacht hull regression ")
# ========================================================
if __name__ == "__main__": main()
52 msdn magazine
Test Run