Introduction
Time series modeling/forecasting plays an important role that contributes to the success of business. Time series modeling/ forecasting is the use of a model to predict future values based on previously observed values. Therefore, the time series modeling applications will vary from market analysis, stock prediction, and sale planning to non-linear dynamic system modeling in robotics and manufacturing applications.
There are many different time series modeling methods, and machine learning equipped with desired function approximation characteristics would be a great candidate to model any nonlinear time series data.
In this article, we will explore how a neural network can learn a generalized model of a function, and we choose a simple sine wave with a small dataset that contains historical sine wave values at a certain time.
For time series modeling applications, the small dataset will cause the generalization problem. This problem is caused by an overfitting issue that occurred during neural network training. The error on the training set is driven to a very small value, but when new data is presented to the network, the error is large. The network has memorized the training examples, but it has not learned to generalize to new situations.
To demonstrate that Ying Xie has used both a Multilayer Perceptron Layer (MLP) neural network, Convolution Neural Network, and Deep Learning LSTM Recurrent Neural Network to model a simple time series sine wave. However, the results are not very promising due to the overfitting issue.
To improve the neural network generalization and avoid overfitting issues, MathWorks has recommended using the automated regularization technique.
This is a complicated technique as it involves probability theory to train a neural network, but let's see how ANNHUB can handle it with ease and can export the trained time series model into a real application.
Data preparation
In this article, we are going to design a neural network time series model to approximate the sine wave function described as follows:
y=sin(2*pi*t), where t is a time in the range -1 to 1.
For each value of time t, we can generate the according to output value for y, which leads to a training data set in a 2D array.
The dataset is small, with only 400 data samples that present the historical data of the sine wave.
The Sine data set can be found in the Examples-> Fitting Example folder under the ANNHUB installation folder.
This data set contains a training data set (SineTraining.csv) used to train a neural network test data set (SimpleTest.csv) to test/evaluate the obtained trained neural network.
The data structure contains 1 input feature (time), and 1 output class (sine wave value at input time) = sin(2*pi*time).
Design a neural network using ANNHUB.
In this article, we are going to design a Bayesian Neural Network (BNN). A BNN refers to extending standard MLP neural networks with posterior inference to achieve auto regularization that will improve the neural network generalization and overfitting for small dataset applications. With this special technique, the validation set is not required during the training process, so we can save more data for training.
Designing BBN in ANNHUB is simple; the following steps are required.
Load training dataset into ANNHUB
When the time series data is in the correct CSV format, it can be easily imported into the ANNHUB.
Configure Neural Network
To configure the Bayesian Neural Network, we only need to choose Bayesian Regularization as the training engine (1), and ANNHUB will automatically treat this network as BNN.
ANNHUB will auto-configure this neural network structure; we only need to choose a number of hidden nodes (if we want) in (5).
Train Neural Network
ANNHUB will suggest the stopping criteria and training parameters for us; we only need to change the changing epochs and the number of Max Fails if the training algorithm cannot find the optimal performance index.
Being configured as a BNN, the auto regularization technique is automatically applied in the training algorithm, and the validation set for the early stopping technique is not required. ANNHUB will run through iterations and determine the best solution for us. Easy as a piece of cake!
Evaluate the trained Neural Network.
Although the advanced auto-regularization technique has been applied in the training process, we still need to evaluate the trained neural network to see if it is able to correctly model random time series data.
ANNHUB provides a built-in evaluation tool, which is regression curves, to allow us to evaluate the trained model in both the training data set and test data set, giving us the confidence to use the trained model in actual production.
Test trained Neural Network with a new data set
We still need to verify if the trained BNN time series model can actually model the sine time-series waveform. ANNHUB provides a test interface that allows us to load test data to verify our trained model before we decide to export it for production/deployment.
As can be seen in the test interface below, the trained BNN time series model can correctly approximate the sine wave with the very desired accuracy, much better than other methods mentioned by Ying Xie.
Export trained neural network model to weight file
After being evaluated, tested, and verified, the trained BNN time series is ready to be exported into supported programming languages for deployment.
LabVIEW Deployment
At this stage, we already have a trained neural network saved in a text file with the .ann extension (for example, TrainModel_lv.ann); we need to get the License Content by using a valid username and password in GetLicenseContent.vi.
Assume that we do not specify any License File Path in GetLicenseContent.vi, so this VI will generate the licensed content, ANNLV.lic, in the working directory.
Note: We only need to get the licensed content once for a given target PC, as this license content file can be used in other applications running on the same PC.
With a trained neural network model and licensed content, we can create a neural network model using Create.vi.
The following block diagram demonstrates 5 steps to use ANNAPI to load a trained neural network and a license content file in order to create a neural network model to approximate the sine wave.
In this article, we have demonstrated how machine learning can be applied to model a time series of data based on previously observed values. We also can see how the Bayesian Neural Network would be the most suitable neural network to avoid the over-fitting issue in small data set problems and achieve very promising prediction results.
Comments