26.9.2017
Through various ways I have come to use neural networks in my applications. The first idea was to recognize different objects in the video stream. When I studied the OpenCV library I found a class for image recognition, neural networks and machine learning. Then I realized that I had an old task, which should be solved with neural network nicely.
The graphical expression of the data package can be distinguished by some characteristic phenomena, which at first glance can easily distinguish the trained specialist but it is very difficult to distinguish such a phenomenon, for example, by statistical methods. The difficulty lies in the amount of data. If you want to explore and categorize millions of charts, it is likely that an expert will run away after the third salary increase.
Artificial neural network is able to learn to recognize typical defects and phenomena. It works at night, on Saturdays and Sundays, every day 24/7. It probably will not reach the professional level of your specialist, but when used on large amount of data neural network can produce robust statistical summaries. Then it is easy to find problems or trends in the statistics. At the end, the neural network can save large amounts of money.
Call: +420 777 566 384 or email to address info@hobrasoft.cz
There are a lot of neural network types exist. Neural network can, for example, find faces in camera finder (convolution networks), recognize hand written text (recurent networks) or predict the stock price (recurent networks).
Neural network can be trained to recognize typical patterns and classify them into several groups. Simple feed-forward neural networks are used for classification. Feed-forward network has a simple topology – several input neurons, a hidden layer, and several output neurons. Input neurons are connected directly to the values in the input data, the output neurons then corresponds to categories, depending on their output value.
Two basic procedures are used to train the network:
The human expert will manualy mark out a number of typical situations. Then the network learn from that examples. After learning the network is able to classify the input data automatically.
Learnig of neural network is very simple but extremly laborious, tedious and time comsuming job. But the resulting neural network works fast and efficiently.
The neural network does not know what can be found in the data. After several processing cycles, however, the neural network can recognize some repeating patterns and is able to create its own categories. After initial classification into categories, the network is trained to recognize them and reassembles the input samples with newly acquired experience. The whole procedure is repeated many times and the result is both the category of phenomena created and the input samples grouped into these categories.
I have found a nice example: neural network is able to recognize tiger or cat on images. It automaticaly found that on the images are two different species and grouped them into categories.
The disadvantage is obvious: you never know what the network will find in the data. And at the same time it is an obvious advantage: the network can retrieve from your data things you never know they exist.
Another disadvantage is not so obvious: processing takes a large amount machine time, even on very powerful computers.
I played with neural network for the first time twenty years ago. The Back Propagation algorithm was already known, but I had to implement it myself and the application did not work well. Due to low power I was not able to train the network fast enought. I had a powerful dual-core Sparc but the machine was not comparabe with current PCs.
Twenty years ago there was no other choice then to try to create my own neural network. The situation is better today. I can take complete library with apropriate license.
When looking for a library I prefered the easy use, C++, OpenCL support and license.
I tried about dozen of libraries. Most interesting libraries for me are these three:
Simple feed-forward neural network.
http://www.comp.nus.edu.sg/~dbsystem/singa/
A very universal and powerful project of the Singapore National University. From examples and the documentation available it is ovious that the project is primarily focused on image processing (convolutional neural networks, 2D input data).
Although it is a Microsoft library, it is an opensource and it is possible to run it on the Linux operating system. I only experimented with the library for a while – missing OpenCL support does not allow me to use it on my graphics card. However, I think it is appropriate to mention it here.
If someone goes into a neural network application with no experience or with no theoretical background, the work can be very frustrating.
First attempts with a neural network do not usually work anywhere. After a month of work the network is still stupid and one regrets the time he spent experimenting on something so stupid.
After some time of normalization and transformation, the network begins to show occasional flashes of some abilities. Data for the neural network must be transformed into a form, which is unusual or unreadable by human. The developer gets into a slightly schizophrenic mood, because he sees other input data than the neural network processes.
Over time, certain signs of intelligence are becoming more and more frequent. Then the whole process unexpectedly switch to "It works! The network is brilliant!"
But such a state only enters the next learning cycle when the network starts behave heavily retarded and in this state of mental retardation the network persists several weeks. Mood moves in the loops – from inappropriate optimism to complete despair. How the ammount of training data increases and the data quality improves, cycles between geniality and debility are fortunately gradually shortened.
Still, working with a neural network can bring a weaker character into a madness. The developer can not get rid of the idea the the web site making can produce much more money.
It is necessary to prepare a sufficient number of representative data samples for the neural network. It can be read in the literature that the neural network can handle noisy data. In practice, training data needs to be very carefully selected. If there are any anomalies in the training data, they can very negatively affect the resulting capabilities of the neural network.
With increasing ammount of training data, the time required for the entire cycle also increases. Even on a powerful graphics card the learning process takes several hours, even though that all fans in the computer will be intensively involved in the calculation. The process of network training is best to go overnight. Each experiment, each minor reconfiguration of the neural network will usually take a whole day.
A good job for irrecoverable criminals and assassins is the data selection for training a neural network. For satisfactory results it is necessary to carefully view and mark at least several hundred thousand packets of data. Because errors occur in every data selection, it is necessary to view the training dataset several times.
As a result, the training of the neural network lies in the constant flogging of criminals, murderers and neural networks.
In the learning process the neural network tries to minimize the error between the expected and real results. But the error function has only one dimension and there are usually many different outputs from the network. So it is needed to develop a methodology to evaluate the neural network results. Without the evaluation process and a feedback it is impossible to find and remove weak points in the application.
The trained neural network does not reach the level of an experienced human expert. The strength of the human expert lies in fact, that he is able to quickly filter appropriate data where searchable phenomena can be found and do not have to deal with noisy, hard recognizable cases. In addition, an expert can build on context knowledge and he can interpret phenomena that would have been hard to explain without additional information.
The power of the artificial neural network lies in its extraordinary performance. The neural network must view all submitted data, whether they are suitable for troubleshooting or not. Often the neural network finds a defect even in noisy or incomplete data that human expert would not even care about, because he would evaluate them as difficult to read.
But just like a human expert, the neural network can determine if the data packet can be used to look for a specific bug. The result of neural network work can be statistics of hundreds of thousands or millions of data samples. Process such amount of data is completely out of the human possibilities. And if the neural network can focus only on the good recognizable cases, it can successfully compete with best human expert.
Results differ for each pattern. If it is a very common pattern, which the neuron network has rich experience with, over 90 % of the occurences is found in the the control group. Less common patterns are successfuly found in over 80 %, however, the pattern are usualy easily interchangeable. Neural network sometimes identifies the pattern wrong, but it is still capable to find a bug in data in most cases.
Patterns that occur sporadically the network can recognize with slight difficulty. The success rate ranges from 60 to 80 %. But in hundreds of thousands of control samples the network mark a few tens of hundreds of cases which in the result leads at least to a suspicion of a detect. If several thousand datasets are available from one device, the statistical evaluation of the neural network outputs can identify a faulty device pretty successfully. And here the neural network shows its full power – the human expert is not able to view and evaluate such ammount of data. The defect would remain unnoticed without the neural network.
If only clear cases are accounted into statistics, the probability of defect detection increases sharply. With a little exaggeration I can say that in decent data the neural network finds double, ie even 180 % of defective devices (the corrector forced me to write a note that this is a joke – it is not possible to mark as a defective 180 of a hundred devices).
