29.9.2017
I was given the task to feed our artificial intelligence with data. We use the AI in »Artificial Intelligence Photovoltaic Analyser«. Petr Bravenec wrote an article about neural network Programming with neural network from the developers view. Now I can offer you a look from the position of a whipped criminal used to train a neural network.
Preparing the data for training a neural network is incredibly tedious and time consuming job. It was told me that there are almost ten million charts available in our database. When I heard it, I almost fainted when I thought I'd have to go through all the charts. Fortunately, it was enough to view and label just over a hundred thousand charts. On the one hand, I was not alone, more than one hundred thousand charts were labeled by colleagues, but primarily the neural network was able to learn a lot of things from such a limited amount of data.
On each chart I searched for sixteen miscellaneous defects or features. On most charts, of course, nothing is bad, I just had to mark the weather (sunny, cloudy or foggy) and save it. But I could not to miss any malfunctions that may be seen in this picture:
I had to mark each malfunction and save it to the database.
Call: +420 777 566 384 or email to address info@hobrasoft.cz
As soon as our artificial intelligence has learned to recognize some sort of defects, we had to compare how neural network marked the defects and how the defects were marked by people. We often get into controversy because the neural network has seen non-existent defects or it has not found existing ones.
Many times happened that the neural network had a different opinion on the type of defect than I had - sometimes it can not be decided clearly what should be actually seen on the graph.
Additionally, we've made a lot of errors that need to be repaired to make the neural network not learning nonsense. Many graphs I had to go through again and again and mark them correctly.
When the controversial charts were corrected, we trained the network again. Our most powerful computer was working usually all the night. The next day I had to solve another portion of poorly labeled graphs.
Every graph has gone through my eyes maybe ten times. To the ten millions of graphs that I was afraid of at first, only a million seen pictures is still far away, but... it's just a lot of work.
Do you also imagine spiders when you see the asterisks?
