Topology of a neural network refers to the way the Neurons are connected, and it is an important factor in network functioning and learning. A common topology in unsupervised learning is a direct mapping of inputs to a collection of units that represents categories (e.g., Self-organizing maps). — Springer In our previous post we learned about the components that form a neural network: Input Nodes Neurons Output Nodes as well that those elements will be organized in the following layers: Input Layer Hidden Layers Output Layer However, it still not very clear what the process for organizing and connecting those neural networks is; how do we determine how many hidden layers we need to create? How do we connect those neurons to the inputs, outputs and to each other? How do we know how many output nodes we need to create? How to organize our Neural Network What we need is a way to represent how neurons are connected in order to form a network, a . The topology of a neural network plays an essential role in its function and performance. Neural Network Topology An important thing to notice is that can be used to learn the same set of data, and they are even likely to produce similar results; and as such there is not ‘best’ topology for a single neural network; that being said topology can greatly impact the amount of time required for a neural network to learn data as well its accuracy when classifying new data. multiple topologies One a approach to topology selection is to simply make it a trial and error process, where we manually tweak the Neural network by modifying things like the number of , and so on. Taking this approach would limit us greatly as there are many possible permutations even for a ; to complicate matters even further, Neural networks are not limited to the relatively simple layout we have seen before, to mention a few: hidden layers connections between nodes, simple neural network Feed-Forward Feed-forward Recurrent Long Short Term Memory Jordan Elman Hopfield So outside of very simple applications, known problems or learning exercises is not a viable option, in order to find the ideal topology we have to let our neural network to learn the topology from the training data as well. The goal is to find a topology for our neural network that * manual topology selection minimizes the the error on new data. * One of most novel approaches for this objective is the use of genetic programming. Genetic Programming and Neuro Evolution Genetic algorithms are based on the evolutionary principles of the , and ; a genetic algorithm will approach a particular problem by generating a large number of potential solutions and finding the solutions with the best . natural selection mutation survival of the fittest fitness score Without going too deep into the weeds, a is a method to measure how close was the solution to known result or to a expected metric; in the case of a neural network where we have training data that we can use as reference point. fitness score The Genetic algorithm will tweak and mutate our neural network generation after generation until the desired fitness score is achieved or until we reach the maximum number of iterations. Disclaimer: The above is a oversimplified description of how genetic algorithms work I highly recommend, for an amazing breakdown of the subject, even if you are not into Erlang the author makes a terrific job on breaking down the concepts and introduction genetic algorithms. The Handbook of Neuroevolution through Erlang Summary I’m important that we understand how topology, that is the of our neural networks can impact their effectiveness; there are many existing types of artificial neural networks some best suited to particular tasks of applications, and is important to familiarize ourselves with the most common types. number of layers and how they are connect Additional, in order to find the optimal topology one can use in order to try to evolve the topology for a given application. Genetic algorithms ‘perfect’ There is still a lot more to learn and go, but feel free to if you find any factual errors or if you think something . leave a comment needs more clarification This article was originally posted on my own site .
