Hey guys, I hope you're okay. As you know, I am a programmer of Machine Learning models and I wanted to share with you some fundamental concepts of one of the most used models, the Artifical Neural Networks (ANN). Concretely, we will see in this post what is an ANN, how they are classified, in what consists the learning algorithm of the Backpropagation and finally the advantages that offer us the ANN.
What is an ANN?
Basically an ANN is a paradigm of learning and automatic processing inspired by the functioning of the human brain. An artificial neural network is composed of a set of neurons interconnected by links. Each neuron is essentially a simple model with inputs resembling dendrites, weights resembling synaptic connections and an activation function that gives an output resembling the axon of the human neuron. The following figure shows the model of an artificial neuron:
In an ANN, each neuron takes as its input the outputs of the neurons from the predecessor layers. Each of these inputs is multiplied by weight, then partial results are added and the output for each neuron is calculated using the activation function. This output is in turn the input of the neurons it precedes. In this way, the union of all these interconnected neurons is what makes up the artificial neural network, as shown in the following figure:
Classification of the ANNs.
Basically we can classify the ANN according to their topology and learning method:
- Classification according to topology:
- Single-layer Neuronal Network ( Perceptron).
- Multilayer Neural Network (Multilayer Perceptron).
- Convolutional Neural Network (CNN).
- Recurrent Neural Network (RNN).
- Radial Base Networks (RBF).
- Classification according to learning method:
- Supervised learning.
- Unsupervised learning.
Learning process of an ANN.
ANNs, like biological networks, learn by repetition; this means that the more data we have to train them and the more times we train them, the neural networks will give us better results. There are several learning algorithms for ANNs according to their classification, the best known of all is the Backpropagation Algorithm (BA) for multilayer neural networks.
In the post How Does Neural Network Backpropagation Work?, its author @kasperfred explains us each one of the mathematical formulas that constitute the BA. Similarly, @cristi in his post Backpropagation Step by Step - The Mathematical Approach gives us a brief explanation of the BA by telling us:
(...) backpropagation simply means back propagating the error you get as you reach the output of a network, with the purpose of updating the weights to minimize this error. Matt details both the forward pass and the backward pass through a neural network, with the equations and derivatives that they imply.
As we can see, the learning process of an ANN depends on its topology and has a mathematical algorithmic design, based fundamentally on the correction of output errors.
Advantages of ANNs.
- Adaptive Learning: ability to learn to perform tasks based on previous experiences.
- Self-organization: ability to create their own representation of the information they receive through a learning stage.
- Tolerance to failures: ability to continue to produce acceptable responses even to partial damage to their structure, this is because they stores redundant information.
- Real-time operation: when they are implemented on parallel architectures in hardware or software.
- Flexibility: ability to handle unimportant changes in input information, such as noisy signals or other input changes.
Conclusions.
ANNs are logical models of learning and processing that attempt to emulate the structure and functioning of the human brain. ANNs consist of simple processors with inputs, weights, and activation functions, called neurons. ANNs are classified according to their topology and training methods, the most popular method being the Backpropagation Algorithm. ANNs offer us multiple advantages, especially in terms of adaptive learning and flexibility.