Neural Network Basic Concepts

Neural networks are computing systems that try to emulate brain functions. Neural networks are trained using real-world data, and once trained, they can perform faster than conventional methods. Neural networks can do classification, pattern recognition, approximation, clustering, prediction, and optimization. Neural networks are composed up of layers, and each layer contains neurons and nodes. These neurons and nodes make communication between different layers possible. Every node has a weight associated with it. This weight decides the importance of input. Every neuron has an internal state, also known as an activation function. The output is generated when the input signal is processed through the activation function.

Brain VS Neural Network

A human brain contains billions of neurons, and each neuron can have thousands of interconnections. A human neuron has the following components, a dendrite that receives information, a soma that processes information, an axon that works as a cable for sending and receiving information, and a synapse that connects the axon with the dendrite of another neuron. An artificial neural network tries to implement this brain process such that soma becomes a node, dendrites become input, synapse becomes weights, and axon becomes the output.

Artificial Neural Network

The functionality of an ANN depends on the topology, weights, and activation functions.

Network Topology

Topology is the organization of a network’s layers, nodes, and interconnections. There are two fundamental network topologies: a feedforward network and another is a feedback network. In the feedforward network, the information flows in the forward direction; the previous layer node is connected to the node of the next layer. The nodes assign weights to inputs from the last layer. A feedforward network can have a single layer or multiple layers. Feedback networks have loop connections that feed the information to the previous layers. They are not linear systems. The networks with closed loops are called recurrent networks. Similarly, if all nodes are connected, then the network becomes fully recurrent. Likewise, if the output is connected to the input, the network is called the Jordon network.

Weights Adjustment and Training

The process of weight adjustment is called training. Training has three main types, supervised practice, unsupervised training, and reinforcement training. The input vector is fed to the ANN in supervised training, and an output vector is generated. This output vector is validated by comparing it to the required output vector. If the two output vectors do not match, then the weights are adjusted until the desired results are achieved. In unsupervised training, the input vectors with identical characteristics are clustered together to form classes. Then, the neural network tries to assign classes when new vectors are fed as input. There is no redefined output vector for validation in unsupervised training, and the network learns all features and patterns on its own. In reinforcement training, the networks have some feedback. This feedback is used to improve the performance of ANN. In this type of network, the feedback is not used for validation; instead, it is used as a reference for weight adjustment.

Activation Functions

These are the processing functions. They compute inputs. There are numerous activation functions; one such function is the linear activation function. It multiplies the input with weights and creates an output. Another activation function is sigmoid. If its output is between zero and one, then it is called a binary sigmoid, and if its output is between the negative one and positive one, then it is called a bipolar sigmoid.

Other useful articles: