Artificial Neural Networks

1
▪ A neural network can be understood as a network of hidden layers, an input layer
and an output layer that tries to mimic the working of a human brain. The hidden
layers can be visualized as an abstract representation of the input data itself.
▪ These layers help the neural network understand various features of the data with
the help of its own internal logic.
▪ A neural network is a mathematical model that helps in processing information. It
is not a set of lines of code, but a model or a system that helps process the
inputs/information and gives result.
▪ The information is processed in the simplest form over basic elements known as
‘neurons’. Neurons are connected and help exchange signals/information between
them with the help of connection links.
▪ This connection links between neurons could be strong or weak, and this strength
of the connection links determines the method in which information is processed.
▪ Every neuron has an internal state which can be determined by the incoming
connections from other neurons.
▪ Every neuron has an activation function which is calculated on its state, and this
helps determine its output signal.

2
A neural network can be understood as a computational graph of
mathematical operations.
Two main characteristics of a neural network −
• Architecture
• Learning
Architecture:
It tells about the connection type: whether it is feedforward, recurrent, multi-
layered, convolutional, or single layered. It also tells about the number of
layers and the number of neurons in every layer.
Learning :
It tells about the method in which the neural network is trained. A common
way to train a neural network is to use gradient descent and backpropagation

3
• What is Artificial Neural Network?
The term "Artificial Neural Network" is derived from Biological neural networks
that develop the structure of a human brain. Similar to the human brain that
has neurons interconnected to one another, artificial neural networks also have
neurons that are interconnected to one another in various layers of the
networks. These neurons are known as nodes. The given figure illustrates the
typical diagram of Biological Neural Network.

4
5
• An Artificial Neural Network in the field of Artificial intelligence where it attempts to
mimic the network of neurons makes up a human brain so that computers will have an
option to understand things and make decisions in a human-like manner. The artificial
neural network is designed by programming computers to behave simply like
interconnected brain cells.
• There are around 1000 billion neurons in the human brain. Each neuron has an
association point somewhere in the range of 1,000 and 100,000. In the human brain,
data is stored in such a manner as to be distributed, and we can extract more than one
piece of this data when necessary from our memory parallelly. We can say that the
human brain is made up of incredibly amazing parallel processors.
• We can understand the artificial neural network with an example, consider an example
of a digital logic gate that takes an input and gives an output. "OR" gate, which takes
two inputs. If one or both the inputs are "On," then we get "On" in output. If both the
inputs are "Off," then we get "Off" in output. Here the output depends upon input. Our
brain does not perform the same task. The outputs to inputs relationship keep
changing because of the neurons in our brain, which are "learning."
6
The architecture of an artificial neural network:
To understand the concept of the architecture of an artificial neural network, we
have to understand what a neural network consists of. In order to define a neural
network that consists of a large number of artificial neurons, which are termed
units arranged in a sequence of layers. Lets us look at various types of layers
available in an artificial neural network. Artificial Neural Network primarily
consists of three layers:

7
• Input Layer:
As the name suggests, it accepts inputs in several different formats provided by the programmer.
• Hidden Layer:
The hidden layer presents in-between input and output layers. It performs all the calculations to
find hidden features and patterns.
Output Layer:
• The input goes through a series of transformations using the hidden layer, which finally results
in output that is conveyed using this layer.
• The artificial neural network takes input and computes the weighted sum of the inputs and
includes a bias. This computation is represented in the form of a transfer function.
• It determines weighted total is passed as an input to an activation function to produce the
output. Activation functions choose whether a node should fire or not. Only those who are
fired make it to the output layer. There are distinctive activation functions available that can be
applied upon the sort of task we are performing.

8
 Introduction to Activation Functions
• Activation functions are functions used in a neural network to compute the
weighted sum of inputs and biases, which is in turn used to decide whether a
neuron can be activated or not. It manipulates the presented data and produces
an output for the neural network that contains the parameters in the data. The
activation functions are also referred to as transfer functions in some literature.
These can either be linear or nonlinear.
• For a linear model, a linear mapping of an input function to output is performed
in the hidden layers before the final prediction for each label is given. The input
vector x transformation is given by
f(x) = wT . x + b
where, x = input, w = weight, and b = bias.
• Linear results are produced from the mappings of the above equation and the
need for the activation function arises here, first to convert these linear outputs
into non-linear output for further computation, and then to learn the patterns in
the data. The output of these models are given by
y = (w1 x1 + w2 x2 + … + wn xn + b) 9
• However, since the outputs are linear in nature, the nonlinear activation
functions are required to convert these linear inputs to non-linear outputs. These
transfer functions, applied to the outputs of the linear models to produce the
transformed non-linear outputs are ready for further processing. The non-linear
output after the application of the activation function is given by
y = α (w1 x1 + w2 x2 + … + wn xn + b)
where α is the activation function.

10
 Why Activation Functions?

The need for these activation functions includes converting the linear input
signals and models into non-linear output signals, which aids the learning of
high order polynomials for deeper networks.

11
How to use it?
• In a neural network every neuron will do two computations:
• Linear summation of inputs: In the above diagram, it has two inputs x1, x2 with
weights w1, w2, and bias b. And the linear sum
z = w1 x1 + w2 x2 + … + wn xn + b
• Activation computation: This computation decides, whether a neuron should be
activated or not, by calculating the weighted sum and further adding bias with it. The
purpose of the activation function is to introduce non-linearity into the output of a
neuron.
What is a Good Activation Function?
• A proper choice has to be made in choosing the activation function to improve the
results in neural network computing. All activation functions must be monotonic,
differentiable, and quickly converging with respect to the weights for optimization
purposes.

12
13