[Link].

org
Seminar
On
Artificial Neural
Networks

Submitted To: Submitted

By:
[Link]
[Link]
 CONTENTS
 INTRODUCTION
 HISTORY
 BIOLOGICAL NEURON MODEL
 ARTIFICIAL NEURON MODEL
 ARTIFICIAL NEURAL NETWORK
 NEURAL NETWORK ARCHITECTURE
 LEARNING
 BACKPROPAGATION ALGORITHM
 APPLICATIONS
 ADVANTAGES
 CONCLUSION
 INTRODUCTION
 “Neural“ is an adjective for neuron, and “network” denotes a
graph like structure.
 Artificial Neural Networks are also referred to as “neural nets” ,
“artificial neural systems”, “parallel distributed processing
systems”, “connectionist systems”.
 For a computing systems to be called by these pretty names, it
is necessary for the system to have a labeled directed graph
structure where nodes performs some simple computations.
 “Directed Graph” consists of set of “nodes”(vertices) and a set
of “connections”(edges/links/arcs) connecting pair of nodes.
 A graph is said to be “labeled graph” if each connection is
associated with a label to identify some property of the
connection
CONTD… multiplier

x1ϵ{0,1}
x1
w1 (x1 w1)
x1 x2
(x2w2)
o = x1 AND x2 o = x1 AND x2
w2
x2ϵ{0,1} x2

Fig 2: AND gate network

The graph structure which connects the
Fig 1: AND gate graph weights modifiable using a learning
This graph cannot be considered a algorithm, qualifies the computing
neural network since the connections system to be called an artificial neural
between the nodes are fixed and networks.
appear to play no other role than
carrying the inputs to the node that
• The field of neural network was pioneered by BERNARD WIDROW of
computed their conjunction.
Stanford University in 1950’s.
 HISTORY
 late-1800's - Neural Networks appear as an analogy to
biological systems
 1960's and 70's – Simple neural networks appear
 Fall out of favor because the perceptron is not effective by itself,
and there were no good algorithms for multilayer nets
 1986 – Backpropagation algorithm appears
 Neural Networks have a resurgence in popularity
 FEATURES OF ARTIFICIAL NEURAL
NETWORKS
 Records (examples) need to be represented as a
(possibly large) set of tuples of <attribute, value>
 The output values can be represented as a discrete
value, a real value, or a vector of values
 Tolerant to noise in input data
 Time factor
 It takes long time for training
 Once trained, an ANN produces output values (predictions) fast
 It is hard for human to interpret the process of prediction
by ANN
 BIOLOGICAL NEURON MODEL

Four parts of a typical nerve

cell : -
 DENDRITES: Accepts the inputs
 SOMA : Process the inputs
 AXON : Turns the processed
inputs into outputs.
 SYNAPSES : The
electrochemical
contact between the
neurons.
 ARTIFICIAL NEURON MODEL
 Inputs to the network are x1
w1
represented by the
mathematical symbol, xn w2 f(w1 x1 + ……+ wnxn)
x2
 Each of these inputs are f

multiplied by a connection
weight , wn
wn
sum = w1 x1 + ……+
xn
wnxn
 These products are simply
summed, fed through the
transfer function, f( ) to
 TERMINOLOGY

Biological Terminology Artificial Neural Network Terminology

Neuron Node/Unit/Cell/Neurode

Synapse Connection/Edge/Link

Synaptic Efficiency Connection Strength/Weight

Firing frequency Node output

 ARTIFICIAL NEURAL NETWORK
 Artificial Neural Network (ANNs) are programs
designed to solve any problem by trying to mimic the
structure and the function of our nervous system.
 Neural networks are based on simulated neurons, Which
are joined together in a variety of ways to form networks.
 Neural network resembles the human brain in the
following two ways: -
* A neural network acquires knowledge through
learning.
*A neural network’s knowledge is stored within the
interconnection strengths known as synaptic
weight.
 CONTD…

ARTIFICIAL NEURAL NETWORK MODEL

Hidden layers

connections Desired
output
Neural network
Input Actual
Including output
output layer output
connections Comp
(called weights) are
between neuron
Input layer

Fig 1 : artificial neural network model Figure showing adjust of neural

network
 NEURAL NETWORK
ARCHITECTURES
Hidden node
Input node

Input node output node

Layer0 Layer3
(Input layer) (Output layer)
output node Layer 1 Layer2
Fig: fully connected Hidden Layer
network
fig: layered network
The neural network in which every node
is connected to every other nodes, and
these connections may be either These are networks in which nodes
excitatory (positive weights), inhibitory are partitioned into subsets called
(negative weights), or irrelevant (almost layers, with no connections from
zero weights). layer j to k if j > k.
 CONTD…

Layer0 Layer3
(Input layer) (Output layer)
Layer0 Layer3
Layer 1 Layer2
(Input layer) (Output layer)
Layer 1 Layer2
Hidden Layer

Hidden Layer
Fig : Acyclic network
fig : Feedforward network
This is the subclass of the layered
networks in which there is no intra- This is a subclass of acyclic
layer connections. In other words, a networks in which a
connection may exist between any connection is allowed from a
node in layer i and any node in layer j node in layer i only to nodes in
for i < j, but a connection is not layer i+1
allowed for i=j.
CONTD…

Fig : Modular neural network

Many problems are best solved using

neural networks whose architecture
consists of several modules, with
sparse interconnections between
them. Modules can be organized in
several different ways as Hierarchial
organization, Successive refinement,
Input modularity
 LEARNING
 Neurons in an animal’s brain are “hard wired”. It is
equally obvious that animals, especially higher
order animals, learn as they grow.
 How does this learning occur?
 What are possible mathematical models of learning?
 In artificial neural networks, learning refers to the
method of modifying the weights of connections
between the nodes of a specified network.
 The learning ability of a neural network is
determined by its architecture and by the
algorithmic method chosen for training.
 CONTD…

S U PE R VISE D LE A R N I N G
UNSUPERVISED
LEARNING
• A teacher is available to indicate
whether a system is performing
 This is learning by doing.
correctly, or to indicate the amount  In this approach no
of error in system performance. sample outputs are
Here a teacher is a set of training
provided to the network
data.
against which it can
• The training data consist of pairs of measure its predictive
input and desired output values
that are traditionally represented in performance for a given
data vectors. vector of inputs.
• Supervised learning can also be
 One common form of
referred as classification, where we unsupervised learning is
have a wide range of classifiers, clustering where we try
(Multilayer perceptron, k nearest to categorize data in
neighbor..etc)
different clusters by their
similarity.
 THE BACKPROPAGATION
ALGORITHM
 The backpropagation algorithm (Rumelhart and
McClelland, 1986) is used in layered feed-forward Artificial
Neural Networks.
 Back propagation is a multi-layer feed forward, supervised
learning network based on gradient descent learning rule.
 we provide the algorithm with examples of the inputs and
outputs we want the network to compute, and then the
error (difference between actual and expected results) is
calculated.
 The idea of the backpropagation algorithm is to reduce
this error, until the Artificial Neural Network learns the
training data.
 The activation function of the artificial neurons in
ANNs implementing the backpropagation
algorithm is a weighted sum (the sum of the inputs
xi multiplied by their respective weights wji) Inputs, x

output
 The most common output function is the sigmoidal
function:

 Since the error is the difference between the

actual and the desired output, the error depends Weights, v weights, w
on the weights, and we need to adjust the weights
in order to minimize the error. We can define the
Fig: Basic Block of
error function for the output of each neuron:
Back propagation neural network
CONTD…
 The backpropagation algorithm now calculates how the error depends on
the output, inputs, and weights.

the adjustment of each weight (Δwji ) will be the negative of a constant eta
(η) multiplied by the dependance of the “wji” previous weight on the error
of the network.
 First, we need to calculate how much the error depends on the output

 Next, how much the output depends on the activation, which in turn
depends on the weights

 And so, the adjustment to each weight will be

CONTD…

 If we want to adjust vik, the weights (let’s call them vik )

of a previous
layer, we need first to calculate how the error depends
not on the
weight, but in the input from the previous layer i.e.
replacing w by x Inputs, x
as shown in below equation. output

where
Weights, v weights, w
 and
 APPLICATIONS
 Neural Networks in Practice
 Neural networks in medicine
• Modelling and Diagnosing the Cardiovascular System
• Electronic noses
• Instant Physician
 Neural Networks in business
 Marketing
 Credit Evaluation
 ADVANTAGES
 It involves human like thinking.
 They handle noisy or missing data.
 They can work with large number of variables
or parameters.
 They provide general solutions with good
predictive accuracy.
 System has got property of continuous
learning.
 They deal with the non-linearity in the world in
which we live.
 CONCLUSION
• Artificial neural networks are inspired by the learning processes
that take place in biological systems.
• Artificial neurons and neural networks try to imitate the
working mechanisms of their biological counterparts.
• Learning can be perceived as an optimisation process.
• Biological neural learning happens by the modification of the
synaptic strength. Artificial neural networks learn in the same
way.
• The synapse strength modification rules for artificial neural
networks can be derived by applying mathematical optimisation
methods.
 CONT…
• Learning tasks of artificial neural networks can be reformulated as
function approximation tasks.
• Neural networks can be considered as nonlinear function
approximating tools (i.e., linear combinations of nonlinear basis
functions), where the parameters of the networks should be found
by applying optimisation methods.
• The optimisation is done with respect to the approximation error
measure.
• In general it is enough to have a single hidden layer neural network
(MLP, RBF or other) to learn the approximation of a nonlinear
function. In such cases general optimisation can be applied to find
the change rules for the synaptic weights.
 REFERENCE

 [Link]
 [Link]
 [Link]
Thanks