UNIT-II

Informed Search Techniques

 Informed search
Informed search techniques are also called as heuristic search
techniques/strategies.
These strategies use the information about the domain or knowledge about the
problem and to move towards the goal nodes.
Informed search algorithm contains an array of knowledge such as how far we are
from the goal, path cost, how to reach to goal node, etc. This knowledge help
agents to explore less to the search space and find more efficiently the goal node.
Heuristic methods do not always find the best solution, but they guarantees to find
a good solution in reasonable amount of time.

[Link]
 Heuristic Search and Heuristics function
• Heuristic search attempts to optimize a problem using heuristic function
• Heuristic is a technique designed for solving a problem more quickly.
• Heuristic is the technique that improves the efficiency of a search process.
• Heuristic are like tour guides. In other words, the heuristic tells us approximately how far the state
is from the goal state.
• Using good heuristic we can get good solutions to hard problems, such as the travelling salesman.
• The purpose of a heuristic function is to guide the search process in the most profitable direction
by suggesting which path to follow first when more than one is available.
• Heuristic is a function which is used in Informed Search, and it finds the most promising path.
• It takes the current state of the agent as its input and produces the estimation of how close agent is
from the goal.
• The heuristic method, however, might not always give the best solution, but it guaranteed to find a
good solution in reasonable time.
• Heuristic function estimates how close a state is to the goal.
• It is represented by h(n), and it calculates the cost of an optimal path between the pair of states.
The value of the heuristic function is always positive.
 GENERATE AND TEST
Generate and Test Search is a heuristic search technique based on Depth First Search with
Backtracking which guarantees to find a solution if done systematically if there exists a
solution.
In this technique, all the solutions are generated and tested for the best solution.
It ensures that the best solution is checked against all possible generated
solutions.
The algorithm for the method is:
1. Generate a possible solution,
2. Test if the possible solution is the real one, i.e., compare with the goal states.
3. Check solution, if true, return the solution, else go to step 1.
Here complete solutions are generated before testing.
It is often called the British Museum method as it’s like looking for
an exhibit at random or finding an object in the British Museum
by wandering randomly.
This approach proceeds systematically and does not consider the paths
that are unlikely to lead to the solution.
Now, this evaluation of which path should be considered is done by
the heuristic function
Properties of Good Generators:
The good generators need to have the following properties
• Complete: Good Generators need to be complete i.e. they should generate all the
possible solutions and cover all the possible states. In this way, we can guaranty
our algorithm to converge to the correct solution at some point in time.
• Non Redundant: Good Generators should not yield a duplicate solution at any
point of time as it reduces the efficiency of algorithm thereby increasing the time
of search and making the time complexity exponential. In fact, it is often said that
if solutions appear several times in the depth-first search then it is better to modify
the procedure to traverse a graph rather than a tree.
• Informed: Good Generators have the knowledge about the search space which
they maintain in the form of an array of knowledge. This can be used to search
how far the agent is from the goal, calculate the path cost and even find a way to
reach the goal.
Let us take a simple example to understand the importance of a good generator.
Consider a pin made up of three 2 digit numbers i.e. the numbers are of the form,

In this case, one way to find the required pin is to generate all the solutions in a
brute force manner for example,
The total number of solutions in this case is (100)3 which is approximately 1M. So
if we do not make use of any informed search technique then it results in
exponential time complexity. Now let’s say if we generate 5 solutions every minute.
Then the total numbers generated in 1 hour are 5*60=300 and the total number of
solutions to be generated are 1M.
Now consider using heuristic function where we have domain knowledge that every
number is a prime number between 0-99 then the possible number of solutions are
(25)3 which is approximately 15,000. Now consider the same case that you are
generating 5 solutions every minute and working for 24 hrs then you can find the
solution in less than 2 days which was being done in 10 weeks in the case of
uninformed search.
• We can conclude for here that if we can find a good heuristic then time complexity
can be reduced gradually.
• But in the worst-case time and space complexity will be exponential. It all
depends on the generator i.e. better the generator lesser is the time complexity.
Advantages
• Easiest method
• Each and every solution is tested
• Less error chance

Drawbacks
• Problem space is very large
• More time required
• Complete solutions must be generated before they can be tested
 BEST FIRST SEARCH(BFS)
BFS is a combination of depth first search and breadth first search.
In DFS, the goal can be reached without any need to compute all the
states, whereas with BFS, it does not get halted or trapped in dead
paths.
The best first search allows us to switch between the paths.
It analyses and checks if the selected node is better than the previous
one.
If not found so, it reverts back to the previous path and proceeds. So,
backtracking occurs
In order to achieve this, even though a move is selected, other options
are kept so that they can be revisited.
 OR Graph
• At each step of the best first search process, we select the most promising of the nodes we
generated so far.
• This is done by applying an appropriate heuristic function to each of them.
• We then expand the chosen node by using the rules to generate its successor. If one of them is the
solution, then quit.
• If not, all these new nodes are added to the set of nodes generated so far. Again the most promising
node is selected and the process continues.
• If a solution is not found, branch will start look less promising than one of the top-level branches
that had been ignored.
• At that point, the now more promising, previously ignored branch will be explored. But the
• old branch is not forgotten, Its last node remains in the set of generated but unexpanded nodes.
• The search can return to it whenever all the other get bad enough that it is again the most
promising path.
 To have this type of graph search, we describe further two lists of
nodes
Open List : It consists of list of nodes that have been generated and on
whom the heuristic function has already been applied, but yet not
examined. We can map them to a priority queue. This queue consists of
the nodes/elements with highest priority. This is dependent on the
heuristic function that associates the cost with the node, and hence,
prioritizes them.
Closed List: It contains the nodes that have already been examined.
When a node is generated, then this is required to check that whether it
has already been generated
In this Heuristic and function evaluation procedures are f(n) and g(n).
f(n) is the function, defined as the sum of two elements, i.e., the cost of
reaching from the initial node to the current node, g(n) and the
additional cost of getting from the current node to the goal state. h(n)
So,f(n)= g(n)+h(n) defines the function.
 BFS Algorithm
Step 1: Place the starting node into the OPEN list.
Step 2: If the OPEN list is empty, Stop and return failure.
Step 3: Remove the node n, from the OPEN list which has the lowest value of
h(n), and places it in the CLOSED list.
Step 4: Expand the node n, and generate the successors of node n.
Step 5: Check each successor of node n, and find whether any node is a goal
node or not. If any successor node is goal node, then return success and
terminate the search, else proceed to Step 6.
Step 6: For each successor node, algorithm checks for evaluation function
f(n), and then check if the node has been in either OPEN or CLOSED list. If
the node has not been in both list, then add it to the OPEN list.
Step 7: Return to Step 2.
Example Description
The values on the edges represent the distances or the costs.
BFS finds a solution from initial state to goal state.
BFS uses heuristic, which is used to calculate the cost.
Consider hypothetical “h” values, which is used to calculate the cost.
Heuristic Values
Home 120
Bank 80
Garden 100
School 70
Railway station 20
Post office 110
Police station 26
 Initial State: A. From A three nodes are connected. I.e
B,C,D

Put B,C,D nodes on Open list according to

heuristic value. Now queue contain C,B,[Link]
Expand C Node.
C Node is connected to two nodes I,e E and F

By using Best First

Search We get good
solution. But may or
may not get optimal
solution.

Since F value is less next we explore that

Since G is Goal state, node.
Path: A->C->F->G
 GREEDY SEARCH
There is a debate whether the best first search and the greedy search
are the same.
So, we need to first understand the relationship among A*, best first
and greedy.
The BFS is said to be the simplified version of A*.
About greedy and best first, it is BFS that is named as greedy best first
search only.
 A*Search
• It is an advanced BFS algorithm that searches for shorter paths first rather than
the longer paths. A* is optimal as well as a complete algorithm.
• A* (pronounced "A-star") is a graph traversal and path search algorithm, which is
often used in many fields of computer science due to its completeness, optimality,
and optimal efficiency
• A* Algorithm is one of the best and popular techniques used for path finding and
graph traversals.
• A lot of games and web-based maps use this algorithm for finding the shortest
path efficiently.
• A* is the most widely used approach for pathfinding.
• Evaluation function f(n) represents the total cost.
• Here f(n) =g(n)+h(n),
where g(n) is the cost to reach node n from the start sate
while h(n) is the estimated cost from n to the goal state.
• The BFS is a special case of A*, where f(n)=h(n).
Algorithm:
• The implementation of A* Algorithm involves maintaining two lists- OPEN and
CLOSED.
• OPEN contains those nodes that have been evaluated by the heuristic function but have
not been expanded into successors yet.
• CLOSED contains those nodes that have already been visited.
The algorithm is as follows-
Step-1:
• Define a list OPEN.
• Initially, OPEN consists solely of a single node, the start node S.
Step2: If the list is empty, return failure and exit.
Step-3:
• Remove node n with the smallest value of f(n) from OPEN and move it to list CLOSED.
• If node n is a goal state, return success and exit.
Step-4: Expand node n.
Step-5:
• If any successor to n is the goal node, return success and the solution by tracing the path from goal
node to S.
• Otherwise, go to Step-06.
Step-6:
• For each successor node,
• Apply the evaluation function f to the node.
• If the node has not been in either list, add it to OPEN.
Step:7:Go back to Step-02.
Draw backs: A* is slow and also the space it requires is a lot as it saves all the possible paths that are
available to us.
Consider the following graph-

• The numbers written on edges represent the distance between the nodes.
• The numbers written on nodes represent the heuristic value.
Find the most cost-effective path to reach from start state A to final state J using A*
Algorithm.
Step-01:

We start with node A.

Node B and Node F can be reached from node A.

A* Algorithm calculates f(B) and f(F).

f(B) = 6 + 8 = 14
f(F) = 3 + 6 = 9

Since f(F) < f(B), so it decides to go to node F.

Path- A → F
Step-2
Node G and Node H can be reached from node F.
A* Algorithm calculates f(G) and f(H).
f(G) = (3+1) + 5 = 9
f(H) = (3+7) + 3 = 13
Since f(G) < f(H), so it decides to go to node G.
Path- A → F → G
Step-03:
Node I can be reached from node G.
A* Algorithm calculates f(I).
f(I) = (3+1+3) + 1 = 8
It decides to go to node I.
Path- A → F → G → I
Step-04:
Node E, Node H and Node J can be reached from node I.
A* Algorithm calculates f(E), f(H) and f(J).
f(E) = (3+1+3+5) + 3 = 15
f(H) = (3+1+3+2) + 3 = 12
f(J) = (3+1+3+3) + 0 = 10
Since f(J) is least, so it decides to go to node J.
Path- A → F → G → I → J
This is the required shortest path from node A to node J.

It is important to note that-

A* Algorithm is one of the best path finding algorithms.
But it does not produce the shortest path always.
This is because it heavily depends on heuristics.
Heuristic Values
Home 120
Bank 80
Garden 100
School 70
Railway station 20
Post office 110
Police station 26
AO* Algorithm
The problem is that the choice of which
node to expand next must depend not
only on the f l value of that node but also
on whether that node is part of the
current best path from the initial node.
Consider the following example:

In Figure 3.7(a) The top node A, has been expanded, producing two arcs, one leading to B and one leading to C and D.
The numbers at each node represent the value of f l at that node. We assume, for simplicity, that every operation has a
uniform cost, so each arc with a single successor has a cost of 1 and each AND arc with multiple successors has a cost of
1 for each of its components. If we look just at the nodes and choose for expansion the one with the lowest f1 value, we
must select C.

By using the information now available, it would be better to explore the path going through B since to use C we must
also use D, for a total cost of 9 (C+D+2) compared to cost of 6 that we get by going through B. The problem is that the
choice of which node to expand next must depend not only on the f l value of that node but also on whether that node is
part of the current best path from the initial node.

The tree shown in Figure 3.7(b) makes this even clearer. The most promising single node is G with an f 1 value of 3. It is
even part of the most promising arc G-H, with a total cost of 9. But that arc is not part of the current best path since to use
it we must also use the arc I-J, with a cost of 27. The path from A, through B, to F and F is better, with a total Cost of 18.
So we should not expand G next: rather we should examine either E or F.

In order to describe an algorithm for searching an AND-OR graph, we need to exploit a value that we call, FUTILITY. If
the estimated cost of solution becomes greater than value of FUTILITY, then we abandon the search.
 A* Vs AO*
[Link] are part of informed search technique and use heuristic values to
solve the problem.
[Link] solution is guaranteed in both algorithm.
3.A* always gives an optimal solution (shortest path with low cost)
But It is not guaranteed to that AO* always provide an optimal
solutions.
[Link]: Because AO* does not explore all the solution path once it
Algorithm: Problem Reduction
 Memory bounded heuristic search
Generally, we tend to run out of the memory before time!
This occurs when the entire search paths are stored and the only
solution to overcome this is to remember partial solutions.
Thus, in order to overcome the space requirements,
memory-bounded heuristic search comes into picture.
Iterative deepening A*(IDA*), Recursive best first
search(RBFS)algorithms that fall under this.
IDA*
• In simple iterative deepening, the search is restricted by the depth
limit, running it recursively. So, it tries to resolve the problem of BFS.
• IDA* resolves the problem of A*, where memory problem is
overcome, same time, optimality is also maintained.
• It doesn’t keep track of each visited node which helps in saving
memory consumption.
• In IDA*, at each iteration, DFS is applied.
• Costs are maintained i.e., f(n)=g(n)+h(n) of each and every node that
is generated.
• IDA* is optimal in terms of time and space.
• No need to manage open list and the closed list
 Algorithm
1. Set the Threshold value as the least value f(n) where f(n)=g(n)+h(n)
2. Check if the threshold value greater than or equal to child then
expand the node or else check for least f(n).
3. Continue it until you get a goal node.
4. All rejected values are added into F-value list

In A* algorithm stores all the child nodes when it is solving the

problem. Its space complexity is more. This problem can be overcome
using IDA* where we maintained F-value list.
• IDA* Problem.
Step-I
In the graph 14>8,but it is not
kept in f-value list.
Since we expanded ‘H’ node
->we got C and I
If we are unable to find
solution we will move to I and
J . At that time we will keep j
value in f-value list.
In the given graph we got the
goal state and hence there is
no need to keep j value in
f-value list.
Recursive Best First Search
It is the simplest recursive algorithm that performs the best first
search with a linear space.
Here is to remember the best path or best alternative and backtrack if
best first gets more expensive, i.e., when the cost exceeds that of
previously expanded node.
But, tracking of the costs is required simultaneously with the
backtracking.
This means f-value need to be maintained.
So, keeping track of f-cost/f-value of alternate paths, the search
process backtracks if current path becomes expensive
As the decision to backtrack is taken, the f-cost of the current node is
replaced with the best f-cost of its children
Example(RBFS)
• The following example shows the working of RBFS for the same
problem Home to University.
• Coming to optimality, RBFS is also optimal if the heuristic is
admissible.
• Evaluation of time complexity is very difficult, as it is dependent on
the switching between nodes and the heuristic.
• Space complexity is O(bd), where ‘b’ is branching factor and ‘d’ is
depth.
 Local search Algorithms and optimization problems
• Search algorithms seen so far are designed to explore search spaces systematically,
where the solution is a sequence of actions.
• Real World problems are more complex.
• When a goal is found, the path to that goal constitutes a solution to the problem. But
depending on the applications, the path may or may not matter,
• If the path does not matter/systematic search is not possible, then consider another class
of algorithms
• Local search algorithms are used when we care only about a solution but not the path to a
solution.
• Local Search=use single current state and move to neighboring states
• Local search is used in most of the models of AI to search for the optimal solution
according to the cost function of that model.
• Local search is used in linear regression, neural networks, telecommunication networks,
clustering models. Hill climbing, simulated annealing, tabu search are some of the local
search algorithms.
• Local search is based on local view.
 Parameters of local search techniques
There are two important parameters:
1. Selected initial solution
2. Stop criterion
Stop criterion defines the condition when the search phase is over and
best solution found is returned.
Example:
Consider an example of a car driver driving a car for the first time on a
road with a number of curves up the hill, with possibility of multiple
junctions and routes. let us assume he is driving on that road first time,
He can see only the road close to her but cannot see the top or even
beyond the next curve.
His decision about the route and speed depends on the local information
and slope.
There is only one way to check, based on the local information.
• The issue with the local search procedure is that no one can assure that
the best solution is found.
• Local search algorithms do not guarantee an optimal solution.
• In local search, larger the elements in the neighborhood, more is the
difficulty in exploring and better is the quality of local optimum.
• Three main entities for local search problem:
1. Search space
2. Neighborhood relations
3. Cost function
 Hill climbing search
• In hill climbing, the search begins with a random point in search
space.
• The basic idea of hill climbing is to always head towards a state which
is better than the current state.
• Always move towards neighbor with better score.
• Neighbor is a state that is reachable from the current state with any
possible legal action.
• In case, there is no neighbor better than the current state, then the
program stops.
• The algorithm generally moves up in the direction of increasing value
that is uphill.
• It breaks its “moving up loop” when it reaches a “peak” where no
neighbor has a higher value.
• It only looks for immediate neighbor of current state.
 Simple Hill Climbing Algorithm
1. Evaluate the initial state, if it is goal then return and quit.
2. Loop until a solution is found or there are no new operators left.
3. Select and apply new operator
4. Evaluate new state
If it is goal state, then quit
If it is better than current state then make it new current state
If it is not better than current then go to step 2.
 Problems of Hill Climbing
1. Local maximum: It is a state which is better than its neighboring state
however there exists a state which is better than it(global maximum).
This state is better because here the value of the objective function is
higher than its neighbors.
2. Plateua/flat local maximum : It is a flat region of state space where
neighboring states have the same value.
3. Ridge : It is region which is higher than its neighbours but itself has a
slope. It is a special kind of local maximum.
 Steepest-Ascent hill climbing:
• The steepest-Ascent algorithm is a variation of simple hill climbing algorithm. This algorithm examines all
the neighboring nodes of the current state and selects one neighbor node which is closest to the goal state.
This algorithm consumes more time as it searches for multiple neighbors
Algorithm for Steepest-Ascent hill climbing:
• Step 1: Evaluate the initial state, if it is goal state then return success and stop, else make current state as
initial state.
• Step 2: Loop until a solution is found or the current state does not change.
• Let SUCC be a state such that any successor of the current state will be better than it.
• For each operator that applies to the current state:
• Apply the new operator and generate a new state.
• Evaluate the new state.
• If it is goal state, then return it and quit, else compare it to the SUCC.
• If it is better than SUCC, then set new state as SUCC.
• If the SUCC is better than the current state, then set current state to SUCC.
• Step 5: Exit.
Simulated Annealing
• Hill climbing always moves uphill and there are no downhill moves.
• As a result, it can get stuck to the local minima, hence it is incomplete
• Simulated Annealing allows Downwards Steps.
• Simulated annealing is based on metallurgy concept of annealing.
• This is about cooling of material in heat bath.
• In mechanical term Annealing is a process of hardening a metal or
glass to a high temperature then cooling gradually, so this allows the
metal to reach a low-energy crystalline state.
• The same process is used in simulated annealing in which the
algorithm picks a random move, instead of picking the best move.
• If the random move improves the state, then it follows the same path.
Otherwise, the algorithm follows the path which has a probability of
less than 1 or it moves downhill and chooses another path.
Local Beam Search
• In hill climbing, we keep just a single node in the memory.
• This cripples the algorithm to a great extent.
• Beam search is a heuristic search algorithm.
• This algorithm explores the states or graph by expanding the most
promising nodes in a limited set.
• At any level, it expands identified best node.
• There can be more than one node identified at each level, say K.
• It is based on breadth first search to build search tree.
Algorithm
• Maintain K states and not just a single state.
• The search begins with K randomly generated states.
• At each iteration, all possible successors of K randomly generated states are
identified.
• If the goal state is found, then halt, else select K best of the successors.

• K number of best nodes are expanded at each level.

• Hence, K is the width of the beam.
• If B is the branching factor, then B*K no of nodes will be evaluated at each
depth.
• Out of that, only K nodes will be selected.
• Hill climbing is special case of local beam search, where K=1.
• In case of no restriction on K it becomes breadth first search.
Ex: Let us assume that we want to select best engineering students from country, the steps are as
follows.
1. First select K states from the country with the best engineering results.
2. Select K cities with the best results
3. Select K colleges with the best results
4. Select K courses with the best results
5. Select K divisions with the best results
6. Select K students who scored maximum marks
7. Select a student among them with the maximum marks.
Tabu Search
• Tabu search was created by Glover in 1986.
• The idea of Tabu search is very similar to the others, where when the
searches get stuck in suboptimal region, the non-improving moves are
tried to search the solution.
• Tracing and backtracking to previous nodes are prevented in tabu
search with the use of list called tabulist, which contains the recent
history of search.
• Hill Climbing when combined with short-term memory results into
simple tabu search.
Two basic elements of tabu search heuristic are:
1. Search space
2. Neighborhood structure
Tabu list with long-term memory maintains history all through the exploration process.
Short-term memory keeps the most recent ones.
 Adversarial Search methods(Game Theory)
Each agent needs to consider the action of other agent and effect of that action on their
performance. So, Searches in which two or more players with conflicting goals are trying to
explore the same search space for the solution, are called adversarial searches, often known as
Games.
Adversarial Search is search when there is an "enemy" or "opponent" changing the state of the problem every
step in a direction you do not want.
• Examples: Chess, business, trading, war.
You change state, but then you don't control the next state.
Opponent will change the next state in a way:
• Unpredictable
• You only get to change (say) every alternate state.
Minimax Algorithm:
It is a Backtracking Algorithm.: In this we will start from root and reach to terminal and we will calculate
values at terminal and propagate that values to root level and we will decide which move should be taken.
Best Move Strategy used
There are two player Max and Min.
• Max will try to maximize its value
• Min will try to minimize its value
• The Minimax algorithm tries to predict the opponent's behaviour.
• It predicts the opponent will take the worst action from our viewpoint.
• We are MAX - trying to maximise our score / move to best state.
Opponent is MIN - tries to minimise our score / move to worst state for us.
• Mini-max algorithm is a recursive or backtracking algorithm which is used in
decision-making and game theory. It provides an optimal move for the player assuming
that opponent is also playing optimally.
• Mini-Max algorithm uses recursion to search through the game-tree.
• Min-Max algorithm is mostly used for game playing in AI. Such as Chess, Checkers,
tic-tac-toe, go, and various tow-players game. This Algorithm computes the minimax
decision for the current state.
• In this algorithm two players play the game, one is called MAX and other is called MIN.
• Both the players fight it as the opponent player gets the minimum benefit while they get
the maximum benefit.
• Both Players of the game are opponent of each other, where MAX will select the
maximized value and MIN will select the minimized value.
• The minimax algorithm performs a depth-first search algorithm for the exploration of the
complete game tree.
• The minimax algorithm proceeds all the way down to the terminal node of the tree, then
backtrack the tree as the recursion.
Time complexity of Mini max is O(bd)where b is the Branching Factor and d is the
depth. In Minimax we are traversing every node at least once.

Ex: If no. of children are 3 and depth is 2 then time complexity will be 9,which is
feasible.
But in some games like chess on an avg 35 choices(moves) are possible, and depth
for one player on avg it may be [Link] two players(35)100

So the Game tree will be very large and it is difficult to traverse that tree.
So minimax algorithm is used in some games like tic-tac-toe
To improve the efficiency of Minimax alpha beta pruning was used.
 Alpha beta pruning
It is an advanced version of minimax search procedure.
• It is an optimization technique for the minimax algorithm.
• As we have seen in the minimax search algorithm that the number of game states it has to examine
are exponential in depth of the tree. Since we cannot eliminate the exponent, but we can cut it to
half. Hence there is a technique by which without checking each node of the game tree we can
compute the correct minimax decision, and this technique is called pruning. This involves two
threshold parameter Alpha and beta for future expansion, so it is called alpha-beta pruning. It is
also called as Alpha-Beta Algorithm.
• Alpha-beta pruning can be applied at any depth of a tree, and sometimes it not only prune the tree
leaves but also entire sub-tree.
The two-parameter can be defined as:
1. Alpha: The best (highest-value) choice we have found so far at any point along the path of
Maximizer. The initial value of alpha is -∞.
2. Beta: The best (lowest-value) choice we have found so far at any point along the path of
Minimizer. The initial value of beta is +∞.
The Alpha-beta pruning to a standard minimax algorithm returns the same move as the standard
algorithm does, but it removes all the nodes which are not really affecting the final decision but
making algorithm slow. Hence by pruning these nodes, it makes the algorithm fast.
Suppose we already find the best path remaining path we will prune(cut-off) we will not explore that
path.
Condition for Alpha-beta pruning:
• The Max player will only update the value of alpha.
• The Min player will only update the value of beta.
• While backtracking the tree, the node values will be passed to upper nodes instead of values of
alpha and beta.
• We will only pass the alpha, beta values to the child nodes.
Constraint Satisfaction problem
[Link]
INTELLIGENT AGENT
• An agent can be anything that perceives environment through sensors and act upon that
environment through actuators. An Agent runs in the cycle of perceiving, thinking, and acting. An
agent can be:
• Human-Agent: A human agent has eyes, ears, and other organs which work for sensors and hand,
legs, vocal tract work for actuators.
• Robotic Agent: A robotic agent can have cameras, infrared range finder, NLP for sensors and
various motors for actuators.
• Software Agent: Software agent can have keystrokes, file contents as sensory input and act on
those inputs and display output on the screen.

Figure depi’cts the general structure and working of an agent.

• An agent interacts with the environment through sensors and
actuators.
• A sensor allows the agent to sense environment and perceive the
present state.
• An actuator allows the agent to take actions with reference to the
environment perceived
Example (comparison of human being)
• Human being 🡪 Agent
• Senses environment🡪 eyes, nose, ears(sensory organs)
• Acts on the environment 🡪 hands, legs and other parts of the body
• Sensory organs🡪 sensors in case of human being
• Actuators 🡪 parts of the body which help to take action
What is an Intelligent Agent
• An intelligent agent is an autonomous entity which act upon an
environment using sensors and actuators for achieving goals. An
intelligent agent may learn from the environment to achieve their
goals. A thermostat is an example of an intelligent agent.
Following are the main four rules for an AI agent:
• Rule 1: An AI agent must have the ability to perceive the
environment.
• Rule 2: The observation must be used to make decisions.
• Rule 3: Decision should result in an action.
• Rule 4: The action taken by an AI agent must be a rational action.
Types of AI Agents
Agents can be grouped into five classes based on their degree of perceived
intelligence and capability.
• Simple Reflex Agent
• Model-based reflex agent
• Goal-based agents
• Utility-based agent
• Learning agent
Simple Reflex Agent
• Simple Reflex Agent works similar to our body’s reflex action (e.g. when we
immediately lift our finger when it touches the tip of the flame).
• Just as the prompt response of our body based on the current situation, the agent
also responds based on the current environment irrespective of the past state of the
environment.
• The reflex agent can work properly only if the decisions to be made are based on
the current percept.
Model-based reflex agent
These are the agents with memory. It stores the information about the previous state, the current state
and performs the action accordingly.
Just as while driving, if the driver wants to change the lane, he looks into the mirror to know the
present position of vehicles behind him.
While looking in front, he can only see the vehicles in front, and as he already has the information on
the position of vehicles behind him (from the mirror a moment ago), he can safely change the lane.
The previous and the current state get updated quickly for deciding the action.
Goal-based Agents
• In some circumstances, just the information of the current state may not help in making the right
decision. If the goal is known, then the agent takes into account the goal information besides the
current state information to make the right decision.
• For, e.g., if the agent is a self-driving car and the goal is the destination, then the information of the
route to the destination helps the car in deciding when to turn left or right.
• ‘Search’ and ‘planning’ are the two subfields of AI that help the agent achieve its goals. Though the
goal-based agent may appear less efficient, yet it is flexible.
• Considering the same example mentioned above, if the destination changes then the agent will
manipulate its actions accordingly. This will not be the case with the reflex agent as all the rules
need to be rewritten with the change in goal.
Utility Agents
• There can be many possible sequences to achieve the goal, but some will be better than others.
Considering the same example mentioned above, the destination is known, but there are multiple
routes.
• Choosing an appropriate route also matters to the overall success of the agent. There are many
factors in deciding the route like the shortest one, the comfortable one, etc. The success depends on
the utility of the agent-based on user preferences.
• The utility is a function that maps a state to a real number that describes the degree of happiness.
Learning Agents
• A learning agent in AI is the type of agent which can learn from its past experiences, or it
has learning capabilities.
• It starts to act with basic knowledge and then able to act and adapt automatically through
learning.
• A learning agent has mainly four conceptual components, which are:
• Learning element: It is responsible for making improvements by learning from
environment
• Critic: Learning element takes feedback from critic which describes that how well
the agent is doing with respect to a fixed performance standard.
• Performance element: It is responsible for selecting external action
• Problem generator: This component is responsible for suggesting actions that will
lead to new and informative experiences.
• Hence, learning agents are able to learn, analyze performance, and look for new ways to
improve the performance.