Local search algorithms
Incomputer science,local searchis
metaheuristicmethod for solving computationally
hardoptimizationproblems. Local search can be used
on problems that can be formulated as finding a
solution maximizing a criterion among a number
ofcandidate solutions. Local search algorithms move
from solution to solution in the space of candidate
solutions (thesearch space) by applying local
changes, until a solution deemed optimal is found or a
time bound is elapsed.
Local search algorithms
In many optimization problems, the
state space is the space of all
possible complete solutions
We have an objective function that
tells us how good a given state is,
and we want to find the solution
(goal) by minimizing or maximizing
the value of this function
Example: n-queens
problem
Put n queens on an n n board with no two
queens on the same row, column, or diagonal
State space: all possible n-queen configurations
Whats the objective function?
Number of pair-wise conflicts
Example: Traveling salesman
problem
Find the shortest tour connecting a given set of
cities
State space: all possible tours
Objective function: length of tour
Local search algorithms
In many optimization problems, the state space is
the space of all possible complete solutions
We have an objective function that tells us how
good a given state is, and we want to find the
solution (goal) by minimizing or maximizing the
value of this function
The start state may not be specified
The path to the goal doesnt matter
In such cases, we can that keep a single current
state anduse local search algorithms gradually try
to improve it
Example: n-queens
problem
Put n queens on an n n board with no two queens
on the same row, column, or diagonal
State space: all possible n-queen configurations
Objective function: number of pairwise conflicts
Whats a possible local improvement strategy?
Move one queen within its column to reduce conflicts
Example: Traveling Salesman
Problem
Find the shortest tour connecting n cities
State space: all possible tours
Objective function: length of tour
Whats a possible local improvement strategy?
Start with any complete tour, perform pairwise
exchanges
Hill-climbing search
Initialize current to starting state
Loop:
Let next = highest-valued successor of
current
If value(next) < value(current) return
current
Else let current = next
Variants: choose first better successor,
randomly choose among better successors
Like climbing mount Everest in thick fog with
amnesia
The state space
landscape
Iterative Improvement and Hill
Climbing
The main problem that hill climbing can encounter
is that of local maxima.
This occurs when the algorithm stops making
progress towards an optimal solution; mainly due
to the lack of immediate improvement in adjacent
states.
Local maxima can be avoided by a variety of
methods: Simulated annealing tackles this issue by
allowing some steps to be taken which decrease
the immediate optimality of the current state.
Random-Restart Hill-Climbing
Another way of solving the local
maxima problem involves repeated
explorations of the problem space.
Random-restart hill-climbing
conducts a series of hill-climbing
searches from randomly generated
initial states, running each until it
halts or makes no distinct progress.
Simulated annealing
search
Idea: escape local maxima by allowing
some "bad" moves but gradually decrease
their frequency
Probability of taking downhill move decreases
with number of iterations, steepness of downhill
move
Controlled by annealing schedule
Inspired by tempering of glass, metal
Simulated annealing
search
Initialize current to starting state
For i = 1 to
If T(i) = 0 return current
Let next = random successor of current
Let = value(next) value(current)
If > 0 then let current = next
Else let current = next with probability
exp(/T(i))
Effect of temperature
exp(/T)
Simulated annealing
search
One can prove: If temperature decreases slowly
enough, then simulated annealing search will find
a global optimum with probability approaching
one
However:
This usually takes impractically long
The more downhill steps you need to escape a local
optimum, the less likely you are to make all of them in a
row
More modern techniques: general family of
Markov Chain Monte Carlo (MCMC) algorithms for
exploring complicated state spaces
Local beam search
Start with k randomly generated states
At each iteration, all the successors of all k states
are generated
If any one is a goal state, stop; else select the k
best successors from the complete list and repeat
Is this the same as running k greedy searches in
parallel?
Greedy search
Beam search
Genetic algorithms
Variant of local beam search with sexual
recombination.
Genetic algorithms
Variant of local beam search with sexual
recombination.
Genetic algorithm
function GENETIC_ALGORITHM( population, FITNESS-FN) return an
individual
input: population, a set of individuals
FITNESS-FN, a function which determines the quality of the
individual
repeat
new_population empty set
loop for i from 1 to SIZE(population) do
x RANDOM_SELECTION(population, FITNESS_FN)
y RANDOM_SELECTION(population, FITNESS_FN)
child REPRODUCE(x,y)
if (small random probability) then child MUTATE(child )
add child to new_population
population new_population
until some individual is fit enough or enough time has elapsed
return the best individual
19
AI 1
Exploration problems
Until now all algorithms were offline.
Offline= solution is determined before executing it.
Online = interleaving computation and action
Online search is necessary for dynamic
and semi-dynamic environments
It is impossible to take into account all possible
contingencies.
Used for exploration problems:
Unknown states and actions.
e.g. any robot in a new environment, a newborn baby,
20
AI 1
Online search problems
Agent knowledge:
ACTION(s): list of allowed actions in state s
C(s,a,s): step-cost function (! After s is determined)
GOAL-TEST(s)
An agent can recognize previous states.
Actions are deterministic.
Access to admissible heuristic h(s)
e.g. manhattan distance
Online search problems
Objective: reach goal with minimal cost
Cost = total cost of travelled path
Competitive ratio=comparison of cost with cost of the
solution path if search space is known.
Can be infinite in case of the agent
accidentally reaches dead ends
The adversary argument
Assume an adversary who can construct the state
space while the agent explores it
Visited states S and A. What next?
Fails in one of the state spaces
No algorithm can avoid dead ends in all state
spaces.
Online search agents
The agent maintains a map of the
environment.
Updated based on percept input.
This map is used to decide next action.
Note difference with e.g. A*
An online version can only expand the node it is
physically in (local order)
Online DF-search
function ONLINE_DFS-AGENT(s) return an action
input: s, a percept identifying current state
static: result, a table indexed by action and state, initially empty
unexplored, a table that lists for each visited state, the action not yet tried
unbacktracked, a table that lists for each visited state, the backtrack not yet tried
s,a, the previous state and action, initially null
if GOAL-TEST(s) then return stop
if s is a new state then unexplored[s] ACTIONS(s)
if s is not null then do
result[a,s] s
add s to the front of unbackedtracked[s]
if unexplored[s] is empty then
if unbacktracked[s] is empty then return stop
else a an action b such that result[b, s]=POP(unbacktracked[s])
else a POP(unexplored[s])
s s
return a
Online DF-search, example
Assume maze problem
on 3x3 grid.
s = (1,1) is initial state
Result, unexplored (UX),
unbacktracked (UB),
are empty
S,a are also empty
Online DF-search, example
S=(1,1)
GOAL-TEST((,1,1))?
S not = G thus false
(1,1) a new state?
True
ACTION((1,1)) -> UX[(1,1)]
{RIGHT,UP}
s is null?
True (initially)
UX[(1,1)] empty?
False
POP(UX[(1,1)])->a
A=UP
AI 1
s = (1,1)
Return a
Online DF-search, example
S=(2,1)
GOAL-TEST((2,1))?
S not = G thus false
(2,1) a new state?
True
ACTION((2,1)) -> UX[(2,1)]
{DOWN}
s is null?
false (s=(1,1))
result[UP,(1,1)] <- (2,1)
UB[(2,1)]={(1,1)}
UX[(2,1)] empty?
False
A=DOWN, s=(2,1) return A
Online DF-search, example
S=(1,1)
GOAL-TEST((1,1))?
(1,1) a new state?
S not = G thus false
false
s is null?
false (s=(2,1))
result[DOWN,(2,1)] <- (1,1)
UB[(1,1)]={(2,1)}
UX[(1,1)] empty?
False
A=RIGHT, s=(1,1) return A
Online DF-search, example
S=(1,2)
GOAL-TEST((1,2))?
S not = G thus false
(1,2) a new state?
True,
UX[(1,2)]={RIGHT,UP,LEFT}
s is null?
false (s=(1,1))
result[RIGHT,(1,1)] <- (1,2)
UB[(1,2)]={(1,1)}
UX[(1,2)] empty?
False
A=LEFT, s=(1,2) return A
Online DF-search, example
S=(1,1)
GOAL-TEST((1,1))?
(1,1) a new state?
AI 1
6/11/15
True
UB[(1,1)] empty? False
A= b for b in result[b,
(1,1)]=(1,2)
false (s=(1,2))
result[LEFT,(1,2)] <- (1,1)
UB[(1,1)]={(1,2),(2,1)}
UX[(1,1)] empty?
false
s is null?
S not = G thus false
B=RIGHT
A=RIGHT, s=(1,1)
Online DF-search
Worst case each node is
visited twice.
An agent can go on a long
walk even when it is close to
the solution.
An online iterative
deepening approach solves
this problem.
Online DF-search works only
when actions are reversible.
Online local search
Hill-climbing is already online
One state is stored.
Bad performance due to local maxima
Random restarts impossible.
Solution: Random walk introduces exploration (can produce
exponentially many steps)
Online local search
Solution 2: Add memory to hill climber
Store current best estimate H(s) of cost to reach goal
H(s) is initially the heuristic estimate h(s)
Afterward updated with experience (see below)
Learning real-time A* (LRTA*)
