Definition - Segmentation

 Partitioning of an image into connected homogenous

regions.
 Homogeneity may be defined in terms of:

n
ay
 Gray value

hu
.B
.K
M
 Colour
 Texture
 Shape
 Motion

3
Segmentation Example

n
ay
hu
.B
.K
M

4
 Segmentation
Mathematically, the segmentation problem is to partition
the image I into regions R1, R2,.....RN Such that

1. I = R  R  .....  R
1 2 N

n
ay
hu
.B
2. R  R = , i  j
.K
M
i j
3. There exists predicate P such that P( Ri ) = TRUE and for adjacent i, j
P ( Ri  R j ) = False

5
Segmentation Approaches

 Thresholding
 Region based
◼ Region Growing

n
ay
hu
◼ Split and Merge

.B
.K
M
 Edge based
◼ User-steered
◼ Active contour
 Topology Based
◼ Watershed
❑ K–means clustering
6
Thresholding
 Assumption
◼ Regions are distinct in terms of gray level range

n
ay
hu
.B
.K
M
Frequency

Gray level

7
Thresholding

 Suppose that an image, f(x,y), is composed of light objects

on a dark background, and the following figure is the
histogram of the image.

n
ay
hu
.B
.K
M

 Then, the objects can be extracted by comparing pixel

values with a threshold T.

8
Thresholding Example

n
ay
hu
.B
.K
M

9
Thresholding

n
ay
hu
.B
.K
M

10
Thresholding

n
ay
hu
.B
.K
M

11
Thresholding

 Non-uniform illumination may change the histogram in a

way that it becomes impossible to segment the image
using a single global threshold.
 Choosing local threshold values may help.

n
ay
hu
.B
.K
M

12
Thresholding

 It is also possible to extract objects that have a specific intensity

range using multiple thresholds.

n
ay
hu
.B
.K
M

Extension to color images is straightforward: There are three color channels, in

each one specify the intensity range of the object… Even if objects are not
separated in a single channel, they might be with all the channels… Application
example: Detecting/Tracking faces based on skin color…

13
Optimal Thresholding
 Approximate the Histogram by weighted sum of
two Gaussians
 The Valley portion represents the threshold

n
ay
hu
.B
.K
M

14
Region Growing

 A region satisfies certain homogeneity measure

 The user is required to select a point within a

n
ay
hu
feature

.B
.K
M
 the region is expanded , adding pixels that are
similar in brightness or colour

15
Region Growing

Given the image and seed points S1, S2, S3 …..,

predicate will be in terms of the seed points,
example, the difference in intensity with respect to a
seed point is less than 5.

n
ay
hu
➢Starting-signal point regions consisting of the

.B
.K
M
seed points.
➢Add the seed points to the pixels for which the
predicate is true.
➢Repeat until all the pixels are segmented.

16
Region Growing

 Pros
◼ Simple
◼ Adaptive to gradual changes/Some noise

n
ay
hu
.B
.K
M
 Cons
◼ Choosing of initial seed points.
◼ Initial seed points may be obtained by human
interpretation/intervention
◼ Seed point selection from the modes of the histogram.

17
One solution of selecting the seed point is the modes of
the histogram. These modes will define seed points.

n
ay
hu
.B
.K
M

18
Region Growing example

n
ay
hu
.B
.K
M

(a) Original Image (b) Region Growing (c) Segmented region

results

19
 M
.K
.B
hu
ay
n

20
Region Splitting

❑ The opposite approach to region growing is region

shrinking ( splitting ).

n
❑ It is a top-down approach and it starts with the

ay
hu
.B
.K
M
assumption that the entire image is homogeneous
❑ If this is not true , the image is split into four sub images
❑This splitting procedure is repeated recursively until we
split the image into homogeneous regions

21
Quadtree

R0

n
ay
hu
.B
.K
M
R1

R2 R4
R3

R11 R12 R13 R14

22
 Split and Merge

❑ If a region R is non homogeneous i.e. P( R) = FALSE

then is split into four sub regions

n
ay
❑ Merge two adjacent regions Ri, Rj if they are

hu
.B
.K
M
homogeneous i.e. P( Ri  R j ) = TRUE

❑ Stop when no further split or merge is possible

23
 n
ay
hu
.B
.K
M

24
[Link]
 M
.K
.B
hu
ay
n

25
 M
.K
.B
hu
ay
n

26
 M
.K
.B
hu
ay
n

27
 Watershed
 Image is modelled as a Topographic surface
◼ Catchment Basin
◼ Minima
◼ Watershed
◼ Dam Construction

n
ay
hu
.B
.K
M

38
Watershed

n
ay
hu
.B
.K
M

Segmentation is performed by labeling connected

components-catchment basins-within an image
39
Watershed Segmentation Algorithm

 Visualize an image in 3D: spatial coordinates and gray levels.

 In such a topographic interpretation, there are 3 types of points:
◼ Points belonging to a regional minimum
◼ Points at which a drop of water would fall to a single minimum.
(→The catchment basin or watershed of that minimum.)

n
ay
hu
◼ Points at which a drop of water would be equally likely to fall to

.B
.K
more than one minimum. (→The divide lines or watershed lines.)

M
Watershed lines

40
Watershed Segmentation Algorithm

 The objective is to find watershed lines.

 The idea is simple:
◼ Suppose that a hole is punched in each regional minimum and that the
entire topography is flooded from below by letting water rise through the
holes at a uniform rate.
◼ When rising water in distinct catchment basins is about the merge, a dam

n
is built to prevent merging. These dam boundaries correspond to the

ay
hu
watershed lines.

.B
.K
M

41
 M
.K
.B
hu
ay
n

42
Watershed Segmentation Algorithm

n
ay
hu
.B
.K
M

43
 Watershed Segmentation Algorithm

◼ Start with all pixels with the lowest possible value.

❑ These form the basis for initial watersheds
◼ For each intensity level k:

n
ay
hu
.B
❑ For each group of pixels of intensity k

.K
M
◼ If adjacent to exactly one existing region, add these pixels to that
region
◼ Else if adjacent to more than one existing regions, mark as
boundary
◼ Else start a new region

44
Watershed

 Pros
◼ Closed boundary
◼ Correct boundary achievable

n
ay
hu
.B
.K
M
 Cons
◼ Difficulties in correct/efficient implementation of
watershed
◼ Oversegmentation Problem
◼ Boundary May not be smooth

45
Watershed

 Over-segmentation
◼ Noise
◼ Small Catchment Basins

n
ay
hu
.B
.K
M
 Solution Approaches
◼ Filtering
◼ Weak edge removal

46
 Image Segmentation
• Image analysis: techniques for extracting information from
an image
• The first step in image analysis is to segment the image.
• Segmentation: subdivides the image into its constituent parts
or objects.
• Autonomous segmentation is one of the most difficult tasks
in image processing.
• Segmentation algorithms for monochrome images generally
are based on two basic properties of gray-level values:
1. Discontinuity
2. Similarity

2
 Image Segmentation
• Discontinuity: the image is partitioned based on abrupt
changes in gray level
• Similarity: partition an image into regions that are similar
– Main approaches are thresholding, region growing and region splitting
and merging.

3
 Detection of Discontinuities
• There are 3 basic types of discontinuities in digital images:
1. Point
2. Line
3. Edges.

• Point detection: detect an isolated point

4
Point Detection

5
 Background
• Abrupt, local changes can be detected by spatial
differentiation
• Since images are digital we should use digital differentiation
operators.
• First and second order derivatives are commonly used.
• We consider 1-D case first and then expand the results to
images.

6
7
 Background
• Comparing first and second order derivatives:
1. First-order derivatives generally produce thicker edges in an
image
2. Second order derivatives have a stronger response to fine
details such as thin lines and isolated points
3. Second order derivates produce a double response at step
changes in gray level

8
 Laplacian
• How to obtain 2-D second order derivative and find a mask
corresponding to it?
• We would like our filter to be isotropic: response of the filter
is independent of the direction of the discontinuity in the
image
• Simplest isotropic second order derivative is the Laplacian:

9
Line Detection

10
 Line Detection
• Line detection: If each mask is moved around an image it
would respond more strongly to lines in the mentioned
direction.

11
12
 Edge detection
• Edge: boundary between two regions with relatively distinct
gray levels.
• Basic idea: computation of a local derivative operator.

13
 Edge detection
• The magnitude of the first derivative can be used to detect an
edge
• The sign (zero crossing) of the second derivative can be used
to detect an edge.
• The same idea can be extended into 2-D. 2-D derivatives
should be used.
• The magnitude of the gradient and sign of the Laplacian are
used.

14
15
 Gradient Operator

• Gradient vector points in the direction of maximum rate of

change of f at (x,y).

Direction of the gradient vector

16
 Gradient Operator
• If we compute the magnitude of gradient of an image and
display it, it will be the edges in the image.
• Direction of edge at every point is perpendicular to the
direction of the gradient vector (given by α (x,y)).
• How to approximate the gradient operator?

17
18
19
20
21
22
23
 Edge linking and Boundary Detection
• Alternative way: Hough transform
• (xi,yi): all the lines passing this point yi=axi+b
• b= yi-axi : is a single line in ab plane.
• Another point (xj,yj) also has a single line in ab plane b= yj-axj
• a’ and b’ are the slope and intercept of the line containing both (xi,yi) and
(xj,yj).
b’
(xi,yi) y b
a’
(xj,yj)

x a

43
 Hough transform
• (amin,amax): expected range of slope
• (bmin,bmax): expected range of intercepts
• A(i,j): the number of points in the cell at coordinates (i,j) in
the ab plane.
bmin 0 bmax
For every point in the image a
plane, we let the value of a min b
equal each of the allowed
subdivisions and find the 0
corresponding b from b= yi-axi
If for ap we get bq the A(p,q) a
max
is incremented.
a

44
 Thresholding
• Thresholding is one of the most important approaches to
image segmentation.
• Gray level histogram of an image f(x,y) composed of a
light object on a dark background.
• To extract the object: select a threshold T that separates
the gray levels of the background and the object.
• Any point with f(x,y)>T belongs to object.

50
Thresholding

51
52
 Thresholding
• Threshold in general can be calculated as:
T=T(x,y,p(x,y),f(x,y))
f(x,y): gray level at (x,y)
p(x,y): some local property of the point (x,y) (e.g., the average gray level
of a neighborhood centered on (x,y).

T depends only on f(x,y): global threshold

T depends on f(x,y) and p(x,y): local threshold
T depends on f(x,y) and p(x,y) and x,y : dynamic threshold

53
 Thresholding
• Poor or non-uniform illumination can affect the image formed
(because of f(x,y)=r(x,y).i(x,y)) and make the segmentation
difficult.
• Solution: if we can determine i(x,y) then we can compensate f
for the effects of i.
• Technique: Project the illumination source onto a constant,
white reflective surface. The image obtained is
g(x,y)=K.i(x,y) (K is a constant).
• Divide f(x,y) by g(x,y): h(x,y)=f(x,y)/g(x,y)=r(x,y)/K
• If r(x,y) can be segmented by thresholding, then h(x,y) can
also be segmented by thresholding.

54
 Thresholding
• How to find a proper value for T?
1. Select an initial estimate for T
2. Segment the image using T. This will produce two group of
pixels G1 (with gray level greater than T) and G2 (with gray
level less than T)
3. Compute average gray level values µ1 and µ2 for pixels in
regions G1 and G2
4. Compute a new threshold: T=(µ1+ µ2)/2
5. Repeat step 2 through 4 until the difference in T in
successive iterations is smaller than a predefined value

55
 Region Based Segmentation
• Let R represent the entire image. Segmentation is a process
that partitions R into n subregions R1,R2,…,Rn such that:

59
 Region growing by Pixel Aggregation
• Pixel aggregation: starts with a set of seed point and from
these grows regions by appending to each seed point those
neighboring pixels that have similar properties (e.g., gray-
level, texture, color).

0 0 5 6 7 A A B B B
1 1 5 8 7 A A B B B
0 1 6 7 7 A A B B B
2 0 7 6 6 A A B B B
0 1 5 6 5 A A B B B
T=3

60
 Region growing by Pixel Aggregation
• Problems:
1. Selection of initial seeds that properly represent regions of interest
2. Selection of suitable properties
3. Formulation of a stopping rule.

61
 Region Splitting and Merging
• Sub-divide an image into a set of disjoint regions and then
merge and/or split the regions in an attempt to satisfy the
condition (P).
• Procedure:
1. Split into 4 disjoint quadrants any region Ri where
P(Ri)= False
2. Merge any adjacent regions Rj and Rk for which P(Rj U Rk)=
True.
3. Stop when no further merging or splitting is possible.

62
Region Splitting and Merging

63
Region Splitting and Merging

63