Digital Image Processing

Image Enhancement
 Histogram Processing: definition of image histogram

• Consider an image with intensity 𝑟𝑘, 𝑘 ∈ [0, 𝐿 − 1] and size 𝑀 × 𝑁.

• The number of pixels with intensity 𝑟𝑘 is 𝑛𝑘.
• The histogram of the image is the function ℎ 𝑟𝑘 = 𝑛𝑘.
• The normalized histogram is the function
𝑛𝑘
𝑝 𝑟𝑘 = for 𝑘 = 0, … , 𝐿 − 1
𝑀𝑁

dark low
image contrast
image

bright high
image contrast
image
 Generic figures of histograms

• The appearance of
histogram reveals a lot of
information about
the contrast of the image and
the mean gray level.

• An image of low contrast has

a histogram that is concentrated
around a small range of
intensities.

• Images of high contrast are

more interesting and pleasant
for the human eye.
 Two different images with the same histogram

• The appearance of
histogram reveals a lot of
information about
the contrast of the image and
the mean gray level BUT :
• It doesn’t give any information
regarding the location or the
type of objects present
in the image. This information
is important because it is related
to the image content.
• Two images can have identical
histograms and still be
completely different in terms of
content.
 Histogram Processing: definition of intensity transformation

• Consider for the moment continuous intensity values 𝑟 ∈ [0, 𝐿 − 1].

• The value 𝑟 = 0 represents black and the value 𝑟 = 𝐿 − 1 represents
white.
• We are looking for intensity transformations of the form:
𝑠 = 𝑇 𝑟 , 0 ≤ 𝑟 ≤ 𝐿 − 1.
• The following conditions are imposed on 𝑇 𝑟 :
o 𝑇(𝑟) is monotonically increasing in 0 ≤ 𝑟 ≤ 𝐿 − 1 or strictly
monotonically increasing in 0 ≤ 𝑟 ≤ 𝐿 − 1.
➢ The above condition guarantees that ordering of the output intensity
values will follow the ordering of the input intensity values (avoids
reversal of intensities).
➢ If 𝑇(𝑟) is strictly monotonically increasing then the mapping from 𝑠
back to 𝑟 will be 1-1.
o 0 ≤ 𝑇(𝑟) ≤ 𝐿 − 1 for 0 ≤ 𝑟 ≤ 𝐿 − 1.
This condition guarantees that the range of intensities of the output will
be the same as the range of in the input.
 Monotonicity versus strict monotonicity

• In the above Figure on the left we cannot perform inverse mapping

(from 𝑠 to 𝑟).
• In the above Figure on the right inverse mapping is possible.
 Modelling intensities as continuous variables
• Assume that an original intensity 𝑟 is mapped to an intensity 𝑠 through the
transformation 𝑠 = 𝑇(𝑟).
• We can view intensities 𝑟 and 𝑠 as random variables.
• Instead of histograms we use probability density functions (pdf) 𝑝𝑟(𝑟) and
𝑝𝑠(𝑠).
• Consider a minimal increment of the original intensity 𝑟 to the intensity 𝑟 +
𝑑𝑟.
• The intensity 𝑟 + 𝑑𝑟 is mapped to an intensity 𝑠 + 𝑑𝑠 through the
transformation 𝑇(𝑟).
• Since 𝑇(𝑟) is (monotonically) increasing we can easily say that 𝑠 + 𝑑𝑠 ≥ 𝑠.
• All values of the original intensity which are within the interval [𝑟, 𝑟 + 𝑑𝑟]
will be mapped to new values within the interval [𝑠, 𝑠 + 𝑑𝑠].
• We can say that:
Probability(𝑟 ≤ original intensity ≤ 𝑟 + 𝑑𝑟)=Probability(𝑠 ≤ new intensity ≤
𝑠 + 𝑑𝑠) or in mathematical terms:
𝑟+𝑑𝑟 𝑠+𝑑𝑠
‫𝑟׬‬ 𝑝𝑟 𝑤 𝑑𝑤 = ‫𝑠׬‬ 𝑝𝑠 𝑤 𝑑𝑤
 Modelling intensities as continuous variables cont.

• Probability(𝑟 ≤ original intensity ≤ 𝑟 + 𝑑𝑟)=Probability(𝑠 ≤ new intensity ≤

𝑠 + 𝑑𝑠) or in mathematical terms:
𝑟+𝑑𝑟 𝑠+𝑑𝑠
‫𝑟׬‬ 𝑝𝑟 𝑤 𝑑𝑤 = ‫𝑠׬‬ 𝑝𝑠 𝑤 𝑑𝑤
• If 𝑑𝑟 is small enough we can assume that 𝑝𝑟 𝑟 remains almost constant
within the interval [𝑟, 𝑟 + 𝑑𝑟] and equal to 𝑝𝑟 𝑟 .
• We can choose 𝑑𝑟 to be as small as to be able to assume that 𝑑𝑠 is small
enough and that 𝑝𝑠 𝑠 remains almost constant within the interval [𝑠, 𝑠 +
𝑑𝑠] and equal to 𝑝𝑠 𝑠 .
𝑟+𝑑𝑟 𝑠+𝑑𝑠
• ‫𝑟׬‬ 𝑝𝑟 𝑤 𝑑𝑤 = ‫𝑠׬‬ 𝑝𝑠 𝑤 𝑑𝑤 ⇒ 𝑝𝑟 𝑟 𝑑𝑟 = 𝑝𝑠 𝑠 𝑑𝑠
• The above analysis is depicted in the Figure of the next slide.
Modelling intensities as continuous variables
 Histogram Equalization: continuous form

• A transformation of particular importance in Image Processing is the

cumulative distribution function (CDF) of a random variable.
𝑟
𝑠 = 𝑇 𝑟 = (𝐿 − 1) න 𝑝𝑟 𝑤 𝑑𝑤
0
• It is an increasing function since for 𝑟2 ≥ 𝑟1 we see that:
𝑟2
𝑠2 = 𝑇 𝑟2 = 𝐿 − 1 ‫׬‬0 𝑝𝑟 𝑤 𝑑𝑤
𝑟 𝑟
= 𝐿 − 1 ‫׬‬0 1 𝑝𝑟 𝑤 𝑑𝑤 + 𝐿 − 1 ‫ 𝑟׬‬2 𝑝𝑟 𝑤 𝑑𝑤
1
𝑟
= 𝑇 𝑟1 + 𝐿 − 1 ‫ 𝑟׬‬2 𝑝𝑟 𝑤 𝑑𝑤 ≥ 𝑇 𝑟1
1
 Histogram Equalization: continuous form

• We showed previously that 𝑝𝑟 𝑟 𝑑𝑟 = 𝑝𝑠 𝑠 𝑑𝑠

𝑝 𝑟
• Therefore, 𝑝𝑠 𝑠 = 𝑟𝑑𝑠 .
𝑑𝑟
• From the definition 𝑟
𝑠 = 𝑇 𝑟 = (𝐿 − 1) න 𝑝𝑟 𝑤 𝑑𝑤
0
we have that:
𝑑𝑠
= 𝑑𝑇(𝑟) = (𝐿 − 1) 𝑑 𝑟
‫׬‬0𝑝 𝑟 𝑤 𝑑𝑤 = 𝐿 − 1 𝑝𝑟 𝑟
𝑑𝑟 𝑑𝑟 𝑑𝑟
𝑝 𝑟 𝑝𝑟 𝑟 1
Hence, 𝑝𝑠 𝑠 = 𝑟𝑑𝑠 = = , 𝑠 = 1, … , 𝐿 − 1.
𝐿−1 𝑝 𝑟 𝑟 (𝐿−1)
𝑑𝑟
Therefore, the new variable 𝑠 is uniformly distributed.
 Histogram Equalization: discrete case

• The formula for histogram equalisation in the discrete case is given by a

straightforward modification of the formula that corresponds to the
continuous-time case.
• Instead of probability density functions (pdf) 𝑝𝑟(𝑟) and 𝑝𝑠(𝑠) we now use
histograms.
• The discrete input intensity 𝑟𝑘 is mapped onto a new discrete intensity 𝑠𝑘
through the following transformation:
𝑠𝑘 = 𝑇 𝑟𝑘 = (𝐿 − 1) σ 𝑘 𝑝 𝑟 = (𝐿−1) σ 𝑘 𝑛 .
𝑗=0 𝑟 𝑗 𝑀𝑁 𝑗=0 𝑗

𝑟𝑘: input intensity

𝑠𝑘: new intensity
𝑛𝑗: frequency of intensity 𝑗
𝑀𝑁: total number of image pixels
 Histogram Equalization: discrete case cont.

• In the ideal continuous case histogram equalization produces a new

variable 𝑠 which is uniformly distributed.
• In the discrete case the histogram of the new discrete variable 𝑠𝑘 is far
from flat but:
o The new histogram is still much more stretched (extended) than the
original histogram.
o The new intensity variable always reaches white since
𝐿−1

𝑠𝐿−1 = 𝑇 𝑟𝐿−1 = (𝐿 − 1) ‫ 𝐿 = 𝑗𝑟 𝑟𝑝 ׬‬− 1

𝑗=0

o In other words, this process usually results in an enhanced

image, with an increase in the dynamic range of pixel values.
 Histogram Equalization: discrete case cont.

• In the Figures below you can see how histogram could look like after
equalizing a digital image.
• It is more “extended” and slightly “flatter” compared to the original
histogram
3000 3000

2500 2500

2000 2000

1500 1500

1000
1000

500
500

0
0 50 100 150 200 0
0 50 100 150 200 250 300

Before equalization After equalization

Histogram Equalization: Examples
Histogram Equalization: Examples
Histogram Equalization: Examples
Histogram Equalization: Examples
Histogram Equalization: Examples
Histogram Equalization: Examples
Examples of histogram equalization
Examples
 Histogram Equalization is not always desirable

Histogram equalization may not always produce desirable results, particularly

if the given histogram is very narrow. It can produce false edges and false
regions. It can also increase image “graininess” and “patchiness.”
 Another example of an unfortunate histogram equalization

• Example of image of Phobos (Mars moon) and its histogram.

• Histogram equalization (bottom of right image) does not always provide the
desirable results.
 Example: discrete case
• A 3 −bit 64 × 64 image with 8 intensities is described in the table.
• Discrete histogram equalised
intensity levels are obtained through:
(𝐿−1) 𝑘
𝑠𝑘 = 𝑇 𝑟𝑘 = σ 𝑗=0 𝑛𝑗 .
𝑀𝑁
After applying histogram equalisation:
(𝐿−1) 0
𝑠0 = 𝑇 𝑟0 = σ 𝑗=0 𝑛𝑗 = 7 𝑛0 = 7∙790 = 1.33
𝑀𝑁 4096 4096
7 7
𝑠1 = 𝑇 𝑟1 = σ 1𝑗=0 𝑛𝑗 = 790 + 1023 = 3.08
4096 4096
7 7
𝑠2 = 𝑇 𝑟2 = σ 2𝑗=0 𝑛𝑗 = 790 + 1023 + 850 = 4.55
4096 4096
7 7
𝑠3 = 𝑇 𝑟3 = σ 3𝑗=0 𝑛𝑗 = 790 + 1023 + 850 + 656 = 5.67
4096 4096
7 7
𝑠4 = 𝑇 𝑟4 = σ 4𝑗=0 𝑛𝑗 = 790 + 1023 + 850 + 656 + 329 = 6.23
4096 4096
7 7
𝑠5 = 𝑇 𝑟5 = σ 5𝑗=0 𝑛𝑗 = 790 + 1023 + 850 + 656 + 329 + 245 = 6.65
4096 4096
7 7
𝑠6 = 𝑇 𝑟6 = σ 6𝑗=0 𝑛𝑗 = 790 + 1023 + 850 + 656 + 329 + 245 + 122 = 6.86
4096 4096
7 7
𝑠6 = 𝑇 𝑟6 = σ 6𝑗=0 𝑛𝑗 = 790 + 1023 + 850 + 656 + 329 + 245 + 122 + = 7
4096 4096
 Example: discrete case cont.
By rounding to the nearest integer we get:
𝑠0 = 1.33 → 1, 𝑠1 = 3.08 → 3, 𝑠2 = 4.55 → 5, 𝑠3 = 5.67 → 6
𝑠4 = 6.23 → 6, 𝑠5 = 6.65 → 7 𝑠6 = 6.86 → 7, 𝑠7 = 7 → 7

The histogram of the new variable is found

as follows:
𝑝 0 =0
𝑝 1 = 𝑝 𝑠0 = 𝑝 𝑟0 = 0.19
𝑝 2 =0
𝑝 3 = 𝑝 𝑠1 = 𝑝 𝑟1 = 0.25
𝑝 4 =0
𝑝 5 = 𝑝 𝑠2 = 𝑝 𝑟2 = 0.21
𝑝 6 = 𝑝 𝑠3 + 𝑝 𝑠4 = 𝑝 𝑟3 + 𝑝 𝑟4 = 0.24
𝑝 7 = 𝑝 𝑠5 + 𝑝 𝑠6 + 𝑝 𝑠7 = 𝑝 𝑟5 + 𝑝 𝑟6 + 𝑝 𝑟7 = 0.1
 Example: discrete case cont.

Refer to the following figures for original histogram, transformation function

and new histogram.
 Example: discrete case cont.

• Refer to the following figures for original histogram, transformation

function and new histogram.
• Notice that due to discretization, the resulting histogram will rarely be
perfectly flat.
• However, it will more “extended” compared to the original histogram.
A set of images with same content but different histograms

dark image low

contrast
image
1 3

bright image high

4 contrast
image
2
Histogram equalization applied to the dark image

1
Histogram equalization applied to the bright image

2
 Histogram equalization applied
to the low and high contrast images
3

4
 Transformation functions for histogram equalization
for the previous example
• The function 𝑇(𝑟) used to equalize the four images of the previous example is
shown below.
• Observe that the transformation function in cases 1,2,3 maps a small range of
intensities to the entire range of intensities.
• Observe that for an image which is already bright, the transformation is an almost
diagonal line.
 Histogram Specification

• We are looking for a technique which can provide an image with any pre-
specified histogram.
• This is called histogram specification.
• We assume that the original image has pdf 𝑝𝑟(𝑟).
• We are looking for a transformation 𝑧 = 𝑇(𝑟) which provides an image
with a specific pdf 𝑝𝑧(𝑧) .
• This technique will use histogram equalization as an intermediate step.
 Histogram Specification cont.

• We first equalize the given image

𝑟
𝑠 = 𝑇 𝑟 = (𝐿 − 1) න 𝑝𝑟 𝑤 𝑑𝑤
0

• If we had the desired image we could equalized it and obtain

𝑧
𝑠 = 𝑇 𝑧 = (𝐿 − 1) න 𝑝𝑧 𝑤 𝑑𝑤
0

• Based on the above we can assume that

𝐺 𝑧 = 𝑇 𝑟 ⇒ 𝑧 = 𝐺−1(𝑇 𝑟 )

• In the case of continuous variables, if 𝑝𝑟 𝑟 and 𝑝𝑧 𝑧 are given we can

obtain 𝑧 after formulating the functions 𝑇, 𝐺 and 𝐺−1.
 Histogram Specification: continuous form

• In the discrete case we first equalize the initial histogram of the image:
𝑠𝑘 = 𝑇 𝑟𝑘 = (𝐿 − 1) σ 𝑘 𝑝 𝑟 = (𝐿−1) σ 𝑘 𝑛 .
𝑗=0 𝑟 𝑗 𝑀𝑁 𝑗=0 𝑗

• Then we equalize the target histogram

(𝐿−1) 𝑘
𝑠𝑘 = 𝐺 𝑧𝑞 = (𝐿 − 1) σ 𝑞𝑖=0 𝑝𝑧 𝑟𝑖 = σ 𝑗=0 𝑛𝑗 .
𝑀𝑁

• Finally, we try to obtain some type of inverse transform

𝑧𝑞 = 𝐺−1 𝑠𝑘 = 𝐺−1 𝑇 𝑟𝑘
 Histogram Specification: Example

original number of probability cumulative equalised normalised

intensities pixels probability intensities equalised
CM CM x 7 intensities
0 790 0.19 0.19 1.33 1
1 1023 0.25 0.44 3.08 3
2 850 0.21 0.65 4.55 5
3 656 0.16 0.81 5.67 6
4 329 0.08 0.89 6.23 6
5 245 0.06 0.95 6.65 7
6 122 0.03 0.98 6.86 7
7 81 0.02 1 7 7
 Histogram Specification: Example cont.

desired probability cumulative equalised normalised

intensities probability intensities equalised
CM CM x 7 intensities
0 0 0 0 0
1 0 0 0 0
2 0 0 0 0
3 0.15 0.15 1.05 1
4 0.2 0.35 2.45 2
5 0.3 0.65 4.55 5
6 0.2 0.85 5.95 6
7 0.15 1 7 7
 Histogram Specification: Example cont.

original equalised desired equalised equalised NEW

intensities intensities intensities intensities intensities intensities
(AVAILABLE) (NOT AVAILABLE!!!) (available) (available)
0 1 0 0 1 3
1 3 1 0 3 4
2 5 2 0 5 5
3 6 3 1 6 6
4 6 4 2 6 6
5 7 5 5 7 7
6 7 6 6 7 7
7 7 7 7 7 7
 Histogram Specification: Example cont.

Notice that due to discretization, the resulting histogram will

rarely be exactly the same as the desired histogram.

• Top left: original pdf

• Top right: desired pdf
• Bottom left: desired CDF
• Bottom right: resulting pdf
Histogram Specification: Example
42

Histogram Specification: Example cont.

43

Histogram Specification: Example cont.

Histogram Specification: Example
 Local Histogram Equalization

• In many cases histograms are needed for local areas of an image.

• Possible applications could be:
o Pattern detection based on histogram.
o Adaptive enhancement.
o Adaptive thresholding.
o Object tracking based on histogram.
 Local Histogram Equalization for local image enhancement

• The histogram processing methods discussed previously are global

(transformation is based on the intensity distribution of the entire image).
• This global approach is suitable for overall enhancement.
• There are cases in which it is necessary to enhance details over small
areas in an image.
• The number of pixels in these areas may have negligible influence on
the computation of a global transformation.
• The solution is to devise transformation functions based on the intensity
distribution in a neighbourhood around every pixel.
• carry other tasks such as detection, tracking and spatially adaptive
thresholding.
Example: Local Histogram Equalization
 Example: Local Histogram Equalization

• Observe the details revealed with local (spatially adaptive) histogram

equalization.
 Example: Local Histogram Equalization

Top left: A window size of [25 25]. Top right: A window size of [64 64].
Bottom left: A window size of [100 100]. Bottom right: A window size of [200 200].
 Example: Local Histogram Equalization

Top left: A window size of [15 15]. Top right: A window size of [30 30].
Bottom left: A window size of [75 75]. Bottom right: A window size of [150 150].
Example: Local Histogram Equalization
 Example: Local Histogram Equalization drawbacks

• Image is split into smaller regions and the traditional histogram

equalization is applied to each region.
• The smaller equalized images are combined into one to obtain a final
resultant image.
• The final image appears to be very blocky in nature and has different
contrast levels for each individual region.
• Post-processing is required to remove the blocking artifacts.
 Spatial filters for image enhancement

• Spatial filters called spatial masks are used for specific

operations on the image. Popular filters are the following:

➢ Low pass filters

➢ High pass filters
➢ High boost filters
➢ Others
 Image averaging in the case of many realizations

• We have 𝑀 different noisy images:

𝑔𝑖 𝑥, 𝑦 = 𝑓 𝑥, 𝑦 + 𝑛𝑖 𝑥, 𝑦 , 𝑖 = 0, … , 𝑀 − 1
• Noise realizations are zero mean and white with the same variance, i.e.,
𝐸 𝑛𝑖 𝑥, 𝑦 = 0
2 2
𝑅𝑛𝑖 𝑘, 𝑙 = 𝐸 𝑛𝑖 𝑥, 𝑦 𝑛𝑖 𝑥 + 𝑘, 𝑦 + 𝑙 = 𝜎𝑛𝑖 𝛿[𝑘, 𝑙] = 𝜎𝑛 𝛿[𝑘, 𝑙]
• We define a new image which is the average
𝑀−1 𝑀−1
1 1
𝑔 𝑥, 𝑦 = ∑ 𝑔𝑖 𝑥, 𝑦 = 𝑓 𝑥, 𝑦 + ∑ 𝑛𝑖(𝑥, 𝑦)
𝑀 𝑀
𝑖=0 𝑖=0
• Notice that average is calculated across realizations.
• Problem: Find the mean and variance of the new image 𝑔 𝑥, 𝑦 .
▪ 𝐸{𝑔 𝑥, 𝑦 } = 𝐸{ 1 ∑𝑀−1
𝑖=0 𝑔 𝑖 (𝑥, 𝑦)} =
1 𝑀−1
∑ 𝐸{𝑔 𝑖(𝑥, 𝑦)} = 𝑓(𝑥, 𝑦)
𝑀 𝑀 𝑖=0
1
▪ 𝜎𝑔(𝑥,𝑦) 2 = 𝜎𝑓(𝑥,𝑦) 2 + 𝜎𝑛(𝑥,𝑦) 2 =0 + 1 𝜎𝑛(𝑥,𝑦) 2 = 1 𝜎𝑛(𝑥,𝑦) 2
𝑀 𝑀 𝑀
▪ 𝜎𝑓(𝑥,𝑦)2: variance of pixel 𝑥, 𝑦 is 0
Example: Image Averaging
Example: Image Averaging
Example: Image Averaging
Spatial masks
Local averaging spatial masks for image de-noising
(smoothing)
 Frequency response of 1D and 2D
local averaging spatial masks
• Recall that a rectangular functi on in time becomes a sinc function in the
frequency domain.
Weighted local averaging spatial masks
Example of local image averaging
 A noiseless image and its noisy version

Original Lena image Lena image with 10db noise

 and some denoising

3 × 3 averaging mask 5 × 5 averaging mask

Median filtering
Example of median filtering
Comparison of local smoothing and median filtering
Median versus local averaging filter response
around a local step edge
Median versus local averaging filter response
around a local ridge edge
 High pass filtering

• We use masks with positive coefficients around the centre of the mask
and negative coefficients in the periphery.
• In the mask shown below the central coefficient is +8 and its 8 nearest
neighbours are -1.
• The reversed signs are equally valid, i.e., -8 in the middle and +1 for the
rest of the coefficients.
• The sum of the mask coefficients must be zero. This leads to the
response of the mask within a constant (or slowly varying) intensity area
being zero (or very small).

1
×
9
Example of high pass filtering
 Example of high pass filtering

High pass filtered image High pass filtered image with 10db noise
 High boost filtering

• The goal of high boost filtering is to enhance the high frequency

information without completely eliminating the background of the image.
• We know that:
(High-pass filtered image)=(Original image)-(Low-pass filtered image)
• We define:
(High boost filtered image)=𝐴 ×(Original image)-(Low-pass filtered image)
(High boost)=(𝐴 − 1) ×(Original)+(Original)-(Low-pass)
(High boost)=(𝐴 − 1) × (Original)+(High-pass)
• As you can see, when 𝐴 > 1, part of the original is added back to the high-
pass filtered version of the image in order to partly restore the low
frequency components that would have been eliminated with standard
high-pass filtering.
• Typical values for 𝐴 are values slightly higher than 1, as for example 1.15,
1.2, etc.
 High boost filtering

• The resulting image looks similar to the original image with some edge
enhancement.
• The spatial mask that implements the high boost filtering algorithm is
shown below.
• The resulting image depends on the choice of 𝐴.

1
×
9

• High boost filtering is used in printing and publishing industry.

Example of high boost filtering
 Example of high boost filtering

𝐴 = 1.15 𝐴 = 1.2
 Edge detection

• Edges are abrupt local changes in the image value (for example grey
level).
• Edges are of crucial important because they are related to the
boundaries of the various objects present in the image.
• Object boundaries are key feature for object detection and
identification.
• Edge information in an image is found by looking at the relationship a
pixel has with its neighborhoods.
• If a pixel’s grey-level value is similar to those around it, there is
probably not an edge at that point.
• If a pixel’s has neighbors with widely varying grey levels, it may present
an edge point.
Abrupt versus gradual change in image intensity
 Type of edges

Step edges

Roof edge Line (ridge) edges

Edges are caused by a variety of factors
(observe the irrelevant edges)

surface discontinuity

depth discontinuity

surface color discontinuity

illumination discontinuity
From a grey level image to an edge image (or edge map)
 How do we detect the presence of a local edge?

• The gradient (first derivative) of the image around a pixel might give
information about how detailed is the area around a pixel and whether
there are abrupt intensity changes.
• The image is a 2-D signal and therefore, the gradient at a location (𝑥, 𝑦) is
a 2-D vector which contains the two partial derivatives of the image with
respect to the coordinates 𝑥, 𝑦.
𝜕𝑓
𝜕𝑥
∇𝑓 = 𝜕𝑓
𝜕𝑦
1
2
𝛛𝑓 𝛛𝑓 2 2 𝛛𝑓 𝛛𝑓
• Edge strength is given by ∇𝑓 = + or ∇𝑓 ≅ + .
𝛛𝑥 𝛛𝑦 𝛛𝑥 𝛛𝑦
𝛛𝑓
• Edge orientation is given by 𝜃 = tan−1 / 𝛛𝑓 .
𝛛𝑦 𝛛𝑥
 Examples of edge orientations

1
2
𝛛𝑓 𝛛𝑓 2 2 𝛛𝑓
• Edge strength is given by ∇𝑓 = 𝛛𝑥
+
𝛛𝑦
or ∇𝑓 ≅ 𝛛𝑥
+
𝛛𝑓
.
𝛛𝑦

𝛛𝑓
• Edge orientation is given by 𝜃 = tan−1 / 𝛛𝑓 .
𝛛𝑦 𝛛𝑥
 Spatial masks for edge detection

• Consider an image region of size 3 × 3 pixels. The coordinates 𝑥, 𝑦 are

shown.
• The quantity 𝑟 denotes grey level values.
𝛛𝑓 𝛛𝑓
• The magnitude ∇𝑓 = + of the gradient at pixel with intensity 𝑟5
𝛛𝑥 𝛛𝑦
can be approximated by:
➢ differences along 𝑥 and 𝑦
∇𝑓 = 𝑟5 − 𝑟6 + 𝑟5 − 𝑟8
➢ cross-differences (along the two main diagonals)
➢ ∇𝑓 = 𝑟6 − 𝑟8 + 𝑟5 − 𝑟9
 Spatial masks for edge detection – Roberts operator

𝛛𝑓 𝛛𝑓
• The magnitude 𝛻𝑓 = + of the gradient at pixel with intensity 𝑟5
𝛛𝑥 𝛛𝑦
can be approximated by:
➢ differences along 𝑥 and 𝑦
𝛻𝑓 = 𝑟5 − 𝑟6 + 𝑟5 − 𝑟8
➢ cross-differences (along the two main diagonals)
𝛻𝑓 = 𝑟6 − 𝑟8 + 𝑟5 − 𝑟9
• Each of the above approximation is the sum of the magnitudes of the
responses of two masks. These sets of masks are the so called Roberts
operator.

abs{ } }+abs{ } abs{ }+abs{ }

+
 Spatial masks for edge detection – Prewitt and Sobel
operators
𝛛𝑓 𝛛𝑓
• The magnitude 𝛻𝑓 = 𝛛𝑥
+ 𝛛𝑦
of the gradient at pixel with intensity 𝑟5 can
be also approximated by involving more pixels:
➢ Roberts operator
𝛻𝑓 = (𝑟3+𝑟6 + 𝑟9) − (𝑟1 + 𝑟4 + 𝑟7) + (𝑟7+𝑟8 + 𝑟9) − (𝑟1 + 𝑟2 + 𝑟3)
➢ Sobel operator
➢ 𝛻𝑓 = (𝑟3+2𝑟6 + 𝑟9) − (𝑟1 + 2𝑟4 + 𝑟7) + (𝑟7+2𝑟8 + 𝑟9) − (𝑟1 + 2𝑟2 + 𝑟3)
• Each of the above approximation is the sum of the magnitudes of the
responses of two masks.

abs{ }+abs{ } abs{ }+abs{ }

Example of edge detection using Prewitt
Original image and its histogram
Example of edge detection using Prewitt
Example of edge detection using Sobel
 Edge detection using second order gradient: Laplacian
mask
• The second order gradient gives also information about how detailed is
the area around a pixel.
• It is defined as:
𝜕 2𝑓 𝜕 2𝑓
∇2𝑓 = 2 + 2
𝜕𝑥 𝜕𝑦
• A mask that can approximate the above function is the Laplacian:
∇2𝑓 ≅ 4𝑟5 − (𝑟2+𝑟4 + 𝑟6 + 𝑟8)
Robert Collins
CSE486

Lecture 11:
LoG and DoG Filters
Robert Collins
CSE486
Today’s Topics

Laplacian of Gaussian (LoG) Filter

- useful for finding edges
- also useful for finding blobs!

approximation using Difference of Gaussian (DoG)

Robert Collins

Recall: First Derivative Filters

CSE486

• Sharp changes in gray level of the input

image correspond to “peaks or valleys” of
the first-derivative of the input signal.

F(x)
F ’(x)

x
(1D example)

[Link], PSU
Robert Collins
CSE486
Second-Derivative Filters

• Peaks or valleys of the first-derivative of the

input signal, correspond to “zero-crossings”
of the second-derivative of the input signal.

F(x) F ’(x) F’’(x)

[Link], PSU
Robert Collins
CSE486
Numerical Derivatives
See also T&V, Appendix A.2

Taylor Series expansion

add

Central difference approx

1 -2 1
to second derivative
Robert Collins
CSE486
Example: Second Derivatives
Ixx=d2I(x,y)/dx2

[ 1 -2 1 ]
I(x,y)
2nd Partial deriv wrt x

1
-2
1 Iyy=d2I(x,y)/dy2
2nd Partial deriv wrt y
Robert Collins
CSE486
Example: Second Derivatives
Ixx Iyy

benefit: you get clear localization of

the edge, as opposed to the “smear”
of high gradient magnitude values
across an edge
Robert Collins
CSE486
Compare: 1st vs 2nd Derivatives
Ixx Iyy

Ix Iy
Robert Collins
CSE486
Finding Zero-Crossings
An alternative approx to finding edges as peaks in
first deriv is to find zero-crossings in second deriv.
In 1D, convolve with [1 -2 1] and look for pixels
where response is (nearly) zero?
Problem: when first deriv is zero, so is second. I.e.
the filter [1 -2 1] also produces zero when convolved
with regions of constant intensity.

So, in 1D, convolve with [1 -2 1] and look for pixels

where response is nearly zero AND magnitude of
first derivative is “large enough”.
 Edge Detection Summary
Robert Collins
CSE486

1D 2D
y
step edge

F(x)
I(x) I(x,y) x
x
2nd deriv 1st deriv

dI(x) |∇I(x,y)| =(Ix 2(x,y) + Iy2(x,y))1/2 > Th

> Th
dx tan θ = Ix(x,y)/ Iy(x,y)

d2I(x) =0 ∇2I(x,y) =Ix x (x,y) + Iyy (x,y)=0

dx2
Laplacian
Robert Collins
CSE486
Finite Difference Laplacian

Laplacian filter ∇2I(x,y)

Robert Collins
CSE486
Example: Laplacian
I(x,y)
Ixx + Iyy
Robert Collins
CSE486
Example: Laplacian
Ixx Iyy

Ixx+Iyy
∇2I(x,y)
Robert Collins
CSE486
Notes about the Laplacian:
• ∇2I(x,y) is a SCALAR
– ↑ Can be found using a SINGLE mask
– ↓ Orientation information is lost
• ∇2I(x,y) is the sum of SECOND-order derivatives
– But taking derivatives increases noise
– Very noise sensitive!
• It is always combined with a smoothing operation:

I(x,y) O(x,y)
Smooth Laplacian

[Link], PSU
Robert Collins
CSE486
LoG Filter
• First smooth (Gaussian filter),
• Then, find zero-crossings (Laplacian filter):
– O(x,y) = ∇2(I(x,y) * G(x,y))

Laplacian of Laplacian of Gaussian (LoG)

Gaussian-filtered image -filtered image
Do you see the distinction?
[Link], PSU
Robert Collins
CSE486
1D Gaussian and Derivatives
x2
−
2σ 2
g ( x) = e

x2 x2
1 − 2 x − 2
g ' ( x) = − 2 2 xe 2σ
=− 2 e 2σ
2σ σ

2 x2
x 1 − 2σ 2
g ' ' ( x) = ( 43 − 2 )e
σ σ

[Link], PSU
Robert Collins
CSE486
Second Derivative of a Gaussian

2 x2
x 1 − 2
g ' ' ( x) = ( 43 − 2 )e 2σ
σ σ

2D
analog

LoG “Mexican Hat”

[Link], PSU
Robert Collins
CSE486
Effect of LoG Operator
Original LoG-filtered

Band-Pass Filter (suppresses both high and low frequencies)

Why? Easier to explain in a moment.
Robert Collins
CSE486
Zero-Crossings as an Edge Detector
Raw zero-crossings (no contrast thresholding)

LoG sigma = 2, zero-crossing

Robert Collins
CSE486
Zero-Crossings as an Edge Detector
Raw zero-crossings (no contrast thresholding)

LoG sigma = 4, zero-crossing

Robert Collins
CSE486
Zero-Crossings as an Edge Detector
Raw zero-crossings (no contrast thresholding)

LoG sigma = 8, zero-crossing

Robert Collins
CSE486
Note: Closed Contours
You may have noticed that zero-crossings form
closed contours. It is easy to see why…

Think of equal-elevation
contours on a topo map.

Each is a closed contour.

Zero-crossings are contours
at elevation = 0 .

remember that in our case, the height map is of a LoG filtered

image - a surface with both positive and negative “elevations”
Robert Collins
CSE486 Other uses of LoG:
Blob Detection

Lindeberg: ``Feature detection with automatic

scale selection''. International Journal of
Computer Vision, vol 30, number 2, pp. 77--
116, 1998.
Robert Collins
CSE486
Pause to Think for a Moment:

How can an edge finder also be used to

find blobs in an image?
Robert Collins
CSE486
Example: LoG Extrema

LoG maxima
sigma = 2

minima
Robert Collins
CSE486
LoG Extrema, Detail
maxima

LoG sigma = 2
Robert Collins
CSE486
LoG Blob Finding
LoG filter extrema locates “blobs”
maxima = dark blobs on light background
minima = light blobs on dark background

Scale of blob (size ; radius in pixels) is determined

by the sigma parameter of the LoG filter.

LoG sigma = 2 LoG sigma = 10

Robert Collins
CSE486
Observe and Generalize
convolve result
with LoG

maxima
Robert Collins
CSE486
Observe and Generalize

LoG looks a bit

like an eye.

and it responds
maximally in the
eye region!
Robert Collins
CSE486
Observe and Generalize
LoG Derivative of Gaussian

Looks like dark blob Looks like vertical and

on light background horizontal step edges

Recall: Convolution (and cross correlation) with a

filter can be viewed as comparing a little “picture” of
what you want to find against all local regions in the
mage.
Robert Collins
CSE486
Observe and Generalize
Key idea: Cross correlation with a filter can be viewed
as comparing a little “picture” of what you want to find
against all local regions in the image.

Maximum response: Maximum response:

dark blob on light background vertical edge; lighter on left
Minimum response: Minimum response:
light blob on dark background vertical edge; lighter on right
Robert Collins
CSE486 Efficient Implementation
Approximating LoG with DoG
LoG can be approximate by a Difference of two
Gaussians (DoG) at different scales

1D example

[Link], CMU
Robert Collins
CSE486
Efficient Implementation

LoG can be approximate by a Difference of two

Gaussians (DoG) at different scales.

Separability of and cascadability of Gaussians applies

to the DoG, so we can achieve efficient implementation
of the LoG operator.

DoG approx also explains bandpass filtering of LoG

(think about it. Hint: Gaussian is a low-pass filter)
Robert Collins
CSE486
Back to Blob Detection

Lindeberg: blobs are detected

as local extrema in space and
scale, within the LoG (or DoG)
scale-space volume.
Robert Collins
CSE486
Other uses of LoG:
Blob Detection

Gesture recognition for

the ultimate couch potato
Robert Collins
CSE486
Other uses for LOG:
Image Coding
• Coarse layer of the Gaussian pyramid predicts the
appearance of the next finer layer.
• The prediction is not exact, but means that it is not
necessary to store all of the next fine scale layer.
• Laplacian pyramid stores the difference.
Robert Collins
CSE486
Other uses for LOG:
Image Coding
256x256 128x128 64x64 32x32

takes less bits to store

compressed versions of
these than to compress
the original full-res
256x256 128x128 64x64 greyscale image

The Laplacian Pyramid as a Compact Image Code Burt, P., and Adelson, E. H.,
IEEE Transactions on Communication, COM-31:532-540 (1983).