What is an image?
What is an image?
Digital Camera
We’ll focus on these in this class
(More on this process later) The Eye
Source: A. Efros
What is an image?
• A grid (matrix) of intensity values
255 255 255 255 255 255 255 255 255 255 255 255
255 255 255 255 255 255 255 255 255 255 255 255
255 255 255 20 0 255 255 255 255 255 255 255
255 255 255 75 75 75 255 255 255 255 255 255
=
255 255 75 95 95 75 255 255 255 255 255 255
255 255 96 127 145 175 255 255 255 255 255 255
255 255 127 145 175 175 175 255 255 255 255 255
255 255 127 145 200 200 175 175 95 255 255 255
255 255 127 145 200 200 175 175 95 47 255 255
255 255 127 145 145 175 127 127 95 47 255 255
255 255 74 127 127 127 95 95 95 47 255 255
255 255 255 74 74 74 74 74 74 255 255 255
255 255 255 255 255 255 255 255 255 255 255 255
255 255 255 255 255 255 255 255 255 255 255 255
(common to use one byte per value: 0 = black, 255 = white)
Pixels
A pixel is the smallest unit of representation for
visual information
Each pixel in a digitized image represents one
intensity (brightness) level (gray scale or color)
13 x 13 grid = 169 pixels
Gray scale Color
How many pixels should be used
If too few pixels used, image appears
“coarse”
16 x16 (256 pixels) 64 x 64 (4096 pixels)
Digitizing Images (gray scale)
• The first step to digitize a “black and white” image composed of
an array of gray shades, is to divide the image into a number of
pixels, depending on the required spatial resolution
• The number of brightness levels to be represented by each pixel
is assigned next
• If we wish to use for example, 6 bits for the brightness level of
each pixel, then each pixel can represent 64 different brightness
levels (shades of gray, from black to white)
• Then, each pixel would have a 6-bit number associated with it,
representing the brightness level (shade) that is closest to the
actual brightness level at that pixel
Digitizing Images (gray scale)
• This process is known as quantization (we will learn more
about this later in the course)-It is the process of rounding
off actual continuous values so that they can be
represented by a fixed number of binary digits
• As a result of the operations just described, the analog
image is digitized and represented by a string of binary
digits…
6-bit image (64 gray levels) 3-bit image (8 gray levels)
1-bit image (black and white)
Monochrome vs. Grayscale
Monochrome: Grayscale:
Each pixel is stored as a
• Each pixel is usually stored as a
single bit (0 or 1) byte (value between 0 to 255)
A 640 x 480
• A 640 x 480 grayscale image
monochrome image requires over 300 KB of
requires 37.5 KB of storage.
storage.
10
How much storage is needed?
• Total number of bits required for storage = total number
of pixels * number of bits used per pixel
• For example – Black and white photo
– 64 x 64 pixels
– Use 32 gray levels (5 bits)
– 64 x 64 x 5 = 20,480 bits = 2560/1024 bytes = 2.5KB
– Remember data storage is in bytes
– KB represents 210 or 1024 bytes
Another example
• Black and White photo
• 256 x 256 pixel
• 6 bits (64 gray levels)
• How much storage is needed?
• 256 x 256 x 6 = 393,216 bits
• 393,216/8 = 49,152 bytes
• 49,152/1024 = 48 KB
Image transformations
• As with any function, we can apply operators
to an image
g (x,y) = f (x,y) + 20 g (x,y) = f (-x,y)
• Today we’ll talk about a special kind of
operator, convolution (linear filtering)
Filters
• Filtering
– Form a new image whose pixels are a combination
of the original pixels
• Why?
– To get useful information from images
• E.g., extract edges or contours (to understand shape)
– To enhance the image
• E.g., to remove noise
• E.g., to sharpen and “enhance image” a la CSI
– A key operator in Convolutional Neural Networks
Image filtering
• Modify the pixels in an image based on some
function of a local neighborhood of each pixel
10 5 3 Some function
4 5 1 7
1 1 7
Local image data Modified image data
Source: L. Zhang
Linear filtering
• One simple version of filtering: linear filtering
(cross-correlation, convolution)
– Replace each pixel by a linear combination (a weighted
sum) of its neighbors
• The prescription for the linear combination is called
the “kernel” (or “mask”, “filter”)
10 5 3 0 0 0
4 6 1 0 0.5 0 8
1 1 8 0 1 0.5
Local image data kernel Modified image data
Source: L. Zhang
Mean filtering
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 10 20 30 30 30 20 10
0 0 0 90 90 90 90 90 0 0 0 20 40 60 60 60 40 20
=
0 0 0 90 90 90 90 90 0 0 0 30 60 90 90 90 60 30
*
1 1 1
1 1 1 0 0 0 90 90 90 90 90 0 0 0 30 50 80 80 90 60 30
0 0 0 90 0 90 90 90 0 0 0 30 50 80 80 90 60 30
1 1 1
0 0 0 90 90 90 90 90 0 0 0 20 30 50 50 60 40 20
0 0 0 0 0 0 0 0 0 0 10 20 30 30 30 30 20 10
0 0 90 0 0 0 0 0 0 0 10 10 10 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
Mean filtering/Moving average
Mean filtering/Moving average
Mean filtering/Moving average
Mean filtering/Moving average
Mean filtering/Moving average
Mean filtering/Moving average
Linear filters: examples
=
0 0 0
* 0
0
1
0
0
0
Original Identical image
Source: D. Lowe
Linear filters: examples
=
0 0 0
* 1
0
0
0
0
0
Original Shifted left
By 1 pixel
Source: D. Lowe
Linear filters: examples
1 1 1
=
* 1
1
1
1
1
1
Original Blur (with a mean filter)
Source: D. Lowe
Linear filters: examples
0 0 0
-
1 1 1
* 0
0
2
0
0
0
1
1
1
1
1
1
=
Sharpening filter
Original (accentuates edges)
Source: D. Lowe
Sharpening
Source: D. Lowe
Filters: Thresholding
