CSCI435/CSCI935

Computer Vision – Algorithms and Systems

Introduction and subject overview

Lecturer: Prof Wanqing Li
Room 3.101
Email: wanqing@[Link]
Web: [Link]

22/07/2024 1
Outline
About Me

What is Compute Vision ?

About the Subject

About OpenCV

Basic Classifiers

22/07/2024 2
The Senses

Hearing

Smell Vision
delivers more
Taste than 80% of
the information

Touch

22/07/2024 5
The Nature of Human Vision System (HVS)

Color/Intensity
Geometric
Semantic

The visual pathway of the

Structure of the eye human vision system

22/07/2024 6
 Optical Illusions

Adelson's checker

[Link]

22/07/2024 7
 Optical Illusions

[Link]

22/07/2024 9
 Optical Illusions

22/07/2024 10
 Optical Illusions

by M. White

22/07/2024 11
Notes on HVS

 Human Vision is a very complex multistage process

 Human Vision is optimised ( may be by evolution) to
provide reasonable reaction time in daily life
 Human Vision keeps focus on objects of interest
ignoring other details
 Human Vision is not the ultimate visual system. It has
poor performance in measurement of parameters and
may lead to incorrect conclusions
 Efficiency of Human Vision declines due to fatigue

22/07/2024 12
 What is Computer Vision ?

22/07/2024 13
What is Computer Vision
 Also known as “Machine Vision”
▪ The science and technology of machines that see.
▪ The process where a digital computer
automatically control the process of image
capturing and image analysis to recognise the
content of the image
 Purpose
▪ To mimic the functions of a human vision system
 Guiding principle
▪ If the eye can do it, so should the machine
22/07/2024 14
A Computer Vision System
 Key components of a computer vision system

Low-Level Vision Mid-Level Vision High-Level Vision

Digital Pre- Feature Vision

Camera Processing Extraction Tasks Content/
Decision
eyes brain
 The structure and algorithms do not necessarily
mimic the actual process of human visual perception,
but may utilise some results of biological and
cognitive studies

22/07/2024 15
Popular topics in Computer Vision
Computational imaging or Computational
Photography
▪ Digital image capture and processing techniques that
use digital computation instead of optical processes
▪ Examples
o Panoramas
o High-dynamic-range images
o Light-field-cameras

[Link]
[Link]
[Link]
Computational Imaging by Peyman Milanfar (2014, 2020), Google Research

22/07/2024 16
Popular topics in Computer Vision…
 Image Restoration/Quality Enhancement - Blurring, noise, and other image
artifacts can be removed by accurately identifying where the object versus
background is in the image. Low resolution image can be resorted to high
resolution

From mobile images to DSLR quality images

Super-Resolution

22/07/2024 17
Invisible Motions (Runinstein
2014)

[Link]
Popular topics in Computer Vision…

 Image classification - An approach of classification of

an images into often semantically meaningful categories

Stanford cs231n
22/07/2024 19
Popular topics in Computer Vision…

 Object detection/recognition – detecting/identifying

instances of semantic objects of a certain class (such
as humans, buildings, or cars) in digital images and
videos.

[Link]

22/07/2024 20
Object Recognition
Captured Pattern

Pattern Matching time ! Pattern 2

Classifier

Due to Pattern Database

Noise
Number of bits per
1 2 3 pattern: 25
Number of possible
patterns: 225 = 33.5 M

Storage space !

22/07/2024 21
Object Detection
Captured Image


 Object Location

 

Image size: 256x256 Object Database

Object size: 5x5
Required number of
operations: 52*2562= 1.6M

22/07/2024 22
Shark, Ripple & Smoke detection

22/07/2024 23
Human detection and recognition

l
l w

w
hc h
h hc

β
α
α
β

22/07/2024 24
Popular topics in Computer Vision…

 Image segmentation - the process of partitioning a

digital image into multiple segments (sets of pixels,
also known as super-pixels).

Berkeley meetshah1995 [Link]

Region segmentation Semantic segmentation Instance segmentation

22/07/2024 25
Popular topics in Computer Vision…
 3D Reconstruction - 3D reconstruction is the process
of capturing the shape and appearance of real
objects - multiple cameras, multiple single images,
video, depth camera

M. Pollefeys et al. (2004), IJCV

22/07/2024 26
Real-time 3D Reconstruction with a single depth camera

22/07/2024 27
Examples of Applications
A key component in many
intelligent systems
 Video Surveillance
 AR/VR systems
 Autonomous driving/vehicles
 Robots/Drones/UAV
 Human-Computer Interface
 Automatic
Assembly/Inspection Lines
 ……

22/07/2024 28
Amazon Go

22/07/2024 29
Industrial Visual Inspection

[Link]

22/07/2024 32
Computer Vision and other fields

[Link]

22/07/2024 33
 About the subject

22/07/2024 34
Subject Learning Outcomes
On successful completion of this subject,
students are expected to:
• Understand the principle of digital image and video
cameras.
• Use image enhancement techniques.
• Use object detection and recognition techniques.
• Use video processing techniques to detect moving
objects.
• Design and implement basic computer vision systems
for real applications.

22/07/2024 35
Topics Covered by the Subject
Week Topic* Task Due
1 Introduction to Computer Vision and Subject overview
Image formation, acquisition and quality: photometry
2-3
colour, digital cameras and image enhancement
Low-level vision and middle-level vision
4-7 • detection of edges, shape and key points, image A1 (w4)
segmentation
Video processing and analysis:
8-9 • Image sequences (videos) and motion, Moving A2(w8)
object extraction
High-level vision:
10-12 • object detection and recognition, deep learning in A3(w13)
computer vision
13 Revision

*The schedule may be subject to change

22/07/2024 36
Method of Presentation
Lectures :
▪ Tuesday: 14:30 – 16:30
▪ Lecture recordings @ [Link]
▪ Access via subject Moodle or login using your uow
email (as user name) and uow password
Assignments : 3 assignments

It should be noted that the amount of time spent on each 6 credit

point subject should be at least 12 hours per week, which includes
lectures/workshops/labs etc.

22/07/2024 37
Assessments

MINIMUM
ASSESSMENT % OF FINAL
REQUIREMENT DUE DATE
ITEMS & FORMAT MARK

Three Due in week 4, 8 &

60%
Assignments 13*

Final Exam 40% 30% (12 marks) Exam period

*These are tentative, please check the assignment description

for actual due dates and times of the assignments

22/07/2024 39
 Assignments
 There will be 3 programming/practical assignments
▪ Anaconda 3 (highly recommended)
▪ OpenCV 4.10.0-Python-3.12 (with only numpy and matplotlib
packages)
▪ Language – Python
▪ Assignments will be marked using these specified versions

 When an assignment is released, download the assignment

description from the Moodle site. Read carefully the specifications
and make sure you understand what you are required to do

 Your completed solutions must be submitted electronically as

specified in assignment descriptions

22/07/2024 40
 Assignments…
 Penalties apply to all late assignments without a granted
special consideration
▪ 50% of the total assessment mark for that task per day including
weekends no matter how many seconds the submission is late.
▪ The late time is calculated in days. For instance, one second late will be
considered as one day; if it is one day and one second late, it will be
considered as 2 days.
▪ All submissions received 2 days (calculated as explained above) after
the due date will receive no marks
▪ Enquiries about the marks can only be made within a maximum of 1 week
after the marking results are released. After 1 week, no more marks can
be changed.
 Requests for extensions should be made electronically by
logging on to SOLS before the due time

22/07/2024 41
Subject Materials
Lecture slides:
▪ The lecture slides will be available on the subject Moodle.
Third party library used in this subject
▪ OpenCV 4.10.0-Python 3.12

Recommended references:
▪ A. Torralba, P. Isola and W. T. Freeman, Foundations of
Computer Vision, MIT Press Academic (14 May 2024)
▪ R. Szeliski, Computer Vision: Algorithms and Applications,
Springer; 2nd ed. 2022 edition (6 January 2023)
▪ OpenCV Course - Full Tutorial with Python
o [Link]

22/07/2024 42
Notes
This subject
▪ Does not teach you programming in Python
▪ Does not teach you OpenCV 4.10.0 either!
o You need learn OpenCV 4.10.0 by yourself
through online tutorials
▪ Does not teach machine learning algorithms
such as classification and clustering, but
we will use a few of them

22/07/2024 43
How to contact me

Consultation times
▪ Monday 14:30 – 16:30
▪ Tuesday 10:30 – 12:30

Email
▪ Set the subject of the email as
o CSCI435 or CSCI935: (topic of the email)
▪ Will be responded as soon as possible

22/07/2024 44
Note

SOLs Mail
▪ Check your SOLs mail regularly
Subject Moodle
▪ Regularly visit subject Moodle

Any information sent out through SOLs mail

and/or posted on the Subject Moodle is
deemed to to have been notified to the
students
22/07/2024 45
 About OpenCV

22/07/2024 46
OpenCV 4.10.0
Open source of computer vision library
▪ More than 2500 optimized algorithms covering many areas for
CV
▪ 47 thousand users with 18 million downloads
▪ Written natively in C++ with a templated interface that works
seamlessly with STL containers.

Supported OS and Languages

▪ Linux, Microsoft Windows and Android
▪ C++, Python, Java and MATLAB interfaces
▪ CPU and GPU

22/07/2024 47
OpenCV 4.10.0
Home Page ([Link]
▪ Download ([Link] )

▪ Installation ([Link]

▪ References ([Link]

▪ Tutorials ([Link] )

▪ Books ([Link]

22/07/2024 48
Modules in OpenCV 4.10.0

[Link]
22/07/2024 49
Installation and Test
import cv2
import numpy as np
import sys
Installation def main (filename):

▪ Instructions image = [Link](filename)

show_image(image)

def show_image(image):
"""
Function to display image
Test :param image:
:return:
"""
▪ [Link] [Link]('image', cv2.WINDOW_NORMAL)
[Link]('image', image)
[Link](0)
[Link]()

def parse_and_run():
file_name = [Link][1]
main(file_name)

if __name__ == '__main__':
parse_and_run()

22/07/2024 50
Note on editing & running python code

 Use any text editor to edit python code

▪ e.g. PyCharm
 Run .py code from a Command Terminal or Window Powershell

python Your python code Augments or

22/07/2024 .py file options 51
Recommended Tutorials/Exercises
 Install Python 3.12.0 and OpenCV-4.10.0-Python
▪ Getting Started with Images
▪ Arithmetic Operations on Images

 OpenCV Course - Full Tutorial with Python

▪ [Link]

22/07/2024 52
 Basic Classifiers

22/07/2024 53
Commonly Used Classifiers

Supervised Classifiers
▪ Bayesian classifiers
▪ K-Nearest neighbor classifiers (KNN)
▪ Support vector machines (SVM)
▪ Multi-layer perception (MLP)
Unsupervised Clustering
▪ K-means K

KNN KNN

K K

22/07/2024 54
K-Nearest Neighbor Classifier

22/07/2024 55
22/07/2024 56
22/07/2024 57
22/07/2024 58
 One of the great things about nearest neighbour algorithms is
that we can add in new data to update the model very easily.

22/07/2024 59
22/07/2024 60
22/07/2024 61
Support Vector Machine (SVM)

22/07/2024 62
22/07/2024 63
22/07/2024 64
K-means: A Simple example (using K=2)
Step 1: Initialization: Randomly we choose following two
centroids (k=2) for two clusters. In this case the 2 centroid are:
m1=(1.0,1.0) and m2=(5.0,7.0).
Step 2:
Thus, we obtain two
clusters containing:
{1,2,3} and {4,5,6,7}.
Their new centroids are:
Step 3:
Now using these
centroids we compute
the Euclidean distance of
each object, as shown in
table.

Therefore, the new

clusters are:
{1,2} and {3,4,5,6,7}

Next centroids are:

m1=(1.25,1.5) and m2 =
(3.9,5.1)
Step 4 :
The clusters obtained
are:
{1,2} and {3,4,5,6,7}

Therefore, there is no
change in the cluster.
Thus, the algorithm
comes to a halt here and
final result consist of 2
clusters {1,2} and
{3,4,5,6,7}.
PLOT
(with K=3)

Step 1 Step 2
PLOT
Reading Materials

Subject Moodle
▪ [Link]

Many tutorials are available online

22/07/2024 73