Discrete Mathematics

PART 1
COMBINATORIAL THEORY
(Lý thuyết tổ hợp)

PART 2
GRAPH THEORY
(Lý thuyết đồ thị)
2
 Content of Part 2
Chapter 1. Fundamental concepts
Chapter 2. Graph representation
Chapter 3. Graph Traversal
Chapter 4. Tree and Spanning tree
Chapter 5. Shortest path problem
Chapter 6. Maximum flow problem
Content
1. Shortest path problem
2. Shortest path properties, Reduce upper bound
3. Bellman-Ford algorithm
4. Dijkstra algorithm
5. Shortest path in acyclic graph
6. Floyd-Warshall algorithm

4
1. Shortest Path
• Generalize distance to weighted setting
• Digraph G = (V,E) with weight function W: E ® R (assigning real values to
edges) w(v1,v2) w(v2,v3) w(vk-1,vk) k -1
• Weight of path p = v1 ® v2 ® … ® vk is
å
w( p ) = w(v , v )
i =1
i i +1

• Shortest path = a path of the minimum weight

min{ω(p) : u v}; if there is a path from u to v,

δ(u,v)= p

¥ otherwise.

• Applications
• static/dynamic network routing
• robot motion planning
• map/route generation in traffic
• speech interpretation (best interpretation of a spoken sentence)
• medical imaging
Example
Graph G = (V, E), source vertex sÎV, find the shortest path from s to
each of remaining vertices.
3
a d
3 4 6
5
Source vertex s 1 c 1
2 f
2
5 3
b e
2

s a b c d e f
path s s,a s,a,b s,a,b,c s,a,d s,a,b,e s,a,b,e,f
weight 0 3 4 6 6 6 9
Shortest-Path Variants
• Single-source shortest-paths problem
• Find a shortest path from a given source (vertex s) to each of the vertices.
• Single-destination shortest-paths problem
• Find a shortest path to a given destination vertex t from each vertex v.
• Single-pair shortest-path problem
• Given two vertices, find a shortest path between them. Solution to single-
source problem solves this problem efficiently, too.
• All-pairs shortest-paths problem
• Find a shortest path from u to v for every pair of vertices u and v
Comment:
• The problems are arranged in order from simple to complex
• Whenever there is an efficient algorithm for solving one of the three
problems, the algorithm can also be used to solve the remaining two
problems.
Negative Weights and Cycles?
• Negative edges are OK.
• Negative weight cycles: NO (otherwise paths with arbitrary small
“lengths” would be possible)
-18 Cycle: (d ® b ®c ®d)
b c
Length = -10
a 3 5
Path from a to e:
2 5 P: a® s(d ® b ®c ®d) ® e
d e
w(P) = 7-10s ® -∞, khi s ®+ ∞
Assumption:
Graph does not contain negative weight cycles
Properties of shortest paths
• Property 1. Shortest-paths can have no cycles (= The shortest path
can always be found among single paths). Path where vertices are distinct.
Proof: Removing a cycle with positive length could reduce the length of
the path. w
u v

C w(C) ³ 0

• Property 2. Any shortest-path in graph G can not traverse through

more than n – 1 edges, where n is the number of vertices
• Consequence of Property 1.
>= n edges à not the simple path

9
Properties of shortest paths
Property 3: Assume P = ‹v1, v2, …, vk› is the shortest path from v1 to vk.
Then, Pij = ‹vi, vi+1, …, vj› is the shortest path from vi to vj, where 1 £ i £ j £ k.

(In words: subpaths of shortest paths are also shortest paths)

Proof (by contradiction). If Pij is not the shortest path from vi to vj, then
one can find P’ij is the shortest path from vi to vj satisfying w(P’ij) <
w(Pij). Then we get P’ is the path obtained from P by substituing Pij by
P’ij, thus:
w(P’) < w(P) ?!
vi
vj
Pij vk
v1

P’ij
if some subpaths were not the shortest paths, one could substitute the shorter
subpath and create a shorter total path
Properties of shortest paths

Denote: δ(u, v) = length of the shortest path from u to v

Assume P is the shortest path from s to v, where P = s… ¾¾® p'

.. u ®v.
Then δ(s, v) = δ(s, u) + w(u, v).

w(u, v)
u v
δ(s, u):length of the
shortest path p’
from s to v Shortest path from s to v:
s

P = s…
¾
¾®
p'
.. u ®v
Properties of shortest paths
Denote: δ(u, v) = length of the shortest path from u to v

Assume P is the shortest path from s to v, where P = s…¾

¾®p'
.. u ®v.
Then δ(s, v) = δ(s, u) + w(u, v).

Property 4: Assume s Î V. For each edge (u,v) Î E, we have

δ(s, v) £ δ(s, u) + w(u,v).

δ(s, u): length of the w(u,v)

u
shortest path v
from s ton u
s
δ(s, v): length of the
shortest path from s to v
Shortest-Path Variants
• Single-source shortest-paths problem
• Find a shortest path from a given source (vertex s) to each of the vertices.
• Single-destination shortest-paths problem
• Find a shortest path to a given destination vertex t from each vertex v.
• Single-pair shortest-path problem
• Given two vertices, find a shortest path between them. Solution to single-
source problem solves this problem efficiently, too.
• All-pairs shortest-paths problem
• Find a shortest path from u to v for every pair of vertices u and v

Comment:
• The problems are arranged in order from simple to complex
• Whenever there is an efficient algorithm for solving one of the three
problems, the algorithm can also be used to solve the remaining two
problems.
Single-source shortest paths
Shortest path representation
Shortest path algorithms works on 2 arrays:
d(v) = the length of shortest path from s to v that algorithm found so far
(upper bound for the length of the shortest path from s to v).

p(v) = a predecessor of v in this shortest path d(s,v) £ d(v)

(used to back trace the path from s to v) .

Initialization
d(v) = 3+2+7+4=16
for v Î V(G) s p(v) = c
do d[v] ¬ ¥ 3
p[v] ¬ NULL p(a) 4 v
d[s] ¬ 0 a 2 7 c p(v)
p(b) b
p(c)
Relaxation
Building the shortest path from s to v:
Assume the current known shortest path from s to v: s … z àv
Relaxing an edge (u,v) means testing
whether we can improve the shortest path to
d[u] u
v found so far by going through u w(u, v)
s v
Relax(u, v) z
p(v)
if (d[v] > d[u] + w(u, v))
{ d[v] ¬ d[u] + w(u, v) d[v] > d[u] + w(u, v)
p[v] ¬ u
}
u p(v)

s v
z
16
Properties of Relaxation
Relax(u, v)
if (d[v] > d[u] + w(u, v))
{
d[v] ¬ d[u] + w(u, v)
p[v] ¬ u
}

Shortest path algorithms differ in

Øhow many times they relax each edge, and
Øthe order in which they relax edges

17
Single source shortest path
1. Bellman-Ford algorithm
2. Dijkstra algorithm

18
Bellman-Ford algorithm

Richard Bellman
1920-1984 Lester R. Ford, Jr.
1927-2017

19
Bellman-Ford algorithm
Bellman-Ford algorithm is used to find the shortest path from a
vertex s to each other vertex in the graph.
• Input: A directed graph G=(V,E) and weight matrix w[u,v] Î R
where u,v Î V, source vertex s Î V; >= 0
• G does not contain negative-weight cycle <0

• Output: Each v Î V
d[v] = d(s, v); Length of the shortest path from s to v
p[v] - the predecessor of v in this shortest path from s to v.

20
Bellman-Ford algorithm: Full version
Bellman-Ford(G, w, s)
// Step 1: Initialize shortest paths of with at most 0 edges
1. Initialize-Single-Source(G, s)
/* Step 2: Calculate shortest paths with at most i edges from shortest
paths with at most i-1 edges */
2. for i in range (1, |V|)
3. for each edge (u, v) Î E
4. Relax(u, v)
5. for each edge (u, v) Î E
6. if d[v] > d[u] + w(u, v)
7. return False // there is a negative cycle
8. return True
Relax(u, v) Initialize-Single-Source(G, s)
if d[v] > d[u] + w(u, v) for v Î V\s
{
{ d[v] = ¥;
d[v] = d[u] + w[u,v] ; p[v] = Null;
p[v] = u ; }
} p[s]=Null; d[s]=0;
21
Bellman-Ford algorithm: Full version
Bellman-Ford(G, w, s)
// Step 1: Initialize shortest paths of with at most 0 edges
1. Initialize-Single-Source(G, s) O(|V|)
/* Step 2: Calculate shortest paths with at most i edges from shortest
paths with at most i-1 edges */
2. for i in range (1, |V|) O(|V||E|)
Lines (2-4): First nested for-loop performs |V|-1
3. for each edge (u, v) Î E relaxation iterations; relax every edge at each
4. Relax(u, v) iteration è Running time O(|V||E|)
5. for each edge (u, v) Î E O(|E|)
6. if d[v] > d[u] + w(u, v)
7. return False // there is a negative cycle
8. return True

• Running time O(|V||E|)

• Memmory space: O(|V|2)

22
Bellman-Ford algorithm
Bellman-Ford(G, w, s)
1. Initialize-Single-Source(G, s) O(|V|)
2. for i in range (1,|V|) O(|V||E|)
3. for each edge (u, v) Î E
4. Relax(u, v)
5. for each edge (u, v) Î E O(|E|)
6. if d[v] > d[u] + w(u, v)
7. return False // there is a negative cycle
8. return True

if Ford-Bellman has not converged after |V| - 1 iterations, then there

cannot be a shortest path tree, so there must be a negative weight cycle.

23
Single source shortest path
1. Bellman-Ford algorithm
2. Dijkstra algorithm

24
Dijkstra algorithm
• In case the weights on the edges are non-
negative, the algorithm proposed by Dijkstra is
more efficient than the Ford-Bellman algorithm.
• Algorithms are built by labeling vertices. The
label of the vertices is initially temporary. At
each iteration there is a temporary label that
becomes a permanent label. If the label of a Edsger [Link]
vertex u becomes fixed, d[u] gives us the length (1930-2002)
of the shortest path from the source s to u.
Algorithm ends when the labels of all vertices
become fixed.

25
Dijkstra algorithm
• Input: A directed graph G=(V,E) and weight matrix w[u,v] ≥ 0
where u,v Î V, source vertex s Î V;
• G does not have negative-weight cycle
• Output: Each v Î V
d[v] = d(s, v); Length of the shortest path from s to v
p[v] - the predecessor of v in this shortest path from s to v.

Use greedy algorithm:

Maintain a set S of vertices for which we know the shortest path
At each iteration:
• grow S by one vertex , choosing shortest path through S to any other
vertex not in S
• If the cost from S to any other vertex has decreased, update it

26
Dijkstra algorithm
Dijkstra ( )
{
for v Î V // Initialize
{
d[v] = w[s,v] ;
p[v] = s;
}
d[s] = 0; S = {s}; // S: the set of vertices with fixed label (shortest path from s to it has been found)
T = V \ {s}; // T: the set of vertices with temporary label
while (T ¹ Æ) //Loop
{
Find vertex u Î T satisfying d[u] = min{ d[z] : z Î T};
T = T \ {u}; S= S È {u}; //Fixed label of vertex u
for v Î adj[u] and v Î T //Assign new label to each vertex v of T if necessary (if value d[v] is decreased)
if (d[v] > d[u] + w[u,v])
{
d[v] = d[u] + w[u,v] ;
p[v] = u ;
}
}
}

27
Dijkstra algorithm
void Dijkstra ( ) • O(|V|2) operations
{
for v Î V // Initialize – (|V|-1) iterations: 1 for each vertex u added
{ to the distinguished set S.
d[v] = w[s,v] ;
p[v]=s;
– (|V|-1) iterations: for each adjacent vertex of
} the one added to the distinguished set.
d[s] = 0; S = {s};
T = V \ {s};
while (T ¹ Æ) //Loop
{
Find vertex u Î T satisfying d[u] = min{ d[z] : z Î T};
T = T \ {u}; S= S È {u};
for v Î adj[u] and v Î T
if (d[v] > d[u] + w[u,v])
{
d[v] = d[u] + w[u,v] ;
p[v] = u ;
}
}
}

28
Shortest path problems

1. Bellman-Ford algorithm Edge weight: >, <, = 0

Running time: O(|V||E|)
Shortest path in the graph having cycle
Non-negative length cycle

2. Dijkstra algorithm Edge weight >=0

Running time: O(|V|2)

3. Shortest path in the directed graph with no cycles

(Directed acyclic graph (DAG))
Edge weight: >, <, = 0 Running time: O(|E|)

29
Single-Source Shortest Paths in DAGs
A topological sort or topological ordering of a DAG is a linear ordering of its
vertices such that for every directed edge (u, v) from vertex u to vertex v, u
comes before v in the ordering. (In orther words: its vertexes can be numbered
so that each directed edge starting from the vertex with the smaller index to the
vertex with a larger index)

1 2 3
A B D 1 2 3

4 C E 5 4 5

T F G 6 7 8

6 7 8
30
Topological sorting algorithm
• We see that: In the DAG, there always exists a vertex with in-dgree = 0

7
5 1
2 3 6
2 1 2 10
1
3
1 4 5
2 4

31
Topological sorting algorithm
• We see that: In the DAG, there always exists a vertex with in-dgree = 0
Indeed, starting at vertex v1 if there is an incoming edge to it from vertex v2
then we move to v2. If there is an edge from v3 to v2, then we switch to v3, ...
Since there is not any cycle in the graph, so after a finite number of such
transfers we have to go to the vertex without incoming edge.

v1 v2 v3 …. x

• Topological sorting algorithm:

First, finding all vertices with in-dgree = 0. We index these vertices starting
from 1.
Next, removing from graphs vertices that have just been indexed together with
the edge going out of them, we get a new graph also without cycle, and we
again starting index vertices on this new graph.
The process is repeated until all vertices of the graph has been indexed.

32
Topological sorting algorithm
• Input: DAG G=(V,E) with the adjacent list Adj(v), v Î V.
• Ouput: For each v Î V the index NR [v] satisfying: Each directed edge
(u, v): NR[u] < NR[v].

NR[u] u v NR[v]

33
Single-Source Shortest Paths in DAGs
Shortest paths are always well-defined in DAGS
Ø no cycles => no negative-weight cycles even if there are negative-weight edges
In a DAG:
• Every path is a subsequence of the topologically sorted vertex order
• If we do topological sort and process vertices in that order
• We will process each path in forward order
Ø Never relax edges out of a vertex until have processed all edges into the vertex
Thus, just 1 iteration is sufficient
DAG-SHORTEST PATHS(G, s)
TOPOLOGICALLY-SORT the vertices of G
INIT(G, s)
for each vertex u taken in topologically sorted order do
for each v Î Adj[u] do
RELAX(u, v)
• Topological sorting: O(|E|)
• Initialized-Single-Source: O(|E|)
• Nested for-loop: each edge is “traversed” exactly once. Hence, it takes O(|E|) time.
Hence, total running time: O(|E|)
34
Single-Source Shortest Paths in DAGs
• Input: DAG G=(V, E) with topological sorting, V={ v[1], v[2], ... , v[n] }.
Each directed edge (v[i], v[j]) Î E, we have i < j.
The adjacent list Adj(v), v Î V.
• Output: The shortest path from v[1] to all other vertices stored in the array
d[v[i]], i = 2, 3, ..., n

DAG-SHORTEST PATHS(G, v[1])

TOPOLOGICALLY-SORT the vertices of G
INIT(G, v[1])
for each vertex u taken in topologically sorted order do
for each v Î Adj[u] do
RELAX(u, v)

35
Single-Source Shortest Paths in DAGs
Critical_Path ( )
{
d[v[1]] = 0;
for j in range (2, n+1) d[v[j]] =∞;
for v[j] Î Adj[v[1]]
Initialization:
d[v[j]] := w(v[1], v[j]) ; Init(G, v[1])
for j in range (2, n+1)
for v Î Adj[v[j]]
Relax(v[j],v)

} d[v[j]]
v[j] w(v[j], v)
v[1] d[v] = min (d[v], d[v[j]] + w(v[j], v));
v
d[v]
Running time: O(|E|), since each edge is traversed exactly once

36
Example
Find the shortest path from a to each other vertices of the
DAG graph where vertices are already topological order

6 1
r a t u v w
5 2 7 –1 –2

4
3
2

37
Example

6 1
r a t u v w
5 2 7 –1 –2
¥ 0 2 6 5 3

4
3
2
Result: The shortest path tree from a represented by
the red edges

38
Shortest-Path Variants
• Single-source shortest-paths problem
• Find a shortest path from a given source (vertex s) to each of the vertices.
• Single-destination shortest-paths problem
• Find a shortest path to a given destination vertex t from each vertex v.
• Single-pair shortest-path problem
• Given two vertices, find a shortest path between them. Solution to single-
source problem solves this problem efficiently, too.
• All-pairs shortest-paths problem
• Find a shortest path from u to v for every pair of vertices u and v
All pairs shortest-paths
All pairs shortest-paths
Problem Given directed graph G = (V, E), with weight on each edge e is
w(e),
for each pair of vertices u, v of V, find the shortest path from u
to v.

Input: weight matrix

Output matrix: element at row u column v is the length of the

shortest path from u to v.

Allow negative-weight edge

Assumption: None negative-length cycle

41
Example
Input
Weight matrix Wnxn = (w)ij where
2 0 if i = j
3 4
w ij = w (i, j) if i ¹ j & (i, j) Î E
8 ¥ otherwise
1 3
7 1
-4 2 -5
0 3 8 ¥ -4
5 4
6 ¥ 0 ¥ 1 7
W5x5 = ¥ 4 0 ¥ ¥
2 ¥ -5 0 ¥
¥ ¥ ¥ 6 0

42
Example
Output
Matrix: element at row u column v is the length of the shortest path from u to v.

2
3 4 Shortest path from 1 to 2 : 1- 5 - 4 - 3 - 2

1 8 3 =– 4+6–5+4
0 1 -3 2 -4
7 1
-4 2 -5 3 0 -4 1 -1
7 4 0 5 3
5 4 2 -1 -5 0 -2
6 8 5 1 6 0

Shortest path from 5 to 1 : 5 - 4 - 1

43
Floyd-Warshall algorithm

d ij(m) = length of the shortest path from i to j using intermediate

vertices in the vertex set {1, 2, …, m}.

i £m £m ... £m j

(n)
Graph with n vertices {1,2,..,n} à length of the shortest path from i to j is d
ij

44
Recursive formula computed d(h)

(0)
d ij = wij i j

d (h) = min ( d (h-1), d (h-1)+ d (h-1) ) if h ³ 1

ij ij ih hj

(h-1)
dih h dhj(h-1)

i j
(h-1)
dij
45
Floyd-Warshall algorithm

void Floyd-Warshall(n, W)
{
D(0) ¬ W
Path going through only intermediate
for (k =1; k<=n; k++) vertices selected from {1,2,..,k}

for (i=1;i<=n;i++)
All pairs (i, j)
for (j=1;j<=n;j++)
($) $&' $&' $&'
𝑑!" ← min(𝑑!" , 𝑑!$ + 𝑑$" )
return D(n) ;
}

Running time Q(n3) !

46
Build the shortest path
(k) (k)
Predecessor matrix P = (p ij ) :
(k)
i p ij j

Shortest path from i to j going through intermediate vertices only selected from {1, 2, …, k}.

(0)
i, if (i, j) Î E
pij =
Nil, if (i, j) Ï E

(k-1) (k-1) (k-1) (k-1)

i (k) p if dij £ d ik + d kj
pij = ij
j (k-1)
k pkj otherwise

47
Example 1: Find shortest path between every pairs of vertices

Input
1 17 2
Weight matrix Wnxn = (w)ij where
13 5 4 19 0 if i = j
3 1 w ij = w (i, j) if i ¹ j & (i, j) Î E
7 ¥ otherwise
3 4

0 17 5 ¥
¥ 0 ¥ 1
W4x4 = 13 4 0 7
3 19 ¥ 0
 1 17 2 D(0) ¬ W
(0) i, nếu (i, j) Î E
13 5 4 19 1 p =
ij Nil, nếu (i, j) Ï E
3
3 7 4
0 17 5 ¥ Nil 1 1 Nil
(0)
D(0) ¥ 0 ¥ 1 P Nil Nil Nil 2
13 4 0 7 3 3 Nil 3
3 19 ¥ 0 4 4 Nil Nil

Could use 1 as intermediate vertex:

(1)
0 17 5 ¥ (1) Nil 1 1 Nil
D ¥ 0 ¥ 1 P Nil Nil Nil 2
13 4 0 7 3 3 Nil 3
4 4 1 Nil
3 19 8 0
 1 17 2 D(0) ¬ W
(0) i, nếu (i, j) Î E
13 5 4 19 1 p =
ij Nil, nếu (i, j) Ï E
3
3 7 4
0 17 5 ¥ Nil 1 1 Nil
(1)
D(1) ¥ 0 ¥ 1 P Nil Nil Nil 2
13 4 0 7 3 3 Nil 3
3 19 8 0 4 4 1 Nil

Could use 2 as intermediate vertex:

(2) 0 17 5 18 Nil 1 1 2
D (2) Nil Nil Nil 2
¥ 0 ¥ 1 P
3 3 Nil 2
13 4 0 5
4 4 1 Nil
3 19 8 0
 1 17 2 D(0) ¬ W
(0) i, nếu (i, j) Î E
13 5 4 19 1 p =
ij Nil, nếu (i, j) Ï E
3
3 7 4

(2) 0 17 5 18 (2) Nil 1 1 Nil

D P Nil Nil Nil 2
¥ 0 ¥ 1
13 4 0 5 3 3 Nil 2
3 19 8 0 4 4 1 Nil

Could use 3 as intermediate vertex:

0 9 5 10 (3) Nil 3 1 2
P
D(3) ¥ 0 ¥ 1 Nil Nil Nil 2
3 3 Nil 2
13 4 0 5 4 3 1 Nil
3 12 8 0
 1 17 2 D(0) ¬ W
(0) i, nếu (i, j) Î E
13 5 4 19 1 p =
ij Nil, nếu (i, j) Ï E
3
3 7 4
0 9 5 10 (3) Nil 3 1 2
(3) P Nil Nil Nil 2
D ¥ 0 ¥ 1
3 3 Nil 2
13 4 0 5 4 3 1 Nil
3 12 8 0

Could use 4 as intermediate vertex:

0 9 5 10 (4) Nil 3 1 2
P
D(4) 4 0 9 1
4 Nil 1 2
4 3 Nil 2
8 4 0 5 4 3 1 Nil
3 12 8 0
 0 9 5 10 Nil 3 1 2
1 17 2 D(4) 4 0 9 1 P (4) 4 Nil 1 2
8 4 0 5 4 3 Nil 2
13 5 4 19 1 4 3 1 Nil
3 12 8 0
3
3 7 4

Shortest path from 1 to 2: 1 3 2

Length = 9