Priority Queues and Heaps

Page 1

Berner Fachhochschule - Technik und Informatik

Algorithms and Data Structures

Priority Queues and Heaps
Dr. Rolf Haenni Fall 2008

Berner Fachhochschule Technik und Informatik

Rolf Haenni Algorithms and Data Structures

Priority Queues and Heaps

Page 2

Outline

Priority Queues Heaps Heap-Based Priority Queues Bottom-Up Heap Construction

Berner Fachhochschule Technik und Informatik

Rolf Haenni Algorithms and Data Structures

Priority Queues and Heaps

Priority Queues

Page 3

Outline

Priority Queues Heaps Heap-Based Priority Queues Bottom-Up Heap Construction

Berner Fachhochschule Technik und Informatik

Rolf Haenni Algorithms and Data Structures

Priority Queues and Heaps

Priority Queues

Page 4

Priority Queue ADT

A priority queue stores a collection of items An item is a pair (key , element) Characteristic operations:
insertItem(k,e): inserts an item with key k and element e removeMin(): removes the item with the smallest key and returns its element minKey(): returns the smallest key of an item (no removal) minElement(): returns the element of an item with smallest key (no removal)

General operations:
size(): returns the number of items isEmpty(): indicates whether the priority queue is empty

Two distinct items in a priority queue can have the same key
Berner Fachhochschule Technik und Informatik Rolf Haenni Algorithms and Data Structures

Priority Queues and Heaps

Priority Queues

Page 5

Total Order
Keys in a priority queue can be arbitrary objects on which a total order is dened Mathematically, a total order is a binary relation dened on a set K which satises three properties:
Totality: x y or y x, for all x, y K Antisymmetry: x y and y x implies x = y , for all x, y K Transitivity: x y and y z implies x z, for all x, y , z K

Totality implies Reexivity: x

x, for all x K

Examples: or for R, alphabetical or reverse alphabetical order for {A, . . . , Z }, lexicographical order for {A, . . . , Z } , etc.

Berner Fachhochschule Technik und Informatik

Rolf Haenni Algorithms and Data Structures

Priority Queues and Heaps

Priority Queues

Page 6

Comparator ADT
A comparator encapsulates the action of comparing two keys according to a given total order relation A generic priority queue uses an auxiliary comparator (passed as a parameter to the constructor) When the priority queue needs to compare two keys, it uses its comparator Operations (all with Boolean return type):
isLessThan(x,y) isLessThanOrEqualTo(x,y) isEqualTo(x,y) isGreaterThan(x,y) isGreaterThanOrEqualTo(x,y) isComparable(x)
Berner Fachhochschule Technik und Informatik Rolf Haenni Algorithms and Data Structures

Priority Queues and Heaps

Priority Queues

Page 7

Sequence-Based Priority Queue

There are two ways to implement a priority queue with a sequence (list or vector) Using an unsorted sequence
insertItem(k,e) runs in O(1) time, since we can insert the item at the beginning of the sequence removeMin(), minKey(), minElement() run in O(n) time since we have to traverse the entire sequence to nd the smallest key

Using a sorted sequence

insertItem(k,e) runs in O(n) time, since we have to nd the place where to insert the item removeMin(), minKey(), minElement() run in O(1) time since the smallest key is at the beginning or end of the sequence
Berner Fachhochschule Technik und Informatik Rolf Haenni Algorithms and Data Structures

Priority Queues and Heaps

Priority Queues

Page 8

UML Diagram: Composition

<<interface>> <<interface>>

Comparator
isLessThan(x,y) isEqualTo(x,y) etc.

BasicSequence
isEmpty() size()

<<interface>>

<<interface>>

<<interface>>

PriorityQueue
insertItem(k,e) removeMin() minKey() minElement() etc.

List
etc.

Position

SortedPQ
list: SinglyLinkedList C: Comparator

UnsortedPQ
list: SinglyLinkedList C: Comparator

SinglyLinkedList
rst: ListNode n: Integer

ListNode
element: Object next: ListNode

Berner Fachhochschule Technik und Informatik

Rolf Haenni Algorithms and Data Structures

Priority Queues and Heaps

Priority Queues

Page 9

UML Diagram: Inheritance

<<interface>> <<interface>>

Comparator
isLessThan(x,y) isEqualTo(x,y) etc.

BasicSequence
isEmpty() size()

<<interface>>

<<interface>>

<<interface>>

PriorityQueue
insertItem(k,e) removeMin() minKey() minElement() etc.

List
etc.

Position

SinglyLinkedList
rst: ListNode n: Integer

ListNode
element: Object next: ListNode

SortedPQ
C: Comparator

UnsortedPQ
C: Comparator

Berner Fachhochschule Technik und Informatik

Rolf Haenni Algorithms and Data Structures

Priority Queues and Heaps

Priority Queues

Page 10

Sorting with a Priority Queue

We can use a priority queue to sort a collection of comparable elements
Insert the elements one by one with a series of insertItem(e,e) operations Remove the elements in sorted order with a series of removeMin() operations

Depending on how the priority queue is implemented (sorted or unsorted), this leads to two well-known sorting algorithms
Selection-Sort Insertion-Sort

Berner Fachhochschule Technik und Informatik

Rolf Haenni Algorithms and Data Structures

Priority Queues and Heaps

Priority Queues

Page 11

PQ-Sort Algorithm
Solution in pseudo-code: Algorithm PQSort(S,C ) P new priority queue with comparator C while not [Link]() do e [Link]([Link]()) [Link](e,e) while not [Link]() do e [Link]() [Link](e) return S

Berner Fachhochschule Technik und Informatik

Rolf Haenni Algorithms and Data Structures

Priority Queues and Heaps

Priority Queues

Page 12

Selection-Sort
Selection-sort is the variation of PQ-sort where the priority queue is implemented with an unsorted sequence The main task is to select the elements from the unsorted sequence in the right order Running time analysis:
Inserting the elements into the priority queue with n insertItem(e,e) operations runs in O(n) time Removing the elements in sorted order from the priority queue with n removeMin() operations takes time proportional to n + . . . + 2 + 1, and thus runs in O(n2 ) time

Selection-sort runs in O(n2 ) time

Berner Fachhochschule Technik und Informatik

Rolf Haenni Algorithms and Data Structures

Priority Queues and Heaps

Priority Queues

Page 13

Insertion-Sort
Selection-sort is the variation of PQ-sort where the priority queue is implemented with an sorted sequence The main task is to insert the elements at the right place Running time analysis:
Inserting the elements into the priority queue with n insertItem(e,e) operations takes time proportional to 1 + 2 + . . . + n, and thus runs in O(n2 ) time Removing the elements in sorted order from the priority queue with n removeMin() operations runs in O(n) time

In general (worst case), insertion-sort runs in O(n2 ) time If the input sequence is in reverse order (best case), it runs in O(n) time

Berner Fachhochschule Technik und Informatik

Rolf Haenni Algorithms and Data Structures

Priority Queues and Heaps

Priority Queues

Page 14

In-Place Sorting
Instead of using external data structures, we can implement insertion- and selection-sort in-place A portion of the input sequence itself serves as priority queue For in-place insertion-sort we
keep the initial portion of the sequence sorted use swapElements(p,q) instead of modifying the sequence

5 5 4 2 2 1

4 4 5 4 3 2

2 2 2 5 4 3

3 3 3 3 5 4

1 1 1 1 1 5

Berner Fachhochschule Technik und Informatik

Rolf Haenni Algorithms and Data Structures

Priority Queues and Heaps

Heaps

Page 15

Outline

Priority Queues Heaps Heap-Based Priority Queues Bottom-Up Heap Construction

Berner Fachhochschule Technik und Informatik

Rolf Haenni Algorithms and Data Structures

Priority Queues and Heaps

Heaps

Page 16

The Heap ADT

A heap is a specialized binary tree storing keys at its nodes (positions) and satisfying the following properties:
Heap-Order: key (p) key (parent(p)), for every node p other than the root Completeness: let h be the height of the tree 1) there are 2i nodes of depth i for i = 0, . . . , h 1 2) at depth h, nodes are lled up from the left

The last node of a heap is the rightmost node of depth h Heap operations:
insertKey(k): inserts a key k into the heap removeMin(): removes smallest key and returns it minKey(): returns the smallest key (no removal)

Berner Fachhochschule Technik und Informatik

Rolf Haenni Algorithms and Data Structures

Priority Queues and Heaps

Heaps

Page 17

Heap Example
Depth 2 5 9 13 10 9 7 8 Last node 12 11 14 9 6 8 11 0 1 2 3 4

17 15 12

A heap storing n keys has height O(log n)

Berner Fachhochschule Technik und Informatik

Rolf Haenni Algorithms and Data Structures

Priority Queues and Heaps

Heaps

Page 18

Insertion to a Heap

The insertion of a key k consists of 3 steps:

Add a new node q (the new last node) Store k at q Restore the heap order property (discussed next)

5 9 7 q

2 5 9 7 1 6

Berner Fachhochschule Technik und Informatik

Rolf Haenni Algorithms and Data Structures

Priority Queues and Heaps

Heaps

Page 19

Upheap
Algorithm upheap restores the heap-order property
Swap k along an upward path from the insertion node Stop when k reaches the root or a node whose parent has a key smaller than or equal to k Since the height of the heap is O(log n), upheap runs in O(log n) time
5 9 7 6

2 1

1 5 9 7 6 2

insertKey(k) runs in O(log n) time

Berner Fachhochschule Technik und Informatik

Rolf Haenni Algorithms and Data Structures

Priority Queues and Heaps

Heaps

Page 20

Removal from a Heap

The removal algorithm consists of 4 steps:

Return the key of the root Replace the root key with the key k of the last node Delete the last node Restore the heap order property (discussed next)
9

5 7

6 Last node

7 5 9 6

Berner Fachhochschule Technik und Informatik

Rolf Haenni Algorithms and Data Structures

Priority Queues and Heaps

Heaps

Page 21

Downheap
Algorithm downheap restores the heap-order property
Swap k along an downward path from the root (always with the smaller key of its children) Stop when k reaches a leaf or a node whose children have keys greater than or equal to k Since the height of the heap is O(log n), downheap runs in O(log n) time
5 9

7 6

5 7 9 6

removeMin() runs in O(log n) time

Berner Fachhochschule Technik und Informatik Rolf Haenni Algorithms and Data Structures

Priority Queues and Heaps

Heaps

Page 22

Finding the Insertion Node

Starting from the last node, the insertion node can be found by traversing a path of O(log n) nodes
If the last node is a left child, return the parent node While the current node is a right child, go to the parent node If the current node is a left child, go to the right child While the current node is internal, go to the left child
2 5 9 13 10 9 7 8 11 Last node 6 8 Insertion node

Berner Fachhochschule Technik und Informatik

Rolf Haenni Algorithms and Data Structures

Priority Queues and Heaps

Heaps

Page 23

Array-Based Heap Implementation

We can represent a heap with n keys directly by means of an array of length N > n
By-pass the binary tree ADT No explicit position ADT needed

Idea similar to array-based implementation of binary trees

The The The The array cell at index 0 is unused root of the heap is at index 1 left child of the node at index i is at index 2i right child of the node at index i is at index 2i + 1

The insertion operations corresponds to inserting at index n + 1 and thus runs in O(1) time

Berner Fachhochschule Technik und Informatik

Rolf Haenni Algorithms and Data Structures

Priority Queues and Heaps

Heaps

Page 24

Array-Based Heap: Example

2 5 9 13 10 9 7 8 11 Last node 6 8

2 5 6 9 7 11 8 13 10 9 8
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14

Berner Fachhochschule Technik und Informatik

Rolf Haenni Algorithms and Data Structures

Priority Queues and Heaps

Heap-Based Priority Queues

Page 25

Outline

Priority Queues Heaps Heap-Based Priority Queues Bottom-Up Heap Construction

Berner Fachhochschule Technik und Informatik

Rolf Haenni Algorithms and Data Structures

Priority Queues and Heaps

Heap-Based Priority Queues

Page 26

Heap-Based Priority Queues

We can use a heap to implement a priority queue by storing a pair item = (key , element) at each node of the heap Running times for dierent implementations
Operation size, isEmpty minElement, minKey insertItem removeMin PQ-Sort Unsorted Sequence 1 n 1 n n2 Sorted Sequence 1 1 n 1 n2 Heap 1 1 log n log n n log n

In the long run, the heap-based implementation beats any sequence-based implementation

Berner Fachhochschule Technik und Informatik

Rolf Haenni Algorithms and Data Structures

Priority Queues and Heaps

Heap-Based Priority Queues

Page 27

UML Diagram: Composition

<<interface>> <<interface>> <<interface>>

Tree
etc.

Comparator
isLessThan(x,y) isEqualTo(x,y) etc.

BasicSequence
isEmpty() size()

<<interface>>

<<interface>>

<<interface>>

BinaryTree
etc.

Heap
insertKey(k) removeMin() minKey()

PriorityQueue
insertItem(k,e) removeMin() minKey() minElement()

ArrayBinaryTree
T: Array n: Integer

BinaryTreeHeap
T: ArrayBinaryTree C: Comparator

ArrayHeap
A: Array C: Comparator

HeapPQ
H: ArrayHeap C: Comparator

Berner Fachhochschule Technik und Informatik

Rolf Haenni Algorithms and Data Structures

Priority Queues and Heaps

Heap-Based Priority Queues

Page 28

Heap-Sort
Heap-sort is the variation of PQ-sort where the priority queue is implemented with a heap Running time analysis:
Inserting the elements into the priority queue with n insertItem(e,e) operations runs in O(n log n) time Removing the elements in sorted order from the priority queue with n removeMin() operations runs in O(n log n) time

In general (worst case), heap-sort runs in O(n log n) time For large n, heap-sort is much faster than quadratic sorting algorithms such as insertion-sort or selection-sort

Berner Fachhochschule Technik und Informatik

Rolf Haenni Algorithms and Data Structures

Priority Queues and Heaps

Heap-Based Priority Queues

Page 29

In-Place Heap-Sort
To implement Heap-Sort in-place, use the front of the original array to implement the heap
Step 1: Use the reverse comparator (e.g. instead of ) to build up the heap (by swapping elements) Step 2: Iteratively remove the maximum element from the heap and insert it in front of the sequence
Phase 1: 4 7 2 5 3 4 7 2 5 3 4 4 7 5 4 Phase 2: 3 2 3 5 4 2 3 7 4 5 2 3 4 2 2 3 2 4 5 7 4 3 2 5 7 3 2 2 3 4 5 7 2 3 4 5 7 7 4 2 5 3 7 4 7 4 2 5 3 7 2 4 5 7 5 2 4 3 7 2 4 5 3 7 5 2 4 3 7 2

7 5 2 4 3

Berner Fachhochschule Technik und Informatik

Rolf Haenni Algorithms and Data Structures

Priority Queues and Heaps

Bottom-Up Heap Construction

Page 30

Outline

Priority Queues Heaps Heap-Based Priority Queues Bottom-Up Heap Construction

Berner Fachhochschule Technik und Informatik

Rolf Haenni Algorithms and Data Structures

Priority Queues and Heaps

Bottom-Up Heap Construction

Page 31

Bottom-up Heap Construction

We can construct a heap storing n = 2h 1 keys using a recursive bottom-up construction with h 1 phases In phase i, pairs of heaps with 2i 1 keys are merged into heaps with 2i+1 1 keys Merging two heaps (and a new key k):
Create a new heap with the root node storing k and with the two heaps as subtrees Perform downheap to restore the heap-order property
2i+11 2 1
i

2 1

Berner Fachhochschule Technik und Informatik

Rolf Haenni Algorithms and Data Structures

Priority Queues and Heaps

Bottom-Up Heap Construction

Page 32

Recursive Algorithm in Pseudo-Code

Algorithm bottomUpHeap(S) // sequence of keys of length 2h1 if S. isEmpty() then return new heap else k S. elemAtRank(0) S. removeAtRank(0) H1 bottomUpHeap(firstHalf(S)) // 1st recursive call H2 bottomUpHeap(secondHalf(S)) // 2nd recursive call return mergeHeaps(k, H1 , H2 ) // merge the results

In the general case, i.e. if the sequence is not of size n = 2h 1, we can adapt the splitting of the sequence accordingly

Berner Fachhochschule Technik und Informatik

Rolf Haenni Algorithms and Data Structures

Priority Queues and Heaps

Bottom-Up Heap Construction

Page 33

Example: Phase 1

16

15

12

23

20

25 16 15 4

5 12 6

11 9 23

27 20

15 16 25 5

4 12 11

6 9 23

20 27

Berner Fachhochschule Technik und Informatik

Rolf Haenni Algorithms and Data Structures

Priority Queues and Heaps

Bottom-Up Heap Construction

Page 34

Example: Phase 2

15 16 25 5

4 12 11

6 9 23

20 27

7 15 16 25 5 4 12 11 6 9

8 20 23 27

4 15 16 25 7 5 12 11 8 9

6 20 23 27

Berner Fachhochschule Technik und Informatik

Rolf Haenni Algorithms and Data Structures

Priority Queues and Heaps

Bottom-Up Heap Construction

Page 35

Example: Phase 3
4 15 16 25 7 5 12 11 8 9 23 6 20 27

10 4 15 16 25 7 5 12 11 8 9 23 6 20 27

4 5 15 16 25 10 7 12 11 8 9 23 6 20 27

Berner Fachhochschule Technik und Informatik

Rolf Haenni Algorithms and Data Structures

Priority Queues and Heaps

Bottom-Up Heap Construction

Page 36

Complexity Analysis
The recursive BottomUpHeap algorithm decomposes a problem of size n into two problems of size n1 2
Rule D2 with q = r = 2 (see Topic 2, Page 31)

The running time f (n) of each recursive step is O(log n)

with an array-based sequence implementation, firstHalf() and secondHalf() run in O(1) time the necessary downheap in mergeHeaps runs in O(log n) time

Apply the second column of D2, i.e. bottom-up heap construction runs in O(n) Speeds up the heap construction (Phase 1) of the Heap-Sort algorithm, but not Phase 2

Berner Fachhochschule Technik und Informatik

Rolf Haenni Algorithms and Data Structures