1

MICT-1201: Cloud Computing

Lecture 6

Lecture 6 Dr. Sajeeb Saha, Dept. of CSE, JnU

 IaaS Cloud Architectures 2

• Introduction to IaaS • Software virtualization

– Hypervisors
• Hardware virtualization
• KVM
• CPU • Xen
• Memory • VirtualBox
• I/O – Full Virtualization
• Network – Para Virtualization
– Host OS Virtualization
– Container-based Virtualization

Lecture 6 Dr. Sajeeb Saha, Dept. of CSE, JnU

 3

Lecture 6 Dr. Sajeeb Saha, Dept. of CSE, JnU

 Overview 4

• What is the problems in conventional case ?

– Companies IT investment for peak capacity
– Lack of agility for IT infrastructure
– IT maintain cost for every company
– Usually suffered from hardware failure risk
– …etc

• These IT complexities force company back !!

Lecture 6 Dr. Sajeeb Saha, Dept. of CSE, JnU

 Overview 5

• How to solve these problem ?

– Let’s consider some kind of out-sourcing solution
• Somebody will handle on demand capacity for me
• Somebody will handle high available resource for me
• Somebody will handle hardware management for me
• Somebody will handle system performance for me
• Somebody will …

– Frankly, that would be a great solution IF there were “somebody”.

– But who can be this “somebody”, and provide all these services ?

Lecture 6 Dr. Sajeeb Saha, Dept. of CSE, JnU

 Overview 6

• Infrastructure as a Service will be the

salvation.
– IaaS cloud provider takes care of all the IT infrastructure complexities.
– IaaS cloud provider provides all the infrastructure functionalities.
– IaaS cloud provider guarantees qualified infrastructure services.
– IaaS cloud provider charges clients according to their resource usage.

• But, what make all of these happen so

magically ?

Lecture 6 Dr. Sajeeb Saha, Dept. of CSE, JnU

 Virtualization 7

• Assume that you are going to be an IaaS cloud provider.

– Then, what are the problems you are facing?
• Clients will request different operating systems.
• Clients will request different storage sizes.
• Clients will request different network bandwidths.
• Clients will change their requests anytime.
• Clients will …

– Is there any good strategy?

• Allocate a new physical machine for each incomer.
• Prepare a pool of pre-installed machines for different
requests.
• or …

Lecture 6 Dr. Sajeeb Saha, Dept. of CSE, JnU

 Virtualization 8

• What if we allocate a new physical machine for each

incomer ?

I want
Windows 7 I want Linux

Customer A Customer B

I want … I want …

I want Windows
… I wantLinux
…

Lecture 6 Dr. Sajeeb Saha, Dept. of CSE, JnU

 Virtualization 9

• How about preparing a pool of pre-installed

physical machines for all kinds of request ?

I want Mac
OS

Somebody might Somebody might

Somebody may want Somebody may want
want… want…

Windows + Office Windows Server Linux + OpenOffice Linux Server

Lecture 6 Dr. Sajeeb Saha, Dept. of CSE, JnU

 Virtualization 10

• Obviously, neither of previous strategies will work.

• We need more powerful techniques to deal with that.

• Virtualization techniques will help.

– For computation resources
• Virtual Machine technique
– For storage resources
• Virtual Storage technique
– For communication resources
• Virtual Network technique

Lecture 6 Dr. Sajeeb Saha, Dept. of CSE, JnU

 IaaS Cloud Architectures 11

• Local Cloud?
– What resources would we need to do this?
• Compute Servers
• Persistent Storage Servers
• VM Image Server(s)
• Cloud Administrative Server(s)
• Network Infrastructure
– Copper

Lecture 6 Dr. Sajeeb Saha, Dept. of CSE, JnU

 IaaS Main Ideas - Virtualization 12

• Memory
– Virtual Memory and memory management
• Multitasking
– Several processes concurrently running on the same hardware
– Hardware is shared thanks to special OS processes and CPU
extensions
• Virtual Machine
– Concept began with the use of IBM mainframes
– Abandoned with the advent of PC’s, but now used again for cloud
computing
– Even different Instruction Set Architectures (ISAs) and/or OS kernels
– Achieved using a Virtual Machine Monitor (VMM) or Hypervisor

Lecture 6 Dr. Sajeeb Saha, Dept. of CSE, JnU

 Virtualization Overview 13

• What is virtualization ?
– Virtualization is the creation of a virtual (rather than
physical) version of something, such as an operating
system, a server, a storage device or network resources.
– It hides the physical characteristics of a resource from
users, instead showing another abstract resource.
• But, where does virtualization come from ?
– Virtualization is NOT a new idea of computer science.
– Virtualization concept comes from the component
abstraction of system design, and it has been adapted in
many system level.

Lecture 6 Dr. Sajeeb Saha, Dept. of CSE, JnU

 What is virtualization? 14

• Virtualization is the ability to run multiple

operating systems on a single physical system and
share the underlying hardware resources1
• Allows one computer to provide the appearance of
many computers.
• Goals:
– Provide flexibility for users
– Amortize hardware costs
– Isolate completely separate users

1 VMWare white paper, Virtualization Overview

Lecture 6 Dr. Sajeeb Saha, Dept. of CSE, JnU

 Virtualization Architecture 15

• A Virtual machine (VM) is an isolated runtime environment

(guest OS and applications)
• Multiple virtual systems (VMs) can run on a single physical
system
Virtual

Applications Applications Applications

Guest OS Guest OS Guest OS
(Windows) (Linux) (VMware ESX)
Virtual Machine Virtual Machine Virtual Machine

Virtual Machine Manager

Physical

Host OS

Hardware

Lecture 6 Dr. Sajeeb Saha, Dept. of CSE, JnU

 Benefits of Virtualization 16

• Sharing of resources helps cost reduction

• Isolation: Virtual machines are isolated from each other as
if they are physically separated
• Encapsulation: Virtual machines encapsulate a complete
computing environment
• Hardware Independence: Virtual machines run
independently of underlying hardware
• Portability: Virtual machines can be migrated between
different hosts.

Lecture 6 Dr. Sajeeb Saha, Dept. of CSE, JnU

 Native and Hosted VM Systems 17

Guest Guest
Applications Applications
Guest
Applications Guest OS Guest OS

Applications
Nonprivileged
Guest OS VMM VMM
modes
Privileged
OS VMM Host OS Host OS modes

Hardware Hardware Hardware Hardware

Traditional Native User-mode Dual-mode

Uniprocessor VM System Hosted Hosted
System VM System VM System

Lecture 6 Dr. Sajeeb Saha, Dept. of CSE, JnU

 Virtual Machine Monitor 18

• What’s Virtual Machine Monitor (VMM) ?

– VMM or Hypervisor is the software layer providing the
virtualization.
• System architecture :

VM1 VM2 VM3

Lecture 6 Dr. Sajeeb Saha, Dept. of CSE, JnU

 Hypervisor 19

• A hypervisor, a.k.a. a virtual machine manager/monitor

(VMM), or virtualization manager, is a program that allows
multiple operating systems to share a single hardware host.

• Each guest operating system appears to have the host's

processor, memory, and other resources all to itself.

• However, the hypervisor is actually controlling the host

processor and resources, allocating what is needed to each
operating system in turn and making sure that the guest
operating systems (called virtual machines) cannot disrupt
each other.

Lecture 6 Dr. Sajeeb Saha, Dept. of CSE, JnU

 IaaS Main Ideas - Isolation 20

• Through virtualization, workloads are isolated since all

program instructions are fully confined inside a virtual
machine (VM).
• Better reliability and performance is also achieved because
software failures inside one VM do not affect others.

Lecture 6 Dr. Sajeeb Saha, Dept. of CSE, JnU

 IaaS Main Ideas – Application Mobility 21

• Workload migration (application mobility) targets at

facilitating hardware maintenance, load balancing, fault
tolerance and disaster recovery.
• It is done by encapsulating a guest OS state within a VM and
allowing it to be suspended, migrated to a different platform,
and resumed immediately or preserved to be restored at a
later date.
– A VM’s state includes a full disk or partition image, configuration files,
and an image of its RAM.

Lecture 6 Dr. Sajeeb Saha, Dept. of CSE, JnU

 IaaS Cloud Architectures 22

Lecture 6 Dr. Sajeeb Saha, Dept. of CSE, JnU

 IaaS Cloud Architectures 23

• Introduction to IaaS • Software virtualization

– Hypervisors
• Hardware virtualization
• KVM
• CPU • Xen
• Memory • VirtualBox
• I/O – Full Virtualization
• Network – Para Virtualization
– Host OS Virtualization
– Container-based Virtualization

Lecture 6 Dr. Sajeeb Saha, Dept. of CSE, JnU

 CPU Architecture - Modes 24

• Modern CPU status is usually classified as several modes.

• In general, we conceptually divide them into two modes :
– Kernel mode (Ring 0)
• CPU may perform any operation allowed by its architecture, including any
instruction execution, I/O operation, area of memory access, and so on.
– User mode (Ring 1 - 3)
• Ring 1: Reserved for device drivers
• Ring 2: Privileged code such as user programs with I/O access permissions
• Ring 3: Unprivileged code (Most user programs)

Lecture 6 Dr. Sajeeb Saha, Dept. of CSE, JnU

 CPU Architecture 25

• By the classification of CPU modes, we divide instructions into

following types:
– Privileged instructions
• Those instructions that trap (interrupt) if the machine is in
user mode and do not trap if the machine is in kernel mode.
– Sensitive instructions
• Those instructions that interact with hardware, which include
control-sensitive and behavior-sensitive instructions.

Lecture 6 Dr. Sajeeb Saha, Dept. of CSE, JnU

 CPU Architecture - Traps 26

• CPU trap:
– When CPU is running in user mode, some internal or
external events, which need to be handled in kernel
mode, take place.
• System calls
• Hardware interrupts
• Exceptions
– Looks like an internal CPU interrupt since a trap
handler acts like an interrupt handler
• Registers and stack pointers are saved
• Context switch occurs in the CPU to handle the trap

Lecture 6 Dr. Sajeeb Saha, Dept. of CSE, JnU

 CPU Architecture - Traps 27

• Trap types
– System Call
• Invoked by application in user mode.
• For example, application ask OS for system I/O.
– Hardware Interrupts
• Invoked by some hardware events in any mode.
• For example, hardware clock timer trigger event.
– Exception
• Invoked when unexpected error or system malfunction occur.
• For example, execute privilege instructions in user mode.

Lecture 6 Dr. Sajeeb Saha, Dept. of CSE, JnU

 CPU Architecture – Emulation vs. Virtualization 28

• Emulation
– Hardware components and interactions are replaced
with software components.
• Advantages:
– Entire hardware architectures can be created by software interfaces
– It is possible to run isolated software on an emulated server, even software
designed for different hardware architectures.
• Disadvantages:
– Software running in emulated environments have no direct access to hardware
devices.
– Using this technique creates an emulation tax (the processing cycles needed to
emulate the hardware).

Lecture 6 Dr. Sajeeb Saha, Dept. of CSE, JnU

 CPU Architecture – Emulation vs. Virtualization 29

• Virtualization
– VMM or hypervisors have direct access to hardware
devices
• Advantages:
– The virtual machine hosted by the hypervisor can access hardware
allowing performance increases.
• Disadvantages:
– VMM schedulers limit what can be run in virtual machines due to
complexities such as context switching and VMM resource
management.

Lecture 6 Dr. Sajeeb Saha, Dept. of CSE, JnU

 Trap and Emulate Model 30

• If we want CPU virtualization to be efficient, how should

we implement the VMM ?
– We should make guest binaries run on CPU as fast as possible.
– Theoretically speaking, if we can run all guest binaries natively, there
will NO overhead at all.
– But we cannot let guest OS handle everything, VMM should be able to
control all hardware resources.

• Solution :

–Run VMM in kernel mode.

• Then VMM will be able to intercept all trapping events.

Lecture 6 Dr. Sajeeb Saha, Dept. of CSE, JnU

 Trap and Emulate Model 31

• Traditional OS:
– System call:
• CPU will trap to interrupt handler
vector in OS.
• CPU will switch to kernel mode (Ring
0) and execute OS instructions.
– Hardware event:
• Hardware will interrupt CPU
execution, and jump to interrupt
handler in OS.

Lecture 6 Dr. Sajeeb Saha, Dept. of CSE, JnU

 Trap and Emulate Model 32

• VMM and Guest OS:

– System Call
• CPU will trap to interrupt handler vector of VMM.
• VMM jump back into guest OS.
– Hardware Interrupt
• Hardware make CPU trap to interrupt handler of
VMM.
• VMM jump to corresponding interrupt handler of
guest OS.
– Privilege Instruction
• Running privilege instructions in guest OS will be
trapped to VMM for instruction processing.
• After processing the instruction, the VMM jumps
back to guest OS.

Lecture 6 Dr. Sajeeb Saha, Dept. of CSE, JnU

 Context Switch 33

• Steps for VMM to switch active virtual machines:

1. Timer Interrupt in running VM.
2. Context switch to VMM.
3. VMM saves state of running VM.
4. VMM determines next VM to execute.
5. VMM sets timer interrupt.
6. VMM restores state of next VM.
7. VMM sets PC to timer interrupt handler of next VM.
8. Next VM active.

Lecture 6 Dr. Sajeeb Saha, Dept. of CSE, JnU

 System State Management 34

• Virtualizing system state:

– VMM will hold the system states of all virtual
machines in memory.
– When VMM context switches from one virtual
machine to another:
• Write the register values to memory
• Copy the register values of next guest OS to CPU
registers.

Lecture 6 Dr. Sajeeb Saha, Dept. of CSE, JnU

 Memory Virtualization 35

• Memory management in OS
– Traditionally, OS fully controls all physical memory space and provide a
continuous addressing space to each process.
– In server virtualization, VMM should make all virtual machines share the
physical memory space without knowing the fact.
• Goals of memory virtualization :
– Address Translation
• Control memory mapping techniques that accesses translation tables in main memory.
– Memory Protection
• Define specific memory access permissions for VMs.
– Access Attribute
• Define attribute and type of memory to be accessed.

Lecture 6 Dr. Sajeeb Saha, Dept. of CSE, JnU

 Memory Virtualization 36

• Not only virtual memory

• Hardware support
– e.g., x86 MMU and TLB
• To run multiple virtual machines on a single system, another level of
memory virtualization is required.
• The VMM is responsible for mapping guest physical memory to the
actual machine memory, and it uses shadow page tables to accelerate
the mappings.

VM1 VM2

Process 1 Process 2 Process 1 Process 2

Virtual memory

Physical memory

Machine memory

Lecture 6 Dr. Sajeeb Saha, Dept. of CSE, JnU

 Memory Architecture 37

• Memory Management Unit (MMU)

– A computer hardware component responsible for handling accesses to memory
requested by the CPU.
– Its functions include translation of virtual addresses to physical addresses,
memory protection, cache control, etc.
– Most modern CPUs include the MMU.

Lecture 6 Dr. Sajeeb Saha, Dept. of CSE, JnU

 Memory Architecture - Address Translation 38

• How to locate the physical address?

– Search impractical (too many pages)
• A page table is a data structure which contains the
mapping of virtual pages to physical addresses
– Usually located in fast CPU caches, special registers, or main
memory
• Each process running in the system has its own page
table
– How to access this data quickly?

Lecture 6 Dr. Sajeeb Saha, Dept. of CSE, JnU

 Memory Architecture - Translation 39

Lookaside Buffer
• TLB is a small cache (on the CPU) that stores recent
translations of virtual memory to physical addresses
for faster retrieval.
• When a virtual memory address is referenced by a
program, the search starts in the CPU.
– First, L1 and L2 caches are checked.
• If the required data is not in these very fast caches, the
system has to look up the data’s physical address.
• At this point, TLB is checked for a quick reference to the
location in physical memory.

Lecture 6 Dr. Sajeeb Saha, Dept. of CSE, JnU

 Memory Architecture - Translation 40

Lookaside Buffer
• The TLB is a small cache of the most recent virtual-
physical mappings.

Lecture 6 Dr. Sajeeb Saha, Dept. of CSE, JnU

 Memory Architecture – Virtualization 41

Techniques
• The performance drop of memory access in Guest OSs can
be unbearable. VMM needs optimization techniques for
data access.

• Accessing memory page tables:

– Translation Lookup Buffer not directly accessible by VM, only
accessible through VMM.
– Page tables in each VM must be accurately mapped to the memory
systems of the host.
• Overhead of memory subsystems and address translations can lead to
dramatic performance decrease.

Lecture 6 Dr. Sajeeb Saha, Dept. of CSE, JnU

 Memory Architecture – Virtualization 42

Techniques
• VMM Shadow Page Table:
– Maps virtual pages used by VMs to actual pages regulated by VMM.
– VMM maps page table addresses to addresses assigned by host
system.
• Increases performance by decreasing translation and lookup time.
• Deduplication:
– Virtual pages containing the same content is shared among VMs
• Memory Balloon:
– Module loaded into VMs as a pseudo device driver that communicates
with the VMM.
• Inflates when memory usage percentage is high.
• Deflates when memory usage percentage is low.

Lecture 6 Dr. Sajeeb Saha, Dept. of CSE, JnU

 Memory Architecture - Direct Memory 43

Access
• DMA allows certain hardware subsystems within the computer to
access system memory for reading and/or writing independently
of the central processing unit.
– DMA increases performance for some data fetch/decode operations in the
VMs.
• Two types of DMA:
– Synchronous DMA
• The DMA operation is caused by software.
• For example, sound card driver may trigger DMA operation to play music.
– Asynchronous DMA
• The DMA operation is caused by devices (hardware).
• For example, network card use DMA operation to load data into memory and interrupt
CPU for further manipulation.

Lecture 6 Dr. Sajeeb Saha, Dept. of CSE, JnU

 I/O Virtualization 44

• Goal :
– Share or create I/O devices for virtual machines.
• Two types of I/O subsystem architecture:
– Port Mapped I/O
• Uses CPU instructions specifically for performing I/O.
– Memory Mapped I/O (MMIO)
• Allows reading/writing to I/O devices in the same way
as reading/writing to system memory.

Lecture 6 Dr. Sajeeb Saha, Dept. of CSE, JnU

 Device and I/O Virtualization 45

• VMM supports all device/IO drivers

• Physically/virtually existed

Source: VMware white paper, “Understanding Full Virtualization, Paravirtualization, and Hardware Assist”

Lecture 6 Dr. Sajeeb Saha, Dept. of CSE, JnU

 Port Mapped I/O 46

• I/O devices are mapped into a separate address

space
– I/O devices have a separate address space from general
memory
• Accomplished by an extra I/O pin on the CPU's physical interface
• Or bus dedicated to I/O.
– Generally found on Intel microprocessors
• Pros & Cons
– Pros
• Less logic is needed to decode a discrete address.
• Benefits for CPUs with limited addressing capability.
– Cons
• More instructions are required to accomplish the same task.
• I/O addressing space size is not flexible.

Lecture 6 Dr. Sajeeb Saha, Dept. of CSE, JnU

 Memory Mapped I/O 47

• I/O devices are mapped into the system memory map

along with RAM and ROM.
– To access a hardware device, simply read or write to
those 'special' addresses using the normal memory access
instructions.

• Pros & Cons

– Pros
• Instructions which can access memory can be used to operate an I/O
device.
• Control I/O devices with fewer instructions.
– Cons
• Physical memory addressing space must be shared with IO devices.
• Generally not accessible by software applications as I/O communication
is restricted to device drivers.

Lecture 6 Dr. Sajeeb Saha, Dept. of CSE, JnU

 I/O Virtualization 48

• Implementation Layers:
– System call
• The interface between applications
and guest OS.
– Driver call
• The interface between guest OS and
I/O device drivers.
– I/O operation request
• The interface between I/O device
driver of guest OS and virtualized
hardware ( in VMM ).

Lecture 6 Dr. Sajeeb Saha, Dept. of CSE, JnU

 Network Virtualization 49

• The process of combining hardware and software network

resources and network functionality into a single, software-
based administrative entity

• Provides network-like functionality to the software

containers on a single system.

Lecture 6 Dr. Sajeeb Saha, Dept. of CSE, JnU

 Network Virtualization 50

• Desirable properties of network virtualization:

– Scalability
• Easy to extend resources in need such as bandwidth and IP/MAC
addresses
• Administrator can dynamically create or delete virtual network
connections
– Resilience
• Recover from failure, failovers
• Virtual network can automatically redirect packets by redundant links
– Security
• Increased path isolation and user segmentation
• Virtual network should work with firewall software
– Availability
• Access network (internal and external) resources anytime

Lecture 6 Dr. Sajeeb Saha, Dept. of CSE, JnU

 Network Virtualization 51

• Network virtualization in different layers:

– Layer 1 (Physical Layer)
• Hypervisor usually do not need to emulate the physical layer.
– Layer 2 (Link Layer)
• Implement virtual L2 network devices, such as switch, in
hypervisor.
• Example: Linux bridge.
– Layer 3 (Network Layer)
• Implement virtual L3 network devices, such as router, in
hypervisor.
– Layer 4 or higher (Application and Transport Layers)
• Layer 4 or higher is usually implemented in guest OS.
• Applications should determine how to communicate.

Lecture 6 Dr. Sajeeb Saha, Dept. of CSE, JnU

 Network Virtualization 52

• Network architecture:
– Bridge (Virtual Switch)
• Make virtual machines on one node share physical NICs.
– DHCP
• Map virtual MAC addresses of VMs to private IPs in the LAN.
– NAT
• Forward the packages to public network (WAN).
– IP/MAC mapping table
• IP addresses are assigned by DHCP service.
• MAC addresses are assigned by hypervisor.
• This mapping table is maintained by the cloud middleware

Lecture 6 Dr. Sajeeb Saha, Dept. of CSE, JnU

 References 53

• James Smith and Ravi Nair, “Virtual Machines: Versatile Platforms for
Systems and Processors”.
• Xen. [Link]
• Kernel-based Virtual Machine (KVM). [Link]
[Link]/page/Main_Page
• From Wikipedia, the free encyclopedia.

Lecture 6 Dr. Sajeeb Saha, Dept. of CSE, JnU

 54

Lecture 6 Dr. Sajeeb Saha, Dept. of CSE, JnU