Understanding Application Layer Protocols
Understanding Application Layer Protocols
Application Layer
Principles of Network Applications
• E-mail • Internet telephone
• Web • Real-time video conference
• Instant messaging • Massive parallel computing
• Remote login
• P2P file sharing
• Multi-user network games
• Streaming stored video clips
2: Application Layer 2
Creating a network app
Write programs that application
transport
network
– run on different end systems data link
physical
and
– communicate over a network.
– e.g., Web: Web server software
communicates with browser
software
No software written for devices
in network core application
application
transport
– Network core devices do not
transport network
network data link
data link
function at app layer physical
physical
2: Application Layer 3
Application architectures
• Client-server
• Peer-to-peer (P2P)
• Hybrid of client-server and P2P
2: Application Layer 4
Client-server archicture
server:
– always-on host
– permanent IP address
– server farms for scaling
clients:
– communicate with server
– may be intermittently
connected
– may have dynamic IP
addresses
– do not communicate directly
with each other
Examples?
2: Application Layer 5
Pure P2P architecture
• no always on server
• arbitrary end systems
directly communicate
• peers are intermittently
connected and change IP
addresses
• example: Gnutella
Highly scalable
2: Application Layer 7
Processes communicating
Process: program running Client process: process that
within a host. initiates communication
• within same host, two Server process: process that
processes communicate waits to be contacted
using inter-process
communication (defined by
OS).
• processes in different hosts Note: applications with P2P
architectures have client processes &
communicate by server processes
exchanging messages
2: Application Layer 8
Sockets
host or host or
• process sends/receives server server
messages to/from its socket
controlled by
• socket analogous to door app developer
process process
– sending process shoves
message out door socket socket
– sending process relies on TCP with TCP with
buffers, Internet buffers,
transport infrastructure on
variables variables
other side of door which brings
message to socket at receiving
process controlled
by OS
API: (1) choice of transport protocol; (2) ability to fix a few parameters (lots more
on this later)
2: Application Layer 9
Port Numbers
Port = 80 Port = 25
TCP
IP = [Link]
2: Application Layer 10
App-layer protocol defines
• Types of messages Public-domain protocols:
exchanged, eg, request & • defined in RFCs
response messages
• Syntax of message types: • allows for interoperability
what fields in messages & • eg, HTTP, SMTP
how fields are delineated Proprietary protocols:
• Semantics of the fields, ie, • eg, KaZaA
meaning of information in
fields
• Rules for when and how
processes send & respond
to messages
2: Application Layer 11
Applications and App-Layer Protocols
UI
HTTP
Web Server
Web Browser
File
HTTP … Access
2: Application Layer 12
What transport service does an app need?
Data loss Bandwidth
some apps (e.g., multimedia)
• some apps (e.g., audio) can require minimum amount of
tolerate some loss bandwidth to be “effective”
• other apps (e.g., file transfer, other apps (“elastic apps”) make
telnet) require 100% reliable use of whatever bandwidth they
get
data transfer
Timing
• some apps (e.g., Internet
telephony, interactive
games) require low delay
to be “effective”
2: Application Layer 13
Transport service requirements of common apps
2: Application Layer 14
Internet transport protocols services
2: Application Layer 15
Internet apps: application, transport protocols
Application Underlying
Application layer protocol transport protocol
2: Application Layer 16
Web and HTTP
First some jargon
• Web page consists of objects
• Object can be HTML file, JPEG image, Java applet, audio file,…
• Web page consists of base HTML-file which includes several
referenced objects
• Each object is addressable by a URL
• Example URL:
[Link]/someDept/[Link]
2: Application Layer 17
HTTP overview
2: Application Layer 18
HTTP overview (continued)
Uses TCP: HTTP is “stateless”
• client initiates TCP connection • server maintains no
(creates socket) to server, port information about past
80 client requests
• server accepts TCP connection
aside
from client Protocols that maintain “state”
• HTTP messages (application- are complex!
layer protocol messages) past history (state) must be
exchanged between browser maintained
(HTTP client) and Web server if server/client crashes, their
(HTTP server) views of “state” may be
• TCP connection closed inconsistent, must be
reconciled
2: Application Layer 19
HTTP
PC running
Explorer
2: Application Layer 20
HTTP connections
Nonpersistent HTTP Persistent HTTP
• At most one object is sent • Multiple objects can be sent
over a TCP connection. over single TCP connection
• HTTP/1.0 uses between client and server.
nonpersistent HTTP • HTTP/1.1 uses persistent
connections in default
mode
2: Application Layer 21
Nonpersistent HTTP
2: Application Layer 22
Response time modeling
Definition of RRT: time to send
a small packet to travel from
client to server and back.
initiate TCP
Response time: connection
• one RTT to initiate TCP RTT
connection request
file
• one RTT for HTTP request RTT
time to
transmit
and first few bytes of HTTP file
response to return file
received
• file transmission time
total = 2RTT+transmit time time time
2: Application Layer 23
Persistent HTTP
2: Application Layer 24
Persistent HTTP
2: Application Layer 25
HTTP request message
request line
(GET, POST, GET /somedir/[Link] HTTP/1.1
HEAD commands) Host: [Link]
User-agent: Mozilla/4.0
header Connection: close
lines Accept-language:fr
2: Application Layer 27
Uploading form input
Post method:
• Web page often includes
form input URL method:
• Input is uploaded to server • Uses GET method
in entity body • Input is uploaded in URL
field of request line:
[Link]/animalsearch?monkeys&banana
2: Application Layer 28
Method types
HTTP/1.0 HTTP/1.1
• GET • GET, POST, HEAD
• POST • PUT
• HEAD – uploads file in entity body to
path specified in URL field
– asks server to leave
requested object out of • DELETE
response – deletes file specified in the
URL field
2: Application Layer 29
HTTP response message
status line
(protocol
status code HTTP/1.1 200 OK
status phrase) Connection close
Date: Thu, 06 Aug 1998 [Link] GMT
Server: Apache/1.3.0 (Unix)
header
Last-Modified: Mon, 22 Jun 1998 …...
lines
Content-Length: 6821
Content-Type: text/html
2: Application Layer 30
HTTP response status codes
In first line in server->client response message.
A few sample codes:
200 OK
– request succeeded, requested object later in this message
301 Moved Permanently
– requested object moved, new location specified later in this
message (Location:)
400 Bad Request
– request message not understood by server
404 Not Found
– requested document not found on this server
505 HTTP Version Not Supported
2: Application Layer 31
Trying out HTTP (client side) for yourself
2: Application Layer 32
User-server state: cookies
Many major Web sites use Example:
cookies (RFC-2109), cookies – Susan access Internet always from
allows sites to keep track of same PC
user. – She visits a specific e-commerce
site for first time
Major portal like yahoo, e- – When initial HTTP requests arrives
commerce (amazon) and at site, site creates a unique ID and
advertising uses cookies. creates an entry in backend
database for ID
Four components: – Website is able to track susan
1) cookie header line in the HTTP activity at the website. However,
response message website does not bother about
name but which page user id
2) cookie header line in HTTP visited, ordered and time is known.
request message – Ex: e-commerce (amazon) site can
3) cookie file kept on user’s host recommend the products user id
and managed by user’s browser searched on webpage when id
4) back-end database at Web site visited site in the past. (one click
shopping)
2: Application Layer 33
Cookies: keeping “state” (cont.)
client server
Cookie file
usual http request msg
amazon: 1678 cookie: 1678 cookie-
ebay: 8734 specific
usual http response msg action
one week later:
2: Application Layer 34
Cookies (continued)
aside
Cookies and privacy:
What cookies can bring: cookies permit sites to learn a lot
• authorization about you
you may supply name and e-mail
• shopping carts to sites
• recommendations search engines use redirection &
cookies to learn yet more
• user session state (Web e- advertising companies obtain info
mail) across sites
2: Application Layer 35
Web caches (proxy server)
Goal: satisfy client request without involving origin server
client
origin
server
2: Application Layer 36
More about Web caching
• Cache acts as both client and Why Web caching?
server • Reduce response time for client
• Typically cache is installed by ISP request.
(university, company, residential • Reduce traffic on an institution’s
ISP) access link.
• Internet dense with caches
enables “poor” content providers
to effectively deliver content (but
so does P2P file sharing)
2: Application Layer 37
Caching example
origin
Assumptions
servers
• average object size = 100,000 bits
public
• avg. request rate from institution’s Internet
browsers to origin servers = 15/sec
• delay from institutional router to
any origin server and back to
1.5 Mbps
router = 2 sec
access link
Consequences
institutional
• utilization on LAN = 15% network
10 Mbps LAN
• utilization on access link = 100%
• total delay = Internet delay + access
delay + LAN delay
= 2 sec + minutes + milliseconds institutional
cache
2: Application Layer 38
Caching example (cont)
origin
Possible solution
servers
• increase bandwidth of access link
to, say, 10 Mbps public
Internet
Consequences
• utilization on LAN = 15%
• utilization on access link = 15%
10 Mbps
• Total delay = Internet delay + access access link
delay + LAN delay
institutional
= 2 sec + msecs + msecs
network
• often a costly upgrade 10 Mbps LAN
institutional
cache
2: Application Layer 39
Caching example (cont)
origin
Install cache servers
• suppose hit rate is .4 public
Consequence Internet
• 40% requests will be satisfied
almost immediately
• 60% requests satisfied by origin
server 1.5 Mbps
access link
• utilization of access link reduced
to 60%, resulting in negligible institutional
delays (say 10 msec) network
10 Mbps LAN
• total avg delay = Internet delay
+ access delay + LAN delay =
.6*(2.01) secs + milliseconds <
1.4 secs
institutional
cache
2: Application Layer 40
Conditional GET
cache server
• Goal: don’t send object if cache
has up-to-date cached version HTTP request msg
• cache: specify date of cached If-modified-since:
object
<date>
copy in HTTP request not
If-modified-since: modified
<date> HTTP response
HTTP/1.0
• server: response contains no 304 Not Modified
object if cached copy is up-to-
date:
HTTP request msg
HTTP/1.0 304 Not If-modified-since:
Modified <date> object
modified
HTTP response
HTTP/1.0 200 OK
<data>
2: Application Layer 41
FTP, SMTP and DNS
2: Application Layer 42
FTP: separate control, data connections
TCP control connection
• FTP client contacts FTP server at port 21
port 21, specifying TCP as
transport protocol
TCP data connection
• Client obtains authorization over FTP port 20 FTP
control connection client server
• Client browses remote directory
by sending commands over Server opens a second TCP data
control connection. connection to transfer another
file.
• When server receives a command
Control connection (separate):
for a file transfer, the server
“out of band”
opens a TCP data connection to
FTP server maintains “state”:
client
current directory is open through
• After transferring one file, server user session, earlier
closes connection. authentication (control
connection)
2: Application Layer 43
outgoing
Electronic Mail message queue
user mailbox
user
Three major components: agent
• user agents mail
user
• mail servers server
agent
• simple mail transfer protocol: SMTP
SMTP mail
server user
User Agent SMTP agent
• a.k.a. “mail reader”
SMTP
• composing, editing, reading mail user
mail
messages agent
server
• e.g., Eudora, Outlook, elm,
Netscape Messenger user
• outgoing, incoming messages agent
stored on server user
agent
2: Application Layer 44
Electronic Mail: mail servers
user
Mail Servers agent
• mailbox contains incoming mail
user
messages for user server
agent
• message queue of outgoing (to SMTP
be sent) mail messages mail
server user
• SMTP protocol between mail SMTP agent
servers to send email messages
– client: sending mail server SMTP
user
– “server”: receiving mail mail
agent
server server
user
agent
user
agent
2: Application Layer 45
Electronic Mail: SMTP [RFC 2821]
• uses TCP to reliably transfer email message from client to server,
port 25
• direct transfer: sending server to receiving server
• three phases of transfer
– handshaking (greeting)
– transfer of messages
– closure
• command/response interaction
– commands: ASCII text
– response: status code and phrase
• messages must be in 7-bit ASCII
2: Application Layer 46
Scenario: Alice sends message to Bob
1) Alice uses UA to compose 4) SMTP client sends Alice’s
message and “to” message over the TCP
bob@[Link] connection
2) Alice’s UA sends message to her 5) Bob’s mail server places the
mail server; message placed in message in Bob’s mailbox
message queue 6) Bob invokes his user agent to
3) Client side of SMTP opens TCP read message
connection with Bob’s mail
server
1 mail
mail
server user
user server
2 agent
agent 3 6
4 5
2: Application Layer 47
Mail access protocols
SMTP SMTP access
user
user protocol
agent
agent
2: Application Layer 48
DNS: Domain Name System
2: Application Layer 49
DNS and Applications
• Which applications use DNS?
• HTTP
– Browser extracts hostname
– Sends hostname to DNS
– DNS does lookup and returns IP address
– Browser sends HTTP GET to IP address
2: Application Layer 50
DNS
doesn’t scale!
2: Application Layer 51
Distributed, Hierarchical Database
Root DNS Servers
2: Application Layer 52
DNS: Root name servers
• contacted by local name server that can not resolve name
• root name server:
– contacts authoritative name server if name mapping not known
– gets mapping
– returns mapping to local name server
a Verisign, Dulles, VA
c Cogent, Herndon, VA (also Los Angeles)
d U Maryland College Park, MD k RIPE London (also Amsterdam,
g US DoD Vienna, VA
h ARL Aberdeen, MD i Frankfurt)
Autonomica, Stockholm (plus 3
j Verisign, ( 11 locations) other locations)
m WIDE Tokyo
e NASA Mt View, CA
f Internet Software C. Palo Alto,
CA (and 17 other locations)
2: Application Layer 53
TLD and Authoritative Servers
• Top-level domain (TLD) servers: responsible
for com, org, net, edu, etc, and all top-level
country domains uk, fr, ca, jp.
– Network solutions maintains servers for com TLD
– Educause for edu TLD
• Authoritative DNS servers: organization’s DNS
servers, providing authoritative hostname to
IP mappings for organization’s servers (e.g.,
Web and mail).
– Can be maintained by organization or service
provider
2: Application Layer 54
Local Name Server
• Does not strictly belong to hierarchy
• Each ISP (residential ISP, company, university)
has one.
– Also called “default name server”
• When a host makes a DNS query, query is sent
to its local DNS server
– Acts as a proxy, forwards query into hierarchy.
2: Application Layer 55
root DNS server
Example
2
• Host at [Link] 3
TLD DNS server
wants IP address for 4
[Link]
5
[Link]
2: Application Layer 56
Recursive queries
root DNS server
recursive query:
puts burden of name 2 3
resolution on contacted
name server 7 6
heavy load? TLD DNS server
iterated query:
contacted server replies local DNS server
4
with name of server to [Link] 5
contact 1 8
“I don’t know this name,
but ask this server”
authoritative DNS server
[Link]
requesting host
[Link]
[Link]
2: Application Layer 57
DNS: caching and updating records
• once (any) name server learns mapping, it caches
mapping
– cache entries timeout (disappear) after some time
– TLD servers typically cached in local name servers
• Thus root name servers not often visited
• update/notify mechanisms under design by IETF
– RFC 2136
– [Link]
2: Application Layer 58
DNS records
DNS: distributed db storing resource records (RR)
RR format: (name, value, type, ttl)
Type=A Type=CNAME
name is hostname name is alias name for some
value is IP address “cannonical” (the real) name
[Link] is really
• Type=NS [Link]
value is cannonical name
– name is domain (e.g. [Link])
– value is IP address of
authoritative name server for Type=MX
this domain value is name of mailserver
associated with name
2: Application Layer 59
Socket Programming
• Network applications consists of a pair of
programs- a client program and a server
program- residing in two different end
systems.
• When these two programs are executed, a
client and a server process are created, and
these processes communicate with each other
by reading from and writing to sockets.
Socket Programming
2: Application Layer 62
Socket-programming using TCP
Socket: a door between application process and end-end-
transport protocol (UDP or TCP)
TCP service: reliable transfer of bytes from one process to
another
controlled by
controlled by process application
application process
developer
developer socket socket
controlled by TCP with TCP with controlled by
buffers, operating
operating buffers, internet variables system
system variables
host or host or
server server
2: Application Layer 63
Socket programming with TCP
Client must contact server • When contacted by client, server
• server process must first be TCP creates new socket for
running server process to communicate
• server must have created with client
socket (door) that welcomes – allows server to talk with
client’s contact multiple clients
Client contacts server by: – source port numbers used to
distinguish clients (more in
• creating client-local TCP socket
Chap 3)
• specifying IP address, port
number of server process application viewpoint
• When client creates socket:
TCP provides reliable, in-order
client TCP establishes
transfer of bytes (“pipe”)
connection to server TCP
between client and server
2: Application Layer 64
Stream jargon
keyboard monitor
• A stream is a sequence of
characters that flow into or
inFromUser
out of a process. input
stream
inFromServer
outToServer
output input
stream stream
attached to an output
source, eg, monitor or client TCP
clientSocket
socket. socket TCP
socket
2: Application Layer 65
Socket programming with TCP
keyboard monitor
Example client-server app:
1) client reads line from standard
inFromUser
input (inFromUser stream) , input
stream
sends to server via socket Client
(outToServer stream) Process
process
2) server reads line from socket
3) server converts line to uppercase,
sends back to client
inFromServer
outToServer
4) client reads, prints modified line output input
from socket (inFromServer stream stream
stream)
client TCP
clientSocket
socket TCP
socket
2: Application Layer 66
Client/server socket interaction: TCP
Server (running on hostid) Client
create socket,
port=x, for
incoming request:
welcomeSocket =
ServerSocket()
write reply to
connectionSocket read reply from
clientSocket
close
connectionSocket close
clientSocket
2: Application Layer 67
Example: Java client (TCP)
import [Link].*;
import [Link].*;
class TCPClient {
sentence = [Link]();
Send line
to server [Link](sentence + '\n');
[Link]();
}
}
2: Application Layer 69
Example: Java server (TCP)
import [Link].*;
import [Link].*;
class TCPServer {
2: Application Layer 70
Example: Java server (TCP), cont
Create output
stream, attached
DataOutputStream outToClient =
to socket
new DataOutputStream([Link]());
Read in line
from socket clientSentence = [Link]();
2: Application Layer 71
Socket programming with UDP
UDP: no “connection” between client
and server
• no handshaking
• sender explicitly attaches IP application viewpoint
address and port of destination
to each packet UDP provides unreliable transfer
of groups of bytes (“datagrams”)
• server must extract IP address,
between client and server
port of sender from received
packet
UDP: transmitted data may be
received out of order, or lost
2: Application Layer 72
Client/server socket interaction: UDP
Server (running on hostid) Client
create socket,
create socket,
port=x, for clientSocket =
incoming request: DatagramSocket()
serverSocket =
DatagramSocket()
Create, address (hostid, port=x,
send datagram request
using clientSocket
read request from
serverSocket
write reply to
serverSocket
specifying client read reply from
clientSocket
host address,
port number close
clientSocket
2: Application Layer 73
Example: Java client (UDP)
keyboard monitor
inFromUser
input
stream
Client
Process
process Input: receives packet
(TCP received “byte
Output: sends packet stream”)
(TCP sent “byte
receivePacket
sendPacket
stream”) UDP UDP
packet packet
client UDP
clientSocket
socket UDP
socket
2: Application Layer 74
Example: Java client (UDP)
import [Link].*;
import [Link].*;
class UDPClient {
public static void main(String args[]) throws Exception
Create {
input stream
BufferedReader inFromUser =
Create new BufferedReader(new InputStreamReader([Link]));
client socket
DatagramSocket clientSocket = new DatagramSocket();
Translate
hostname to IP InetAddress IPAddress = [Link]("hostname");
address using DNS
byte[] sendData = new byte[1024];
byte[] receiveData = new byte[1024];
String modifiedSentence =
new String([Link]());
2: Application Layer 76
Example: Java server (UDP)
import [Link].*;
import [Link].*;
class UDPServer {
public static void main(String args[]) throws Exception
Create {
datagram socket
at port 9876 DatagramSocket serverSocket = new DatagramSocket(9876);
while(true)
{
Create space for
received datagram DatagramPacket receivePacket =
new DatagramPacket(receiveData, [Link]);
Receive [Link](receivePacket);
datagram
2: Application Layer 77
Example: Java server (UDP), cont
String sentence = new String([Link]());
Get IP addr
port #, of InetAddress IPAddress = [Link]();
sender
int port = [Link]();
sendData = [Link]();
Create datagram
to send to client DatagramPacket sendPacket =
new DatagramPacket(sendData, [Link], IPAddress,
port);
Write out
datagram [Link](sendPacket);
to socket }
}
} End of while loop,
loop back and wait for
another datagram
2: Application Layer 78