Chapter 6

The Application Layer

Some network apps
 e-mail
 web
 text messaging
 remote login
 P2P file sharing
 multi-user network games
 streaming stored video (YouTube, Netflix)
 voice over IP (e.g., Skype)
 real-time video conferencing

2
Creating a network app application
transport
network
data link
write programs that: physical

 run on (different) end systems

 communicate over network
 e.g., web server software
communicates with browser
software
no need to write software for
network-core devices
 network-core devices do not run
application
transport
user applications network
data link application
 applications on end systems allows physical transport
network
for rapid app development, data link
physical
propagation

3
Application architectures

possible structure of applications:

 client-server
 peer-to-peer (P2P)
 Hybrid combination of client-server and P2P

4
 Client-server architecture
server:
 always-on
 permanent IP address
 data centers for scaling

clients:
 communicate with server
 may be intermittently connected
 may have dynamic IP addresses
client/server  do not communicate directly with
each other

Example: [Link], [Link],

[Link], [Link]….

5
P2P architecture
 no always-on server peer-peer
 arbitrary end systems
directly communicate
 peers request service from
other peers, provide service
in return to other peers
 peers are intermittently
connected and change IP
addresses
 complex management

example:
 file distribution
(BitTorrent)
6
Processes communicating
process: program running within a clients, servers
host
client process: process that initiates
 within same host, two processes
communication
communicate using inter-
process communication (defined server process: process that waits
by OS) to be contacted
 processes in different hosts
communicate by exchanging
messages

 aside: applications with P2P

architectures have client
processes & server processes

7
Sockets
 process sends/receives messages to/from its socket
 socket analogous to door
 sending process shoves message out door
 sending process relies on transport infrastructure on
other side of door to deliver message to socket at
receiving process

application application
socket controlled by
process process app developer

transport transport
network network controlled
link by OS
link Internet
physical physical

8
Addressing processes
 To receive messages, process must have identifier
 host device has unique 32-bit IP address

 Q: does IP address of host on which process runs suffice for

identifying the process?
 A: no, many processes can be running on same host

 identifier includes both IP address and port numbers associated with process
on host.
 example port numbers:
 HTTP server: 80
 mail server: 25
 to send HTTP message to [Link] web server:
 IP address: [Link]
 port number: 80
9
App-layer protocol defines
 types of messages exchanged,
 e.g., request, response
 message syntax:
 what fields in messages & how fields are delineated
 message semantics
 meaning of information in fields
 rules for when and how processes send & respond to
messages

10
How the application messages are transported
Using TCP and UDP protocols over the transport layer.
data integrity ( vs. data loss)
 some apps (e.g., file transfer, web transactions) require 100% reliable data
transfer
 other apps (e.g., audio) can tolerate some loss

Timing (delay)
 some apps (e.g., Internet telephony, interactive games) require low delay to
be “effective”

Throughput (bandwidth)
 some apps (e.g., multimedia) require minimum amount of throughput to be
“effective”
 other apps (“elastic apps”) make use of whatever throughput they get

11
Internet transport protocols services

TCP service: UDP service:

 reliable transport between sending  unreliable data transfer
and receiving process between sending and
receiving process
 flow control: sender won’t
overwhelm receiver
 does not provide: reliability,
flow control, congestion
 congestion control: throttle sender control, timing, throughput
when network overloaded
guarantee, security, or
connection setup,
 does not provide: timing, minimum
throughput guarantee, security
Q: why bother? Why is
 connection-oriented: setup required there a UDP?
between client and server
processes

12
Web and HTTP
 web page consists of objects : object can be HTML file,
JPEG image, Java applet, audio file,…

 web page consists of base HTML-file which includes several

referenced objects

 each object is addressable by a URL, e.g.,

[Link]/someDept/[Link]

host name path name

13
HTTP overview
HTTP: hypertext transfer protocol
 Web’s application layer protocol

 client/server model PC running

 client: browser that requests, Firefox browser
receives, (using HTTP
protocol) and “displays” Web
objects

 server: Web server sends (using server

HTTP protocol) objects in running
response to requests Apache Web
server

iphone running
Safari browser

14
HTTP overview
uses TCP: HTTP is “stateless”
 client initiates TCP connection  server maintains no
(creates socket) to server, port information about past
80 client requests

 server accepts TCP connection

from client aside
protocols that maintain “state”
are complex!
 HTTP messages (application-
layer protocol messages)  past history (state) must be
exchanged between browser maintained
(HTTP client) and Web server  if server/client crashes, their
(HTTP server) views of “state” may be
inconsistent, must be
 TCP connection closed reconciled

15
HTTP connections
non-persistent HTTP ( http/1.0)

 at most one object sent over TCP connection

 connection then closed

 downloading multiple objects required multiple connections

persistent HTTP (http/1.1 default)

 multiple objects can be sent over single TCP connection

between client, server

16
 Non-persistent HTTP
(contains,
suppose user enters URL: references to 10
[Link]/someDepartment/[Link] jpeg images)

1a. HTTP client initiates TCP connection to

HTTP server (process) at 1b. HTTP server at host
[Link] on port 80 [Link] waiting for TCP
connection at port 80. “accepts”
2. HTTP client sends HTTP request message connection, notifying client
(containing URL) into TCP connection
socket. Message indicates that client 3. HTTP server receives request message,
wants object forms response message containing
someDepartment/[Link] requested object, and sends message
into its socket
4. HTTP server closes TCP connection.
5. HTTP client receives response message
containing html file, displays html.
Parsing html file, finds 10 referenced
jpeg objects

6. Steps 1-5 repeated for each of 10 jpeg

objects
time
17
Non-persistent HTTP: response time
RTT (definition): time for a small
packet to travel from client to
server and back

HTTP response time: initiate TCP

connection
 one RTT to initiate TCP
connection RTT
 one RTT for HTTP request and request
file
first few bytes of HTTP response
time to
to return RTT transmit
 file transmission time file
 non-persistent HTTP response file
received
time =
2RTT+ file transmission time
time time

18
Persistent HTTP: response time
 server leaves connection open after sending response
 subsequent HTTP messages between same client/server sent over
open connection

19
 HTTP request message
 two types of HTTP messages: request, response
 HTTP request message:
 ASCII (human-readable format)
carriage return character
line-feed character
request line
(GET, POST, GET /[Link] HTTP/1.1\r\n
HEAD commands) Host: [Link]\r\n
User-Agent: Firefox/3.6.10\r\n
Accept: text/html,application/xhtml+xml\r\n
header Accept-Language: en-us,en;q=0.5\r\n
lines Accept-Encoding: gzip,deflate\r\n
Accept-Charset: ISO-8859-1,utf-8;q=0.7\r\n
carriage return, Keep-Alive: 115\r\n
line feed at start Connection: keep-alive\r\n
of line indicates \r\n
end of header lines HTTP keep-alive, a.k.a., HTTP persistent connection, is an instruction
that allows a single TCP connection to remain open for multiple HTTP
requests/responses. By default, HTTP connections close after each
20
request.
HTTP request message: general format

method sp URL sp version cr lf request

line
header field name value cr lf
header
~
~ ~
~ lines

header field name value cr lf

cr lf

~
~ entity body ~
~ body

21
Method types
HTTP/1.0: HTTP/1.1:
 GET  GET, POST, HEAD
 POST  PUT
 HEAD  uploads file in entity
 asks server to leave body to path specified
requested object out of in URL field
response and give meta
information in headers  DELETE
about it  deletes file specified in
the URL field

22
 HTTP response message
status line
(protocol
status code HTTP/1.1 200 OK\r\n
status phrase) Date: Sun, 26 Sep 2010 [Link] GMT\r\n
Server: Apache/2.0.52 (CentOS)\r\n
Last-Modified: Tue, 30 Oct 2007 [Link]
GMT\r\n
header ETag: "17dc6-a5c-bf716880"\r\n
Accept-Ranges: bytes\r\n
lines Content-Length: 2652\r\n
Keep-Alive: timeout=10, max=100\r\n
Connection: Keep-Alive\r\n
Content-Type: text/html; charset=ISO-8859-
1\r\n
\r\n
data, e.g., data data data data data ...
requested
HTML file
23
HTTP response status codes
 status code appears in 1st line in server-to-
client response message.
 some sample codes:
200 OK
 request succeeded, requested object later in this msg
301 Moved Permanently
 requested object moved, new location specified later in this msg
(Location:)
400 Bad Request
 request msg not understood by server
404 Not Found
 requested document not found on this server
505 HTTP Version Not Supported

24
Trying out HTTP (client side) for yourself
1. Telnet to your favorite Web server:

telnet [Link] 80 opens TCP connection to port 80

(default HTTP server port) at [Link].
anything typed in sent
to port 80 at [Link]

2. type in a GET HTTP request:

GET /~ross/ HTTP/1.1 by typing this in (hit carriage
Host: [Link] return twice), you send
this minimal (but complete)
GET request to HTTP server

3. look at response message sent by HTTP server!

25
User-server state: cookies
many Web sites use cookies to keep track of users (stateless http)
four components:
1) cookie header line of HTTP response message
2) cookie header line in HTTP request message
3) cookie file kept on user’s host, managed by user’s browser
4) back-end database at Web site

example:
 Susan always access Internet from PC
 visits specific e-commerce site for first time
 when initial HTTP requests arrives at site, site creates:
 unique ID
 entry in backend database for ID

26
 Cookies: keeping “state”
client server

ebay 8734
usual http request msg Amazon server
cookie file creates ID
usual http response
1678 for user create backend
ebay 8734
set-cookie: 1678 entry database
amazon 1678
usual http request msg
cookie: 1678 cookie- access
specific
usual http response msg action

one week later:

access
ebay 8734 usual http request msg
amazon 1678 cookie: 1678 cookie-
specific
usual http response msg action
27
Cookies
what cookies can be used for: aside
 authorization cookies and privacy:
 shopping carts  cookies permit sites to
 recommendations learn a lot about you
 you may supply name
 user session state (Web e-mail) and e-mail to sites

how to keep “state”:

 cookies: http messages carry state

28
 Web caches (proxy server)
goal: satisfy client request without involving origin server
 user sets browser: Web
accesses via cache

 browser sends all HTTP proxy

requests to cache server
 object in cache: cache client
returns object origin
server

 else cache requests object

from origin server, then
returns object to client

client origin
server

29
More about Web caching
 cache acts as both client and server
 server for original requesting client
 client to origin server

typically cache is installed by ISP (university, company,

residential ISP)

why Web caching?

 reduce response time for client request
 reduce traffic on an institution’s access link

Up-to-date objects?

30
Conditional GET
client server

 Goal: don’t send object if cache has

up-to-date cached version HTTP request msg
 no object transmission delay If-modified-since: <date> object
 lower link utilization not
modified
HTTP response
before
 cache: specify date of cached copy in HTTP/1.1
<date>
HTTP request 304 Not Modified
If-modified-since: <date>

 server: response contains no object if HTTP request msg

cached copy is up-to-date:
If-modified-since: <date> object
HTTP/1.1 304 Not Modified
modified
HTTP response after
HTTP/1.1 200 OK <date>
<data>
31
FTP: the file transfer protocol
file transfer
FTP FTP FTP
user client server
interface
user
at host remote file
local file system
system

 transfer file to/from remote host

 client/server model
 client: side that initiates transfer (either to/from remote)
 server: remote host

 ftp: RFC 959

 ftp server: port 21

32
FTP: separate control, data connections

 FTP client contacts FTP server at TCP control connection,

server port 21
port 21, using TCP

 client authorized over control FTP

TCP data connection,
server port 20 FTP
connection client server

 client browses remote directory,  server opens another TCP data

sends commands over control connection to transfer another
connection file

 when server receives file transfer  control connection: “out of band”

command, server opens 2nd TCP data
connection (for file) to client
 FTP server maintains “state”:
current directory, earlier
 after transferring one file, server authentication
closes data connection

33
FTP commands, responses
sample commands: sample return codes
 sent as ASCII text over control  status code and phrase (as in
channel HTTP)
 331 Username OK,
 USER username password required
 PASS password
 125 data connection
 LIST return list of file in already open; transfer
starting
current directory

 RETR filename retrieves  425 Can’t open data

connection
(gets) file

 STOR filename stores  452 Error writing file

(puts) file onto remote host

34
 Electronic mail outgoing
message queue
Three major components: user mailbox
 user agents
user
agent
 mail servers
 simple mail transfer protocol: SMTP
mail user
server agent
User Agent
 “mail reader”: composing, editing, reading mail SMTP mail user
messages (e.g., outlook, iPhone mail client…)
server agent
 outgoing, incoming messages stored on server
SMTP
mail servers:
 mailbox contains incoming messages for user
SMTP user
agent
mail
 message queue of outgoing (to be sent) mail
server
messages user
agent
 SMTP protocol between mail servers to send
email messages user
agent
 client: sending mail server
 “server”: receiving mail server
35
Electronic Mail: SMTP
 uses TCP port 25 to reliably transfer email message from client to
server

 direct transfer: sending server to receiving server

 three phases of transfer : handshaking (greeting), transfer of

messages, closure

 command/response interaction (like HTTP, FTP)

 commands: ASCII text
 response: status code and phrase

36
Scenario: Alice sends message to Bob
1) Alice uses UA to compose message “to” bob@[Link]

2) Alice’s UA sends message to her mail server; message placed in message queue

3) client side of SMTP opens TCP connection with Bob’s mail server

4) SMTP client sends Alice’s message over the TCP connection

5) Bob’s mail server places the message in Bob’s mailbox

6) Bob invokes his user agent to read message

1 user mail user

mail agent
agent server server
2 3 6
4
5
Alice’s mail server Bob’s mail server
37
 Try SMTP interaction for yourself:
 telnet servername 25
 see 220 reply from server
 enter HELO, MAIL FROM, RCPT TO, DATA, QUIT commands

above lets you send email without using email client (reader)

Sample SMTP interaction

S: 220 [Link]
C: HELO [Link]
S: 250 Hello [Link], pleased to meet
you
C: MAIL FROM: <alice@[Link]>
S: 250 alice@[Link]... Sender ok
C: RCPT TO: <bob@[Link]>
S: 250 bob@[Link] ... Recipient ok
C: DATA
S: 354 Enter mail, end with "." on a line
by itself
C: Do you like ketchup?
C: How about pickles?
C: .
S: 250 Message accepted for delivery
C: QUIT
S: 221 [Link] closing connection

38
 Mail message format
SMTP: protocol for exchanging
email msgs
header
2-blank
RFC 822: standard for text message line
format:
 1-header lines, e.g.,
 To:
 From: body
 Subject:

 3-Body: the “message”

 ASCII characters only

39
Mail access protocols
user
mail access user
SMTP SMTP protocol
agent agent
(e.g., POP,
IMAP)

sender’s mail receiver’s mail

server server

 SMTP: delivery/storage to receiver’s server

 mail access protocol: retrieval from server
 POP: Post Office Protocol [RFC 1939]: authorization, download (TCP port
110)

 IMAP: Internet Mail Access Protocol [RFC 1730]: more features, including
manipulation of stored msgs on server (TCP port 143)

 HTTP: gmail, Hotmail, Yahoo! Mail, etc.

40
POP3 (more) and IMAP
more about POP3 IMAP
 previous example uses  keeps all messages in one
POP3 “download and place: at server
delete” mode  allows user to organize
 Bob cannot re-read e- messages in folders
mail if he changes  keeps user state across
client sessions:
 POP3 “download-and-  names of folders and
keep”: copies of messages mappings between
on different clients message IDs and folder
 POP3 is stateless across name
sessions

41
DNS: domain name system
people: many identifiers:
 SSN, name, passport #

Internet hosts, routers:

Domain Name System:
 IP address (32 bit) - used  distributed database implemented
for addressing datagrams in hierarchy of many name servers
 “name”, e.g.,
[Link] - used by  application-layer protocol (UDP
humans port 53): hosts, name servers
communicate to resolve names
Q: how to map between IP (address/name translation)
address and name, and
vice versa ?

42
DNS: services, structure
DNS services why not centralize
 hostname to IP address translation DNS?
 single point of failure
 host aliasing  traffic volume
 canonical, alias names  distant centralized database
 maintenance
 mail server aliasing

 load distribution A: doesn’t scale!

 replicated Web servers: many IP
addresses correspond to one
name

43
DNS: a distributed, hierarchical database
Root DNS Servers

… …

com DNS servers org DNS servers edu DNS servers

[Link] [Link] [Link]

[Link] [Link]
DNS servers DNS serversDNS servers
DNS servers DNS servers

client wants IP for [Link]; 1st approx:

 client queries root server to find com DNS server
 client queries .com DNS server to get [Link] DNS server
 client queries [Link] DNS server to get IP address for
[Link]

44
 DNS: a distributed, hierarchical database
root name servers
 contacted by local name server that can not resolve name
 root name server:
 contacts authoritative name server if name mapping not known
 gets mapping
 returns mapping to local name server
c. Cogent, Herndon, VA (5 other sites)
d. U Maryland College Park, MD k. RIPE London (17 other sites)
h. ARL Aberdeen, MD
j. Verisign, Dulles VA (69 other sites ) i. Netnod, Stockholm (37 other sites)

e. NASA Mt View, CA m. WIDE Tokyo

f. Internet Software C. (5 other sites)
Palo Alto, CA (and 48 other
sites)

a. Verisign, Los Angeles CA 13 root name

(5 other sites)
b. USC-ISI Marina del Rey, CA
“servers”
l. ICANN Los Angeles, CA worldwide
(41 other sites)
g. US DoD Columbus,
OH (5 other sites)

45
DNS: a distributed, hierarchical database
top-level domain (TLD) servers:
 responsible for com, org, net, edu, aero, jobs, museums, and all top-
level country domains, e.g.: uk, fr, ca, jp

authoritative DNS servers:

 organization DNS server(s): providing authoritative hostname to IP
mappings for organization hosts
 can be maintained by organization or service provider

Local DNS name server:

 does not strictly belong to hierarchy: maintained by the organization

 when host makes DNS query, query is sent to its local DNS server
 has local cache of recent name-to-address translation pairs
 acts as proxy, forwards query into hierarchy

46
 DNS name root DNS server
resolution example
2
 host at [Link] 3
TLD DNS server
wants IP address for 4
[Link]
5

local DNS server

iterated query: [Link]
 contacted server replies 7 6
1 8
with name of server to
contact
authoritative DNS server
[Link]
requesting host
 “I don’t know this name, [Link]
but ask this server”
[Link]

47
DNS name root DNS server
resolution example
2 3
recursive query: 7
6
 puts burden of name TLD DNS
server
resolution on
contacted name local DNS server
server [Link] 5 4

 heavy load at upper 1 8

levels of hierarchy?
authoritative DNS server
[Link]
requesting host
[Link]

[Link]

48
DNS: caching, updating records
 once (any) name server learns mapping, it caches
mapping

 cache entries timeout (disappear) after some time (TTL)

 TLD servers typically cached in local name servers

• thus root name servers not often visited

49
DNS records
DNS: distributed db storing resource records (RR)
RR format: (name, value, type, ttl)

type=A type=CNAME
 name is hostname  name is alias name for some
 value is IP address “canonical” (the real) name
type=NS  [Link] is really
 name is domain (e.g., [Link]
[Link])  value is canonical name
 value is hostname of
authoritative name type=MX
server for this domain  value is name of mailserver
associated with name

50
DNS protocol, messages
 query and reply messages, both with same message
format 2 bytes 2 bytes

msg header identification flags

 identification: 16 bit # for # questions # answer RRs

query, reply to query uses
# authority RRs # additional RRs
same #
 flags: questions (variable # of questions)
 query or reply
 recursion desired answers (variable # of RRs)
 recursion available
 reply is authoritative authority (variable # of RRs)

additional info (variable # of RRs)

51
DNS protocol, messages

2 bytes 2 bytes

identification flags

# questions # answer RRs

# authority RRs # additional RRs

name, type fields

questions (variable # of questions)
for a query
RRs in response answers (variable # of RRs)
to query
records for
authority (variable # of RRs)
authoritative servers
additional “helpful” additional info (variable # of RRs)
info that may be used
52
Inserting records into DNS
 example: new startup “Network Utopia”
 register name [Link] at DNS registrar
(e.g., Network Solutions)
 provide names, IP addresses of authoritative name server

 registrar inserts two RRs into .com TLD server:

([Link], [Link], NS)
([Link], [Link], A)
 create authoritative server type A record for
[Link]; type NS record for
[Link]

53