Introduction to Probability Concepts
Introduction to Probability Concepts
By
Amritansh
What do I mean?
Your odds of coming late to office is 40 %
Non-deterministic
Deterministic
Deterministic Phenomena
Non-deterministic Phenomena
Non-deterministic
Deterministic
Haphazard
Random
Stochastic phenomena
– Unable to predict the outcomes, but in the long-run, the
outcomes exhibit statistical regularity.
Examples
1. Tossing a fair coin – outcomes S ={Head, Tail}
Unable to predict on each toss whether is Head or Tail.
In the long run can predict that 50% of the time heads will
occur and 50% of the time tails will occur
2. Rolling a die – outcomes
S ={ , , , , , }
Unable to predict outcome but in the long run can one can
determine that each outcome will occur 1/6 of the time.
Use symmetry. Each side is the same. One side should not
occur more frequently than another side in the long run. If the
die is not balanced this may not be true.
Definitions
The sample Space, S
The sample space, S, for a random phenomena is the set of all
possible outcomes.
Examples
1. Tossing a coin – outcomes S ={Head, Tail}
={1, 2, 3, 4, 5, 6}
Event , E
The event, E, is any subset of the sample space, S. i.e. any set of
outcomes (not necessarily all outcomes) of the random phenomena
Venn
S diagram
E
The event, E, is said to have occurred if after the outcome has been
observed the outcome lies in E.
S
E
Examples
1. Rolling a die – outcomes
S ={ , , , , , }
={1, 2, 3, 4, 5, 6}
AB
A B
The event A B occurs if the event A occurs or
the event and B occurs .
AB
A B
Intersection
Let A and B be two events, then the intersection
of A and B is the event (denoted by AB) defined
by:
A B = {e| e belongs to A and e belongs to B}
AB
A B
The event A B occurs if the event A occurs and
the event and B occurs .
AB
A B
Complement
Let A be any event, then the complement of A
(denoted by A ) defined by:
A
A
The event A occurs if the event A does not
occur
A
A
In problems you will recognize that you are working with:
A B
If two events A and B are are mutually
exclusive then:
1. They have no outcomes in common.
They can’t occur at the same time. The outcome
of the random experiment can not belong to both
A and B.
A B
Definition: probability of an Event E.
Suppose that the sample space S = {o1, o2, o3, … oN} has a finite number, N, of
oucomes.
Also each of the outcomes is equally likely (because of symmetry).
Then for any event E
if A B = f
(A and B mutually exclusive)
If two events A and B are are mutually
exclusive then:
1. They have no outcomes in common.
They can’t occur at the same time. The outcome
of the random experiment can not belong to both
A and B.
A B
P[A B] = P[A] + P[B]
i.e.
P[A or B] = P[A] + P[B]
A B
Rule The additive rule
(In general)
or
P[A or B] = P[A] + P[B] – P[A and B]
Logic
A B
A B
A B
Example:
Saskatoon and Moncton are two of the cities competing
for the World university games. (There are also many
others). The organizers are narrowing the competition to
the final 5 cities.
There is a 20% chance that Saskatoon will be amongst
the final 5. There is a 35% chance that Moncton will be
amongst the final 5 and an 8% chance that both
Saskatoon and Moncton will be amongst the final 5.
What is the probability that Saskatoon or Moncton will
be amongst the final 5.
Solution:
Let A = the event that Saskatoon is amongst the final 5.
Let B = the event that Moncton is amongst the final 5.
Given P[A] = 0.20, P[B] = 0.35, and P[A B] = 0.08
What is P[A B]?
Note: “and” ≡ , “or” ≡ .
P A B P A P B P A B
0.20 0.35 0.08 0.47
Rule for complements
2. P A 1 P A
or
P not A 1 P A
Complement
A
A
The event A occurs if the event A does not
occur
A
A
Logic:
A and A are mutually exclusive.
and S A A
A
A
thus 1 P S P A P A
and P A 1 P A
Conditional Probability
Conditional Probability
P A B if P B 0
P A B
P B
Rationale:
If we’re told that event B has occurred then the
sample space is restricted to B.
The probability within B has to be normalized, This
is achieved by dividing by P[B]
The event A can now only occur if the outcome is
in of A ∩ B. Hence the new probability of A is:
A
P A B B
P A B
P B A∩B
An Example
The academy awards is soon to be shown.
For a specific married couple the probability that the husband watches the
show is 80%, the probability that his wife watches the show is 65%, while the
probability that they both watch the show is 60%.
If the husband is watching the show, what is the probability that his wife is
also watching the show
Solution:
The academy awards is soon to be shown.
Let B = the event that the husband watches the show
P[B]= 0.80
Let A = the event that his wife watches the show
P[A]= 0.65 and P[A ∩ B]= 0.60
P A B 0.60
P A B 0.75
P B 0.80
Independence
Definition
Two events A and B are called independent if
P A B P A P B
Note if P B 0 and P A 0 then
P A B P A P B
P A B P A
P B P B
P A B P A P B
and P B A P B
P A P A
Thus in the case of independence the conditional probability of
an event is not affected by the knowledge of the other event
Difference between independence
and mutually exclusive
mutually exclusive
Two mutually exclusive events are independent only in
the special case where
P A 0 and P B 0. (also P A B 0
Mutually exclusive events are
A highly dependent otherwise. A
B
and B cannot occur
simultaneously. If one event
occurs the other event does not
occur.
Independent events
P A B P A P B
P A B P A
or P A
P B P S
S
A B
The ratio of the probability of the
A B set A within B is the same as the
ratio of the probability of the set
A within the entire sample S.
The multiplicative rule of probability
P A P B A if P A 0
P A B
P B P A B if P B 0
and
P A B P A P B
and
P[A B] = P[A] + P[B] if A B = f
The Rule for complements
P E 1 P E
Conditional probability
P A B
P A B
P B
The multiplicative rule of probability
P A P B A if P A 0
P A B
P B P A B if P B 0
and
P A B P A P B
if A and B are independent.
Let A = A1 A2 A3 ….
A3
n3 A4
n4
Rule 2
Suppose we carry out two operations in sequence
Let
n2
n
1
n2
n2
n2
Examples
1. We have a committee of 10 people. We choose from this
committee, a chairman and a vice chairman. How may ways can
this be done?
Solution:
Let n1 = the number of ways the chairman can be
chosen = 10.
Let n2 = the number of ways the vice-chairman
can be chosen once the chair has been
chosen = 9.
Then N = n1n2 = (10)(9) = 90
2. In Black Jack you are dealt 2 cards. What is the probability that
you will be dealt a 21?
Solution:
The number of ways that two cards can be selected from
a deck of 52 is N = (52)(51) = 2652.
A “21” can occur if the first card is an ace and the
second card is a face card or a ten {10, J, Q, K} or the
first card is a face card or a ten and the second card is an
ace.
The number of such hands is (4)(16) +(16)(4) =128
Thus the probability of a “21” = 128/2652 = 32/663
The Multiplicative Rule of Counting
n2
n3
n1
n2
n2
Examples
1. Permutations: How many ways can you
order n objects
Solution:
Ordering n objects is equivalent to performing n operations in
sequence.
1. Choosing the first object in the sequence (n1 = n)
2. Choosing the 2nd object in the sequence (n2 = n -1).
…
k. Choosing the kth object in the sequence (nk = n – k + 1)
…
n. Choosing the nth object in the sequence (nn = 1)
The total number of ways this can be done is:
N = n(n – 1)…(n – k + 1)…(3)(2)(1) = n!
Example How many ways can you order the 4 objects
{A, B, C, D}
Solution:
N = 4! = 4(3)(2)(1) = 24
Here are the orderings.
n!
Pk =n n 1 n k 1
n
n k !
for k = n
n! n!
n Pn n!
0! 1
Example How many permutations of size 3 can be
found in the group of 5 objects {A, B, C, D, E}
5!
Solution: 5 P3 = 5 4 3 60
5 3 !
ABC ABD ABE ACD ACE ADE BCD BCE BDE CDE
ACB ADB AEB ADC AEC AED BDC BEC BED CED
BAC BAD BAE CAD CAE DAE CBD CBE DBE DCE
BCA BDA BEA CDA CEA DEA CDB CEB DEB DEC
CAB DAB EAB DAC EAC EAD DBC EBC EBD ECD
CAB DBA EBA DCA ECA EDA DCB ECB EDB EDC
Example We have a committee of n = 10 people and
we want to choose a chairperson, a vice-
chairperson and a treasurer
10! 10!
10 P3 = 10 9 8 720
10 3! 7!
Example We have a committee of n = 10 people and we want
to choose a chairperson, a vice-chairperson and a treasurer.
Suppose that 6 of the members of the committee are male and 4
of the members are female. What is the probability that the
three executives selected are all male?
Solution: Again we want to select 3 persons from the
committee of 10 in a specific order. (Permutations of size 3
from a group of 10).The total number of ways that this can be
done is:
10! 10!
10 P3 = 10 9 8 720
10 3! 7!
This is the size, N = n(S), of the sample space S. Assume all
outcomes in the sample space are equally likely.
Let E be the event that all three executives are male
6! 6!
n E 6 P3 = 6 5 4 120
6 3! 3!
Hence
nE 120 1
PE
n S 720 6
n 0 1 n 1 2 n2
n
x y C x 0 n
y + C
n 1 x y + C
n 2 x y
+ n Ck x k y n k + + n Cn x n y 0
n 0 n n 1 n 1 n 2 n 2
x y + x y + x y +
0 1 2
n k nk n n 0
+ x y + + x y
k n
Proof: The term xkyn - k will arise when we select x
from k of the factors of (x + y)n and select y from
the remaining n – k factors. The no. of ways that
this can be done is: n
k
n
Hence there will be k terms equal to xkyn = k and
n 0 n n 1 n 1 n 2 n 2
x y
n
x y + x y + x y +
0 1 2
n k nk n n 0
+ x y + + x y
k n
Pascal’s triangle – a procedure for calculating
binomial coefficients
1
1 1
1 2 1
1 3 3 1
1 4 6 4 1
1 5 10 10 5 1
1 6 15 20 15 6 1
1 7 21 35 35 21 7 1
The two edges of Pascal’s triangle contain 1’s
The interior entries are the sum of the two nearest entries in the row
above
The entries in the nth row of Pascals triangle are the values of the
binomial coefficients
2
x y x 2
2 xy y 2
x y x 3x y 3xy y
3 3 2 2 3
4
x y x 4
4 x 3
y 6 x 2 2
y 4 xy 3
y 3
x y
5
x 5 x y 10 x y 10 x y 5 xy y
5 4 3 2 2 3 4 5
x y
6
x 6 6 x5 y 15 x 4 y 2 20 x3 y 3 15 x 2 y 4 6 xy 5 y 6
x y x7 7 x6 y 21x5 y 2 35 x 4 y 3 35 x3 y 4 21x 2 y 5 7 xy 6 y 7
7
Summary of counting rules
Rule 1
n(A1 A2 A3 …. ) = n(A1) + n(A2) + n(A3) + …
if the sets A1, A2, A3, … are pairwise mutually exclusive
(i.e. Ai Aj = f)
Rule 2
N = n1 n2 = the number of ways that two operations can be
performed in sequence if
2. Permutations
n!
n Pk The number of ways that you can
n k ! choose k objects from n in a
specific order
3. Combinations
n n!
n Ck The number of ways that you
k k ! n k ! can choose k objects from n
(order of selection irrelevant)
Applications to some counting problems
The trick is to use the basic counting formulae together with the Rules
We will illustrate this with examples
Counting problems are not easy. The more practice better the
techniques
Application to Lotto 6/49
Here you choose 6 numbers from the integers 1, 2, 3, …, 47, 48, 49.
Six winning numbers are chosen together with a bonus number.
How many choices for the 6 winning numbers
49 49! 49 48 47 46 45 44
49 C6
6 6!43! 6 5 4 3 2 1
13,983,816
You can lose and win in several ways
6 43
6 962,598 = 5,775,588
1 5
3. Two winning numbers – lose
Two numbers are chosen from the six winning numbers and the
remaining four have to be chosen from the losing numbers (bonus
not included)
6 42
15 111,930 = 1,678,950
2 4
Three numbers are chosen from the six winning numbers and the
remaining three have to be chosen from the losing numbers + the
bonus number
6 43
20 12,341 = 246,820
3 3
Five numbers are chosen from the six winning numbers and the
remaining number has to be chosen from the losing numbers
(excluding the bonus number)
6 42
6 42 = 252
5 1
6 1
6 1 = 6
5 1
9. six winning numbers (no bonus) – win approx. $4,000,000.00
6
1
6
Summary
n Prize Prob
0 winning 6,096,454 nil 0.4359649755
1 winning 5,775,588 nil 0.4130194505
2 winning 1,678,950 nil 0.1200637937
2 + bonus 172,200 $ 5.00 0.0123142353
3 winning 246,820 $ 10.00 0.0176504039
4 winning 13,545 $ 80.00 0.0009686197
5 winning 252 $ 2,500.00 0.0000180208
5 + bonus 6 $ 100,000.00 0.0000004291
6 winning 1 $ 4,000,000.00 0.0000000715
Total 13,983,816
Summary of counting rules
Rule 1
n(A1 A2 A3 …. ) = n(A1) + n(A2) + n(A3) + …
if the sets A1, A2, A3, … are pairwise mutually exclusive
(i.e. Ai Aj = f)
Rule 2
N = n1 n2 = the number of ways that two operations can be
performed in sequence if
2. Permutations
n!
n Pk The number of ways that you can
n k ! choose k objects from n in a
specific order
3. Combinations
n n!
n Ck The number of ways that you
k k ! n k ! can choose k objects from n
(order of selection irrelevant)
Applications to some counting
problems
The trick is to use the basic counting formulae together with the Rules
We will illustrate this with examples
Counting problems are not easy. The more practice better the
techniques
Another Example
counting poker hands
A poker hand consists of five cards chosen at random from a deck of 52 cards.
A A
A 6
A A
The total number of poker hands is
A
A
6
52
N 2,598,960
5
Counting poker hands
1. Nothing Hand {x, y, z, u, v}
• Not all in sequence or not all the same suit
2. Pair {x, x, y, z, u}
3. Two pair {x, x, y, y, z}
6. Flush {x, y, z, u, v}
• Not all in sequence but all the same suit
7. Full House {x, x, x, y, y}
8. Four of a kind {x, x, x, x, y}
4
• Select the suits for the for x. 4
3
4
• Select the suits for the for y. 6
2