Outline Motivating example Bayesian statistics Why and how it works Example (continuation) Summary/conclusions
Bayesian Statistics
A Users Perspective
Ivo M. Foppa
Novel Statistics for Entomologists 12/12/2010
Outline Motivating example Bayesian statistics Why and how it works Example (continuation) Summary/conclusions
Outline
A motivating example
Bayesian statistics
Bayes theorem
Frequentist vs. Bayesian statistics
Appeal of Bayesian statistics
Why and how it works
MCMC
WinBUGS
Example (continuation)
Implementation
Results/comparison
Summary/conclusions
Outline Motivating example Bayesian statistics Why and how it works Example (continuation) Summary/conclusions
Host clustering and percapita mosquito feeding
Release Culex quinquefasciatus (Say); two host locations
(high and low aggregation)
Do percapita rates differ by aggregation?
Outline Motivating example Bayesian statistics Why and how it works Example (continuation) Summary/conclusions
data
Left Right Total
1
(2
) 9 (9) 10 (11)
1 (1) 8 (13) 9 (14)
7 (12) 1 (5) 8 (17)
8 (54) 1 (10) 9 (64)
1 (1) 9 (29) 10 (30)
9 (6) 1 (2) 10 (8)
9 (29) 1 (15) 10 (44)
1 (9) 9 (21) 10 (30)
Number of chickens
Number of blood fed mosquitoes
Outline Motivating example Bayesian statistics Why and how it works Example (continuation) Summary/conclusions
Goal: to estimate the biting rate ratio (single vs. group); null
hypothesis: H
0
: = 1
A reasonable model for the data might be
x
i
Binomial(p
i
, n
i
)
logit(p
i
) =
i
+ s
i
1
+
3
k=1
a
ki
k
+
i
i
= exp
i
+
3
k=1
a
ki
x
i
(N
i
1)
n
i
x
i
,
x
i
, n
i
, N
i
: numbers of blood fed mosquitoes retrieved from the single chicken, the total number retrieved,
and the numbers of chickens used, respectively
i
= logit(
1
m
i
), with m
i
representing the total number of chickens used
s
i
is the side of the bunker with the single chicken
a
ki
for k (1, 2, 3) are dummy variables that are one if m
i
8, 9, 10
1
and
k
are model coefcients
i
N(0, ) is an overdispersion parameter
Outline Motivating example Bayesian statistics Why and how it works Example (continuation) Summary/conclusions
Why not use the conventional approach?
Model is not implemented in SAS, Stata, R etc.
Numbers are small for asymptotic methods
Even if model chosen without random error, standard error
for may not be straightforward
Outline Motivating example Bayesian statistics Why and how it works Example (continuation) Summary/conclusions
Bayes theorem
A and B are two discrete events
Classical expression:
P(A|B) =
P(B|A) P(A)
P(B)
(1)
Event A: Collected insect is of species a
Event B: Collected insect is from location b
P(A|B): Probability that an insect collected in location b is
species a
P(B|A): Probability that an insect of species a has been
collected in location b
Outline Motivating example Bayesian statistics Why and how it works Example (continuation) Summary/conclusions
A more relevant formulation:
Goal: to characterize the parameter (mean, variance,
etc.) given the data x.
General case:
f
(|x) =
L(x|) f
()
L(x|) f
() d
= C L(x|) f
()
L(x|) f
()
f
(|x): Posterior distribution of
f
(): Prior distribution of
L(x|): Likelihood of x given
C: Normalization constant (marginal density of x)
We will come back to this . . .
Outline Motivating example Bayesian statistics Why and how it works Example (continuation) Summary/conclusions
Frequentist vs. Bayesian statistics
Outline Motivating example Bayesian statistics Why and how it works Example (continuation) Summary/conclusions
Frequentist vs. Bayesian statistician
Outline Motivating example Bayesian statistics Why and how it works Example (continuation) Summary/conclusions
Frequentist vs. Bayesian statistics
Why this cultural divide?
Estimation process is tainted by prior knowledge . . .
Why shouldnt we use prior knowledge???
[. . . ] deep philosophical differences between Bayesian and
frequentist inference (Spiegelhalter 2009)
Despite popular belief use of BS must not be a
philosophical choice
Bayesian methods offer valid and highly exible tools for
parameter estimation and inference.
Outline Motivating example Bayesian statistics Why and how it works Example (continuation) Summary/conclusions
The earlier result implies that the distribution of the
parameter(s) of interest, , is proportional to the likelihood
of the data and the prior distribution of the parameter:
f
(|x) L(x|) f
()
Markov chain Monte Carlo (=MCMC) methods allow us to
simulate draws from f
(|x), and thus estimate all desirable
properties of based on the proportional quantity on the
righthand side!
The price: Have to specify prior distribution (but it can be
uninformative)
The benet: Enormous analytic exibility
Outline Motivating example Bayesian statistics Why and how it works Example (continuation) Summary/conclusions
MCMC
Markov chain Monte Carlo methods allow us to sample
from f
(|x) even though we only know L(x|) f
()
A Markov process is a random variable X
t
whose transition
probability to X
t+1
(going from one state to a particular
other) only depends the current state and not on the history
A Markov chain is a sequence of random variables
(X
0
, . . . , X
n
) that was generated by a Markov process
The matrix P that denes the transitions probabilities from j
to k is called the transmission matrix
Under certain conditions, the Markov chain may reach a
stationary distribution,
, such that
P =
Outline Motivating example Bayesian statistics Why and how it works Example (continuation) Summary/conclusions
MCMC
Monte Carlo methods simulate draws from probability
distributions using pseudo random number generators.
MCMC allows us to draw values from f
(|x) even though
only a proportional distribution is known.
All desirable properties of f
(|x) can be determined at
desired accuracy.
Only condition is that model converges.
As we draw directly from f
(|x) results are exact rather
than asymptotic!
Outline Motivating example Bayesian statistics Why and how it works Example (continuation) Summary/conclusions
MCMC and Bayesian statistics
MCMC methods draw samples (based on pseudorandom
numbers) (Monte Carlo part) from a target distribution
(posterior distribution of parameter given data) which is the
stationary distribution of a Markov chain
These methods are required, because the posterior is
typically not fully known (only up to proportionality
constant)
The Bayesian connection: the prior, which is necessary for
the deriving posterior from likelihood
Bayesian statistics therefore somewhat misleading
Outline Motivating example Bayesian statistics Why and how it works Example (continuation) Summary/conclusions
WinBUGS
The platform most commonly used for Bayesian inference
is WinBUGS (Imperial College and Medical Research
Council, UK)
BUGS=Bayesian inference Using Gibbs Sampling
The Gibbs sampler (Geman and Geman 1884) is a
particularly powerful MCMC algorithm
Outline Motivating example Bayesian statistics Why and how it works Example (continuation) Summary/conclusions
Using WinBUGS for analysis
Outline Motivating example Bayesian statistics Why and how it works Example (continuation) Summary/conclusions
Using WinBUGS for analysis
Outline Motivating example Bayesian statistics Why and how it works Example (continuation) Summary/conclusions
Outline Motivating example Bayesian statistics Why and how it works Example (continuation) Summary/conclusions
MLE vs Bayesian
Parameter MLE Bayesian
1
0.557 (0.014,1.101) 0.530 (-0.047,1.060)
3
0.931 (0.182,1.679) 0.947 (0.187,1.706)
1
-0.898 (-1.680,-0.115) -0.922 (-1.721,-0.141)
Point estimate (95% condence interval)
Posterior mean (95% credible interval)
difference in point estimates is due to small numbers!
Overdispersed
Parameter Post. mean (95% CI)
1
0.575 (-0.483,1.684)
3
0.867 (-0.594,2.224)
1
-1.011 (-2.497,0.243)
Outline Motivating example Bayesian statistics Why and how it works Example (continuation) Summary/conclusions
Selected issues
MCMC methods are simulation based and thus
computationally intensive
Estimating a model may take a long time; problematic for
model selection
All inference is based on posterior distribution f
(|x);
instead of condence intervals we have credible
intervals with (supercially) same interpretation.
Validity of inference depends on model convergence (often
visually determined)
Estimates (posterior means etc) are slightly different after
each run
Analysis using WinBUGS interface is painful; but can call
from R or other programs
Need to think about model specically
Outline Motivating example Bayesian statistics Why and how it works Example (continuation) Summary/conclusions
Summary
For an average user, the Bayesian framework via MCMC
methods is attractive because of its analytic exibility
MCMC is Bayesian because prior distributions have to be
specied
Typically, these priors are uninformative
Resulting posterior means/medians are typically virtually
identical from frequentist estimates
Outline Motivating example Bayesian statistics Why and how it works Example (continuation) Summary/conclusions
Thank you!
Resource Page
Bayesian Statistics: A User's Perspective (Symposium Novel Statistics for Entomologists
12/12/2010)
Ivo M. Foppa
The presentation (pdf format) is attached.
A general introduction to Bayesian statistics can be found in Spiegelhalter and Rice
(2009), Scholarpedia, 4(8):5230: [Link]
WinBUGS can be downloaded free of charge at:
o [Link]
o The site contains detailed instructions how to install and use the software.
o The following paper describes the theory behind WinBUGS: Lunn, D. J ., A.
Thomas, N. Best, and D. Spiegelhalter. 2000. WinBUGS-a Bayesian modelling
framework: concepts, structure, and extensibility. Statistics and Computing 10:
325-337.
o The software itself has many worked examples integrated.
I like to run WinBUGS from R. R is a free powerful statistical software that can be
obtained from [Link]
I am happy to help you setting up R (and provide you with R code) so you can run
WinBUGS from R (Warning: I am not an expert!)
