Mathematical Statistics for Applied Econometrics

8

Interval Estimation

CONTENTS

8.1 Confidence Intervals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183

8.2 Bayesian Estimation .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192

8.3 Bayesian Confidence Intervals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195

8.4 Chapter Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195

8.5 Review Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196

8.6 Numerical Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196

As we discussed when we talked about continuous distribution functions, the

probability of a specific number under a continuous distribution is zero. Thus,

if we conceptualize any estimator, either a nonparametric estimate of the

mean or a parametric estimate of a function, the probability that the true

value is equal to the estimated value is obviously zero. Thus, we usually talk

about estimated values in terms of confidence intervals. As in the case when

we discussed the probability of a continuous variable, we define some range of

outcomes. However, this time we usually work the other way around, defining a

certain confidence level and then stating the values that contain this confidence

interval.

8.1 Confidence Intervals

Amemiya [1, p. 160] notes a difference between confidence and probability.

Most troubling is our classic definition of probability as “a probabilistic state-

ment involving parameters.” This is troublesome due to our inability, without

some additional Bayesian structure, to state anything concrete about proba-

bilities.

Example 8.1. Let X

i

be distributed as a Bernoulli distribution, i =

1, 2, ···N. Then

T =

¯

X

A

∼ N



θ,

θ (1 − θ)

N



. (8.1)

183

184 Mathematical Statistics for Applied Econometrics

TABLE 8.1

Confidence Levels

k γ/2 γ

1.0000 0.1587 0.3173

1.5000 0.0668 0.1336

1.6449 0.0500 0.1000

1.7500 0.0401 0.0801

1.9600 0.0250 0.0500

2.0000 0.0228 0.0455

2.3263 0.0100 0.0200

Breaking this down a little more – we will construct the estimate of the

Bernoulli parameter as

T =

¯

X =

1

N

N

X

i=1

X

i

(8.2)

where T =

ˆ

θ. If the X

i

are independent, then

V (T ) =

1

N

V (X

i

) =

1

N

θ (1 − θ) . (8.3)

Therefore, we can construct a random variable Z that is the difference between

the true value of the parameter θ and the value of the observed estimate.

Z =

T −θ

r

θ (1 − θ)

N

A

∼ N (0, 1) . (8.4)

Why? By the Central Limit Theory. Given this distribution, we can ask ques-

tions about the probability. Specifically, we know that if Z is distributed

N (0, 1), then we can define

γ

k

= P (|Z| < k) . (8.5)

Essentially, we can either choose a k based on a target probability or we can

define a probability based on our choice of k. Using the normal probability, the

one tailed probabilities for the normal distribution are presented in Table 8.1.

Taking a fairly standard example, suppose that I want to choose a k such

that γ/2 = 0.025, or that we want to determine the values of k such that

the probability is 0.05 that the true value of γ will lie outside the range. The

value of k for this choice is 1.96. This example is comparable to the standard

introductory example of a 0.95 confidence level.

The values of γ

k

can be derived from the standard normal table as

P









|T −θ|

r

θ (1 − θ)

n

< k









= γ

k

. (8.6)