Appendix: Useful Math Results
1. Summations
(a) Binomial:
n
P
j=0
C
n
j
a
j
b
(nj)
= (a + b)
n
, where C
n
j
=
n!
j!(nj)!
.
(b) Geometric:
i.
P
j=0
r
j
=
1
1r
, |r| < 1.
ii.
P
j=1
r
j
=
r
1r
, |r| < 1.
iii.
n
P
j=0
r
j
=
1r
(n+1)
1r
, −∞ < r < +.
(c) N egative Binomial:
P
j=0
C
j+k
k
π
j
= (1 π)
(k+1)
, 0 < π < 1, k a positive
integer.
(d) Exponential:
P
j=0
x
j
j!
= e
x
, −∞ < x < +.
(e) S ums of Integers:
i.
n
P
i=1
i =
n(n+1)
2
.
ii.
n
P
i=1
i
2
=
n(n+1)(2n+1)
6
.
iii.
n
P
i=1
i
3
=
h
n(n+1)
2
i
2
.
2. Limits
(a) lim
n→∞
1 +
a
n
n
= e
a
, −∞ < a < +.
3. Important Calculus-Based Results
(a) L’Hˆopital’s Rule: For diﬀerentiable functions f(x) and g(x) and an
“extended” rea l number c (i.e., c
1
or c = ±), suppose that
lim
xc
f(x) = lim
xc
g(x) = 0, or that lim
xc
f(x) = lim
xc
g(x) =
±∞. Suppose also that lim
xc
f
(x)/g
(x) e xists [in particular, g
(x) 6= 0
353
354 APPENDIX: USEFUL MATH RESULTS
near c, except possibly at c]. Then,
lim
xc
f(x)
g(x)
= lim
xc
f
(x)
g
(x)
.
L’Hˆopital’s Rule is also valid for one-sided limits.
(b) Integration by Parts: Let u = f(x) and v = g(x), with diﬀerentials du =
f
(x)dx and dv = g
(x)dx. Then,
Z
u dv = uv
Z
v du.
(c) Jacobians for One- and Two-Dimensional Change-of-Variable Transfor-
mations: Let X be a scalar variable with support A
1
. Consider a
one-to-one transformation U = g(X) tha t maps A B
1
. Denote
the inverse of U as X = h(U ). Then, the corresponding o ne -dimensional
Jacobian of the transformation is deﬁned as
J =
d[h(U)]
dU
,
so that
Z
A
f(X)dX =
Z
B
f[h(U)]|J|dU.
Similarly, consider scalar variables X and Y deﬁned on a two-
dimensional set A
2
, and let U = g
1
(X, Y ) and V = g
2
(X, Y ) deﬁne
a one-to- one transformation that maps A in the xy-plane to B
2
in
the uv-plane. Deﬁne X = h
1
(U, V ) and Y = h
2
(U, V ). The n, the Ja-
cobian of the (two-dimensional) transformation is given by the second-
order determinant
J =
h
1
(U,V )
U
h
1
(U,V )
V
h
2
(U,V )
U
h
2
(U,V )
V
,
so that
Z Z
A
f(X, Y )dXdY =
Z Z
B
f[h
1
(U, V ), h
2
(U, V )]|J|dU dV.
4. Special Functions
(a) Gamma Function:
i. For any real number t > 0, the Gamma function is deﬁned as
Γ(t) =
Z
0
y
t1
e
y
dy.
355
ii. For any real number t > 0, Γ(t + 1) = tΓ(t).
iii. For any positive integer n, Γ(n) = (n 1)!
iv. Γ(1/2) =
π; Γ(3/2) =
π/2; Γ(5 /2) = (3
π)/4.
(b) Beta Function:
i. For α > 0 and β > 0, the Beta function is deﬁned as
B(α, β) =
Z
1
0
y
α1
(1 y)
β1
dy.
ii. B(α, β) =
Γ(α)Γ(β)
Γ(α+β)
(c) Convex and Concave Functions: A real-valued function f(·) is sa id to be
convex if, for any two points x and y in its domain and any t [0, 1], we
have
f[tx + (1 t)y] tf(x) + (1 t)f(y).
Likewise, f(·) is said to be concave if
f[tx + (1 t)y] tf(x) + (1 t)f(y).
Also, f(x) is concave on [a, b] if and only if f(x) is convex on [a, b].
5. Approximations
(a) Stirling’s Approximation:
For n a large non-negative integer, n!
2πn
n
e
n
.
(b) Taylor S eries Approximations:
(i) Univariate Taylor Series: If f(x) is a real- valued function of x that
is inﬁnitely diﬀerentiable in a neighbor hood of a real number a, then a
Taylor series expansion of f(x) around a is equal to
f(x) =
X
k=0
f
(k)
(a)
k!
(x a)
k
,
where
f
(k)
(a) =
"
d
k
f(x)
dx
k
#
|x=a
, k = 0, 1, . . . , .
When a = 0, the inﬁnite se ries expa nsion above is called a Maclaurin
series.
As examples, a ﬁ r st-order (or linear) Taylor series approximation to f(x)
around the real number a is equal to
f(x) f(a) +
df(x)
dx
|x=a
(x a),
356 APPENDIX: USEFUL MATH RESULTS
and a second-order Taylor series approximation to f(x) around the real
number a is e qual to
f(x) f(a) +
df(x)
dx
|x=a
(x a) +
1
2!
d
2
f(x)
dx
2
|x=a
(x a)
2
.
(ii) Multivariate Taylor series: For p 2, if f(x
1
, x
2
, . . . , x
p
) is a real-
valued function of x
1
, x
2
, . . . , x
p
that is inﬁnitely diﬀerentiable in a neigh-
borhood of (a
1
, a
2
, . . . , a
p
), where a
i
, i = 1, 2, . . . , p, is a real number,
then a multivariate Taylor series expansion of f(x
1
, x
2
, . . . , x
p
) around
(a
1
, a
2
, . . . , a
p
) is equal to
f(x
1
, x
2
, . . . , x
p
) =
X
k
1
=0
X
k
2
=0
···
X
k
p
=0
f
(k
1
+k
2
+···+k
p
)
(a
1
, a
2
, . . . , a
p
)
k
1
!k
2
! ···k
p
!
×
p
Y
i=1
(x
i
a
i
)
k
i
,
where
f
(k
1
+k
2
+···+k
p
)
(a
1
, a
2
, . . . , a
p
)
=
"
k
1
+k
2
+···+k
p
f(x
1
, x
2
, . . . , x
p
)
x
k
1
1
x
k
2
2
···x
k
p
p
#
|(x
1
,x
2
,...,x
p
)=(a
1
,a
2
,...,a
p
)
.
As examples, when p = 2, a ﬁrst-order (or linear) multivariate Taylor
series approximation to f(x
1
, x
2
) around (a
1
, a
2
) is equal to
f(x
1
, x
2
) f (a
1
, a
2
) +
2
X
i=1
f(x
1
, x
2
)
x
i
|(x
1
,x
2
)=(a
1
,a
2
)
(x
i
a
i
),
and a second-order multivariate Taylor series approximation to f(x
1
, x
2
)
around (a
1
, a
2
) is equal to
f(x
1
, x
2
) f(a
1
, a
2
) +
2
X
i=1
f(x
1
, x
2
)
x
i
|(x
1
,x
2
)=(a
1
,a
2
)
(x
i
a
i
)
+
1
2!
2
X
i=1
2
f(x
1
, x
2
)
x
2
i
|(x
1
,x
2
)=(a
1
,a
2
)
(x
i
a
i
)
2
+
2
f(x
1
, x
2
)
x
1
x
2
|(x
1
,x
2
)=(a
1
,a
2
)
(x
1
a
1
)(x
2
a
2
).
6. Lagrange Multipliers: The method of Lagrange multipliers provides a

Get Exercises and Solutions in Statistical Theory now with O’Reilly online learning.

O’Reilly members experience live online training, plus books, videos, and digital content from 200+ publishers.