book

Matrix Differential Calculus with Applications in Statistics and Econometrics, 3rd Edition

Name: Matrix Differential Calculus with Applications in Statistics and Econometrics, 3rd Edition
ISBN: 9781119541202

by Jan R. Magnus, Heinz Neudecker

March 2019

Intermediate to advanced

504 pages

11h 3m

English

Wiley

Read now

Unlock full access

Cover
Preface
Part One: Matrices
Chapter 1: Basic properties of vectors and matrices
1 INTRODUCTION2 SETS3 MATRICES: ADDITION AND MULTIPLICATION4 THE TRANSPOSE OF A MATRIX5 SQUARE MATRICES6 LINEAR FORMS AND QUADRATIC FORMS7 THE RANK OF A MATRIX8 THE INVERSE9 THE DETERMINANT10 THE TRACE11 PARTITIONED MATRICES12 COMPLEX MATRICES13 EIGENVALUES AND EIGENVECTORS14 SCHUR'S DECOMPOSITION THEOREM15 THE JORDAN DECOMPOSITION16 THE SINGULAR‐VALUE DECOMPOSITION17 FURTHER RESULTS CONCERNING EIGENVALUES18 POSITIVE (SEMI)DEFINITE MATRICES19 THREE FURTHER RESULTS FOR POSITIVE DEFINITE MATRICES20 A USEFUL RESULT21 SYMMETRIC MATRIX FUNCTIONS
Chapter 2: Kronecker products, vec operator, and Moore‐Penrose inverse
1 INTRODUCTION 2 THE KRONECKER PRODUCT 3 EIGENVALUES OF A KRONECKER PRODUCT4 THE VEC OPERATOR6 EXISTENCE AND UNIQUENESS OF THE MP INVERSE7 SOME PROPERTIES OF THE MP INVERSE8 FURTHER PROPERTIES9 THE SOLUTION OF LINEAR EQUATION SYSTEMSBIBLIOGRAPHICAL NOTES
Chapter 3: Miscellaneous matrix results
1 INTRODUCTION2 THE ADJOINT MATRIX3 PROOF OF THEOREM 3.14 BORDERED DETERMINANTS5 THE MATRIX EQUATION AX = 06 THE HADAMARD PRODUCT7 THE COMMUTATION MATRIX Kmn 8 THE DUPLICATION MATRIX Dn 9 RELATIONSHIP BETWEEN Dn +1 AND Dn , I10 RELATIONSHIP BETWEEN Dn +1 AND Dn , II11 CONDITIONS FOR A QUADRATIC FORM TO BE POSITIVE (NEGATIVE) SUBJECT TO LINEAR CONSTRAINTS12 NECESSARY AND SUFFICIENT CONDITIONS FOR r(A : B) = r(A) + r(B)13 THE BORDERED GRAMIAN MATRIX14 THE EQUATIONS X 1 A + X 2 B′ = G 1, X 1 B = G 2 BIBLIOGRAPHICAL NOTES
Part Two: Differentials: the theory
Chapter 4: Mathematical preliminaries
1 INTRODUCTION2 INTERIOR POINTS AND ACCUMULATION POINTS3 OPEN AND CLOSED SETS4 THE BOLZANO‐WEIERSTRASS THEOREM5 FUNCTIONS6 THE LIMIT OF A FUNCTION7 CONTINUOUS FUNCTIONS AND COMPACTNESS8 CONVEX SETS9 CONVEX AND CONCAVE FUNCTIONS
Chapter 5: Differentials and differentiability
1 INTRODUCTION2 CONTINUITY3 DIFFERENTIABILITY AND LINEAR APPROXIMATION4 THE DIFFERENTIAL OF A VECTOR FUNCTION5 UNIQUENESS OF THE DIFFERENTIAL6 CONTINUITY OF DIFFERENTIABLE FUNCTIONS7 PARTIAL DERIVATIVES8 THE FIRST IDENTIFICATION THEOREM9 EXISTENCE OF THE DIFFERENTIAL, I10 EXISTENCE OF THE DIFFERENTIAL, II11 CONTINUOUS DIFFERENTIABILITY12 THE CHAIN RULE13 CAUCHY INVARIANCE14 THE MEAN‐VALUE THEOREM FOR REAL‐VALUED FUNCTIONS15 DIFFERENTIABLE MATRIX FUNCTIONS16 SOME REMARKS ON NOTATION17 COMPLEX DIFFERENTIATIONBIBLIOGRAPHICAL NOTES
Chapter 6: The second differential
1 INTRODUCTION2 SECOND‐ORDER PARTIAL DERIVATIVES3 THE HESSIAN MATRIX4 TWICE DIFFERENTIABILITY AND SECOND‐ORDER APPROXIMATION, I 5 DEFINITION OF TWICE DIFFERENTIABILITY6 THE SECOND DIFFERENTIAL7 SYMMETRY OF THE HESSIAN MATRIX8 THE SECOND IDENTIFICATION THEOREM9 TWICE DIFFERENTIABILITY AND SECOND‐ORDER APPROXIMATION, II10 CHAIN RULE FOR HESSIAN MATRICES11 THE ANALOG FOR SECOND DIFFERENTIALS12 TAYLOR'S THEOREM FOR REAL‐VALUED FUNCTIONS13 HIGHER‐ORDER DIFFERENTIALS14 REAL ANALYTIC FUNCTIONS15 TWICE DIFFERENTIABLE MATRIX FUNCTIONSBIBLIOGRAPHICAL NOTES

Chapter 7: Static optimization
1 INTRODUCTION2 UNCONSTRAINED OPTIMIZATION3 THE EXISTENCE OF ABSOLUTE EXTREMA4 NECESSARY CONDITIONS FOR A LOCAL MINIMUM5 SUFFICIENT CONDITIONS FOR A LOCAL MINIMUM: FIRST‐DERIVATIVE TEST6 SUFFICIENT CONDITIONS FOR A LOCAL MINIMUM: SECOND‐DERIVATIVE TEST7 CHARACTERIZATION OF DIFFERENTIABLE CONVEX FUNCTIONS8 CHARACTERIZATION OF TWICE DIFFERENTIABLE CONVEX FUNCTIONS9 SUFFICIENT CONDITIONS FOR AN ABSOLUTE MINIMUM10 MONOTONIC TRANSFORMATIONS11 OPTIMIZATION SUBJECT TO CONSTRAINTS12 NECESSARY CONDITIONS FOR A LOCAL MINIMUM UNDER CONSTRAINTS13 SUFFICIENT CONDITIONS FOR A LOCAL MINIMUM UNDER CONSTRAINTS14 SUFFICIENT CONDITIONS FOR AN ABSOLUTE MINIMUM UNDER CONSTRAINTS15 A NOTE ON CONSTRAINTS IN MATRIX FORM16 ECONOMIC INTERPRETATION OF LAGRANGE MULTIPLIERSAPPENDIX: THE IMPLICIT FUNCTION THEOREM
Part Three: Differentials: the practice
Chapter 8: Some important differentials
1 INTRODUCTION2 FUNDAMENTAL RULES OF DIFFERENTIAL CALCULUS3 THE DIFFERENTIAL OF A DETERMINANT4 THE DIFFERENTIAL OF AN INVERSE5 DIFFERENTIAL OF THE MOORE‐PENROSE INVERSE6 THE DIFFERENTIAL OF THE ADJOINT MATRIX7 ON DIFFERENTIATING EIGENVALUES AND EIGENVECTORS8 THE CONTINUITY OF EIGENPROJECTIONS9 THE DIFFERENTIAL OF EIGENVALUES AND EIGENVECTORS: SYMMETRIC CASE10 TWO ALTERNATIVE EXPRESSIONS FOR dλ 11 SECOND DIFFERENTIAL OF THE EIGENVALUE FUNCTION
Chapter 9: First‐order differentials and Jacobian matrices
1 INTRODUCTION2 CLASSIFICATION3 DERISATIVES4 DERIVATIVES5 IDENTIFICATION OF JACOBIAN MATRICES6 THE FIRST IDENTIFICATION TABLE7 PARTITIONING OF THE DERIVATIVE8 SCALAR FUNCTIONS OF A SCALAR9 SCALAR FUNCTIONS OF A VECTOR10 SCALAR FUNCTIONS OF A MATRIX, I: TRACE11 SCALAR FUNCTIONS OF A MATRIX, II: DETERMINANT12 SCALAR FUNCTIONS OF A MATRIX, III: EIGENVALUE13 TWO EXAMPLES OF VECTOR FUNCTIONS14 MATRIX FUNCTIONS15 KRONECKER PRODUCTS16 SOME OTHER PROBLEMS17 JACOBIANS OF TRANSFORMATIONS
Chapter 10: Second‐order differentials and Hessian matrices
1 INTRODUCTION2 THE SECOND IDENTIFICATION TABLE3 LINEAR AND QUADRATIC FORMS4 A USEFUL THEOREM5 THE DETERMINANT FUNCTION6 THE EIGENVALUE FUNCTION7 OTHER EXAMPLES8 COMPOSITE FUNCTIONS9 THE EIGENVECTOR FUNCTION10 HESSIAN OF MATRIX FUNCTIONS, I11 HESSIAN OF MATRIX FUNCTIONS, II
Part Four: Inequalities
Chapter 11: Inequalities
1 INTRODUCTION2 THE CAUCHY‐SCHWARZ INEQUALITY3 MATRIX ANALOGS OF THE CAUCHY‐SCHWARZ INEQUALITY4 THE THEOREM OF THE ARITHMETIC AND GEOMETRIC MEANS5 THE RAYLEIGH QUOTIENT6 CONCAVITY OF λ 1 AND CONVEXITY OF λ n 7 VARIATIONAL DESCRIPTION OF EIGENVALUES8 FISCHER'S MIN‐MAX THEOREM9 MONOTONICITY OF THE EIGENVALUES10 THE POINCARÉ SEPARATION THEOREM11 TWO COROLLARIES OF POINCARÉ'S THEOREM12 FURTHER CONSEQUENCES OF THE POINCARÉ THEOREM13 MULTIPLICATIVE VERSION14 THE MAXIMUM OF A BILINEAR FORM15 HADAMARD'S INEQUALITY16 AN INTERLUDE: KARAMATA'S INEQUALITY17 KARAMATA'S INEQUALITY AND EIGENVALUES18 AN INEQUALITY CONCERNING POSITIVE SEMIDEFINITE MATRICES19 A REPRESENTATION THEOREM FOR 20 A REPRESENTATION THEOREM FOR (tr A)21 HÖLDER'S INEQUALITY22 CONCAVITY OF log|A|23 MINKOWSKI'S INEQUALITY24 QUASILINEAR REPRESENTATION OF |A|25 MINKOWSKI'S DETERMINANT THEOREM26 WEIGHTED MEANS OF ORDER p 27 SCHLÖMILCH'S INEQUALITY28 CURVATURE PROPERTIES OF Mp (x, a)29 LEAST SQUARES30 GENERALIZED LEAST SQUARES31 RESTRICTED LEAST SQUARES32 RESTRICTED LEAST SQUARES: MATRIX VERSION
Part Five: The linear model
Chapter 12: Statistical preliminaries
1 INTRODUCTION2 THE CUMULATIVE DISTRIBUTION FUNCTION3 THE JOINT DENSITY FUNCTION4 EXPECTATIONS5 VARIANCE AND COVARIANCE6 INDEPENDENCE OF TWO RANDOM VARIABLES7 INDEPENDENCE OF n RANDOM VARIABLES8 SAMPLING9 THE ONE‐DIMENSIONAL NORMAL DISTRIBUTION10 THE MULTIVARIATE NORMAL DISTRIBUTION11 ESTIMATION
Chapter 13: The linear regression model
1 INTRODUCTION 2 AFFINE MINIMUM‐TRACE UNBIASED ESTIMATION 3 THE GAUSS‐MARKOV THEOREM 4 THE METHOD OF LEAST SQUARES 5 AITKEN'S THEOREM 6 MULTICOLLINEARITY 7 ESTIMABLE FUNCTIONS 8 LINEAR CONSTRAINTS: THE CASEℳ(R) ⊂ ℳ(X) 9 LINEAR CONSTRAINTS: THE GENERAL CASE 10 LINEAR CONSTRAINTS: THE CASEℳ(R) ∩ ℳ(X) = {0} 11 A SINGULAR VARIANCE MATRIX: THE CASEℳ(X) ⊂ ℳ(V) 12 A SINGULAR VARIANCE MATRIX: THE CASE r(X′V X) = r(X) 13 A SINGULAR VARIANCE MATRIX: THE GENERAL CASE, I 14 EXPLICIT AND IMPLICIT LINEAR CONSTRAINTS 15 THE GENERAL LINEAR MODEL, I 16 A SINGULAR VARIANCE MATRIX: THE GENERAL CASE, II 17 THE GENERAL LINEAR MODEL, II 18 GENERALIZED LEAST SQUARES 19 RESTRICTED LEAST SQUARES
Chapter 14: Further topics in the linear model
1 INTRODUCTION2 BEST QUADRATIC UNBIASED ESTIMATION OF σ 3 THE BEST QUADRATIC AND POSITIVE UNBIASED ESTIMATOR OF σ 4 THE BEST QUADRATIC UNBIASED ESTIMATOR OF σ 5 BEST QUADRATIC INVARIANT ESTIMATION OF σ 6 THE BEST QUADRATIC AND POSITIVE INVARIANT ESTIMATOR OF σ 7 THE BEST QUADRATIC INVARIANT ESTIMATOR OF σ 8 BEST QUADRATIC UNBIASED ESTIMATION: MULTIVARIATE NORMAL CASE9 BOUNDS FOR THE BIAS OF THE LEAST‐SQUARES ESTIMATOR OF σ, I10 BOUNDS FOR THE BIAS OF THE LEAST‐SQUARES ESTIMATOR OF σ, II11 THE PREDICTION OF DISTURBANCES12 BEST LINEAR UNBIASED PREDICTORS WITH SCALAR VARIANCE MATRIX13 BEST LINEAR UNBIASED PREDICTORS WITH FIXED VARIANCE MATRIX, I14 BEST LINEAR UNBIASED PREDICTORS WITH FIXED VARIANCE MATRIX, II15 LOCAL SENSITIVITY OF THE POSTERIOR MEAN16 LOCAL SENSITIVITY OF THE POSTERIOR PRECISION
Part Six: Applications to maximum likelihood estimation
Chapter 15: Maximum likelihood estimation
1 INTRODUCTION2 THE METHOD OF MAXIMUM LIKELIHOOD (ML)3 ML ESTIMATION OF THE MULTIVARIATE NORMAL DISTRIBUTION4 SYMMETRY: IMPLICIT VERSUS EXPLICIT TREATMENT5 THE TREATMENT OF POSITIVE DEFINITENESS6 THE INFORMATION MATRIX7 ML ESTIMATION OF THE MULTIVARIATE NORMAL DISTRIBUTION: DISTINCT MEANS8 THE MULTIVARIATE LINEAR REGRESSION MODEL9 THE ERRORS‐IN‐VARIABLES MODEL10 THE NONLINEAR REGRESSION MODEL WITH NORMAL ERRORS11 SPECIAL CASE: FUNCTIONAL INDEPENDENCE OF MEAN AND VARIANCE PARAMETERS12 GENERALIZATION OF THEOREM 15.6
Chapter 16: Simultaneous equations
1 INTRODUCTION2 THE SIMULTANEOUS EQUATIONS MODEL3 THE IDENTIFICATION PROBLEM4 IDENTIFICATION WITH LINEAR CONSTRAINTS ON B AND Γ ONLY5 IDENTIFICATION WITH LINEAR CONSTRAINTS ON B, Γ, AND Σ6 NONLINEAR CONSTRAINTS7 FIML: THE INFORMATION MATRIX (GENERAL CASE)8 FIML: ASYMPTOTIC VARIANCE MATRIX (SPECIAL CASE)9 LIML: FIRST‐ORDER CONDITIONS10 LIML: INFORMATION MATRIX11 LIML: ASYMPTOTIC VARIANCE MATRIXBIBLIOGRAPHICAL NOTES
Chapter 17: Topics in psychometrics
1 INTRODUCTION2 POPULATION PRINCIPAL COMPONENTS3 OPTIMALITY OF PRINCIPAL COMPONENTS4 A RELATED RESULT5 SAMPLE PRINCIPAL COMPONENTS6 OPTIMALITY OF SAMPLE PRINCIPAL COMPONENTS7 ONE‐MODE COMPONENT ANALYSIS8 ONE‐MODE COMPONENT ANALYSIS AND SAMPLE PRINCIPAL COMPONENTS9 TWO‐MODE COMPONENT ANALYSIS10 MULTIMODE COMPONENT ANALYSIS11 FACTOR ANALYSIS12 A ZIGZAG ROUTINE13 NEWTON‐RAPHSON ROUTINE14 KAISER'S VARIMAX METHOD15 CANONICAL CORRELATIONS AND VARIATES IN THE POPULATION16 CORRESPONDENCE ANALYSIS17 LINEAR DISCRIMINANT ANALYSISBIBLIOGRAPHICAL NOTES
Part Seven: Summary
Chapter 18: Matrix calculus: the essentials
1 INTRODUCTION2 DIFFERENTIALS3 VECTOR CALCULUS4 OPTIMIZATION5 LEAST SQUARES6 MATRIX CALCULUS7 INTERLUDE ON LINEAR AND QUADRATIC FORMS8 THE SECOND DIFFERENTIAL9 CHAIN RULE FOR SECOND DIFFERENTIALS10 FOUR EXAMPLES11 THE KRONECKER PRODUCT AND VEC OPERATOR12 IDENTIFICATION13 THE COMMUTATION MATRIX14 FROM SECOND DIFFERENTIAL TO HESSIAN15 SYMMETRY AND THE DUPLICATION MATRIX16 MAXIMUM LIKELIHOODFURTHER READING
Bibliography
Index of symbols
Subject index
End User License Agreement

Content preview from Matrix Differential Calculus with Applications in Statistics and Econometrics, 3rd Edition

Chapter 12Statistical preliminaries

1 INTRODUCTION

The purpose of this chapter is to review briefly those statistical concepts and properties that we shall use in the remainder of this book. No attempt is made to be either exhaustive or rigorous.

It is assumed that the reader is familiar (however vaguely) with the concepts of probability and random variables and has a rudimentary knowledge of Riemann integration. Integrals are necessary in this chapter, but they will not appear in any other chapter of this book.

2 THE CUMULATIVE DISTRIBUTION FUNCTION

If x is a real‐valued random variable, we define the cumulative distribution function F by

Thus, F(ξ) specifies the probability that the random variable x is at most equal to a given number ξ. It is clear that F is nondecreasing and that

Similarly, if (x₁, … , x_n)′ is an n × 1 vector of real random variables, we define the cumulative distribution function F by

which specifies the probability of the joint occurrence x_i ≤ ξ_i for all i.

3 THE JOINT DENSITY FUNCTION

Let F be the cumulative distribution function of a real‐valued random variable x. If there exists a nonnegative real‐valued (in fact, Lebesgue‐measurable) function f such that ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Linear Algebra and Matrix Analysis for Statistics

Publisher Resources

ISBN: 9781119541202Purchase book

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business