Contents
Foreword .........................................................................................................................................xix
Preface.............................................................................................................................................xxi
List of Figures..............................................................................................................................xxvii
List of Tables...............................................................................................................................xxxiii
PART I INTRODUCTION TO IDENTIFICATION AND MODELS FOR LINEAR
DETERMINISTIC SYSTEMS
Chapter 1 Introduction ..................................................................................................................2
1.1 Motivation..........................................................................................................................2
1.2 Historical developments ....................................................................................................8
1.3 System Identification.......................................................................................................13
1.3.1 Three Facts of Identification..............................................................................14
1.3.2 Notion of a Model .............................................................................................15
1.3.3 Quantitative vs. Qualitative Models ..................................................................17
1.3.3.1 Deterministic vs. Stochastic Models ................................................18
1.3.3.2 Non-Parametric vs. Parametric Models............................................18
1.4 Systematic identification .................................................................................................19
1.4.1 Data Generation and Acquisition ......................................................................19
1.4.2 Data Pre-Processing...........................................................................................21
1.4.3 Data Visualization .............................................................................................22
1.4.4 Model Development ..........................................................................................22
1.4.5 Model Assessment and Validation.....................................................................24
1.4.6 Prior Process Knowledge...................................................................................24
1.4.7 Suggestions for Obtaining a Good Model.........................................................25
1.5 Flow of learning material ................................................................................................26
1.6 Software...........................................................................................................................29
Chapter 2 A Journey into Identification......................................................................................31
2.1 Identifiability ...................................................................................................................31
2.2 Signal-to-Noise ratio .......................................................................................................34
2.3 Overfitting........................................................................................................................35
2.4 A modeling example: liquid level system .......................................................................38
2.4.1 The Physical Process .........................................................................................38
2.4.2 Data Generation.................................................................................................38
2.4.3 Data Visualization and Preliminary Analysis....................................................40
vii
viii Contents
2.4.4 Building Non-Parametric Models......................................................................41
2.4.5 Building Parametric Models..............................................................................43
2.4.6 Goodness of the Model......................................................................................45
2.4.7 Developing a State-Space Model ......................................................................50
2.5 Reflections and summary ................................................................................................53
Chapter 3 Mathematical Descriptions of Processes: Models......................................................56
3.1 Definition of a model.......................................................................................................56
3.2 Classification of models ..................................................................................................57
3.2.1 Types of Models ................................................................................................58
3.2.2 Models for Identification ...................................................................................66
Chapter 4 Models for Discrete-Time LTI Systems.....................................................................68
4.1 Convolution model ..........................................................................................................68
4.1.1 Impulse Response..............................................................................................69
4.2 Response models .............................................................................................................72
4.2.1 Finite Impulse Response (FIR) Model ..............................................................72
4.2.2 Step Response Model ........................................................................................73
4.2.3 Frequency Response Model ..............................................................................75
4.3 Difference equation form ................................................................................................79
4.3.1 Motivating Remarks ..........................................................................................79
4.3.2 Parametrization of Impulse Response ...............................................................80
4.3.3 Transfer Function Operator ...............................................................................81
4.3.4 Stability and Poles .............................................................................................84
4.4 State-space descriptions...................................................................................................86
4.4.1 Background........................................................................................................86
4.4.2 State Variable.....................................................................................................87
4.4.3 State-Space Models ...........................................................................................90
4.4.3.1 Forms of State-Space Representations.............................................92
4.4.4 State-space ↔ Transfer Function Operator Form..............................................98
4.5 Illustrative example in MATLAB: estimating LTI models............................................101
4.5.1 Data Generation...............................................................................................101
4.5.2 Estimation of FIR Model.................................................................................102
4.5.3 Estimation of Step-Response Model ...............................................................102
4.5.4 Estimation of Difference Equation Model.......................................................103
4.5.5 Estimation of a State-Space Model .................................................................104
4.6 Summary........................................................................................................................107
Chapter 5 Transform-Domain Models for Linear TIme-Invariant Systems .............................109
5.1 Frequency response function.........................................................................................109
5.1.1 Characteristics of FRF.....................................................................................109
5.2 Transfer function form...................................................................................................112
5.2.1 Response to Damped Oscillatory Signals........................................................112
5.2.2 z-Transforms....................................................................................................113
Contents ix
5.2.2.1 Properties of z-Transforms.............................................................115
5.2.3 Transfer Functions...........................................................................................117
5.2.3.1 FRF: Special Case of Transfer Function ........................................121
5.3 Empirical transfer function (ETF).................................................................................123
5.4 Closure...........................................................................................................................125
Chapter 6 Sampling and Discretization ....................................................................................129
6.1 Discretization.................................................................................................................129
6.1.1 Sampled-Data System .....................................................................................131
6.1.2 Zero-Order Hold..............................................................................................131
6.1.3 Sampler............................................................................................................132
6.1.4 State-Space Approach .....................................................................................133
6.1.5 Transfer Function Approach............................................................................136
6.2 Sampling........................................................................................................................141
6.2.1 Choice of Sampling Rate.................................................................................142
6.2.2 Sampling Theorem ..........................................................................................144
6.2.3 Practical Guidelines for Sampling...................................................................146
6.3 Summary........................................................................................................................147
PART II MODELS FOR RANDOM PROCESSES
Chapter 7 Random Processes....................................................................................................151
7.1 Introductory remarks .....................................................................................................151
7.2 Random variables and probability.................................................................................152
7.3 Probability theory ..........................................................................................................153
7.3.1 Probability Distribution Functions ..................................................................154
7.4 Statistical properties of random variables .....................................................................158
7.4.1 Mean and Variance ..........................................................................................158
7.4.2 Multivariate Case.............................................................................................163
7.4.2.1 Covariance and Correlation............................................................165
7.4.3 Partial Correlation ...........................................................................................169
7.5 Random signals and processes .....................................................................................171
7.5.1 Definitions .......................................................................................................171
7.5.2 Notion of Realization ......................................................................................173
7.5.3 Statistical Properties ........................................................................................175
7.5.4 Stationarity ......................................................................................................176
7.5.5 Non-Stationarities............................................................................................178
7.5.6 Ergodicity ........................................................................................................181
7.6 Time-series analysis ......................................................................................................182
7.7 Summary........................................................................................................................184
Chapter 8 Time-Domain Analysis: Correlation Functions .......................................................186
8.1 Motivation......................................................................................................................186
8.2 Auto-covariance function .............................................................................................187
x Contents
8.2.1 Auto-Correlation Function (ACF)...................................................................187
8.3 White-noise process.......................................................................................................190
8.3.1 Theoretical ACFs of Elementary Processes ....................................................192
8.4 Cross-covariance function ............................................................................................195
8.4.1 Properties and Uses of CCF ............................................................................196
8.5 Partial correlation functions ..........................................................................................198
8.5.1 Partial ACF ......................................................................................................198
8.5.2 Partial CCF ......................................................................................................201
8.6 Summary........................................................................................................................202
Chapter 9 Models for Linear Stationary Processes...................................................................204
9.1 Motivation......................................................................................................................204
9.2 Basic ideas.....................................................................................................................205
9.3 Linear stationary processes............................................................................................207
9.3.1 Non-Uniqueness of Time-Series Models ........................................................209
9.4 Moving average models.................................................................................................210
9.4.1 ACVF Signature of an MA Process ................................................................210
9.4.2 Invertibility of an MA Process ........................................................................212
9.5 Auto-regressive models .................................................................................................215
9.5.1 Stationary Representations ..............................................................................216
9.5.2 ACF of AR Processes......................................................................................217
9.5.3 Order Determination and PACF ......................................................................220
9.5.4 Alternative Representations of the AR Process...............................................222
9.5.5 Equivalence Between AR and MA Representations .......................................224
9.6 Auto-regressive moving average models ......................................................................226
9.7 Auto-regressive integrated moving average models......................................................227
9.8 Summary........................................................................................................................234
Chapter 10 Fourier Analysis and Spectral Analysis of Deterministic Signals ...........................238
10.1 Motivation......................................................................................................................238
10.2 Definitions .....................................................................................................................242
10.2.1 Periodic and Aperiodic signals........................................................................242
10.2.2 Energy and Power Signals...............................................................................243
10.2.3 Cross-Covariance Functions for Deterministic Signals...................................244
10.3 Fourier representations of deterministic processes........................................................248
10.3.1 Fourier Series...................................................................................................249
10.3.2 Power Spectrum...............................................................................................250
10.3.3 Fourier Transform............................................................................................251
10.3.4 Discrete-Time Fourier Series...........................................................................253
10.3.5 Discrete-Time Fourier Transform....................................................................255
10.3.6 Properties of DTFT..........................................................................................258
10.4 Discrete Fourier Transform (DFT) ................................................................................262
10.4.1 Spectrum and Spectral Density .......................................................................266
10.5 Summary........................................................................................................................267
Contents xi
Chapter 11 Spectral Representations of Random Processes.......................................................270
11.1 Introduction ...................................................................................................................270
11.2 Power spectral density of a random process..................................................................271
11.2.1 PSD from Ensemble Averaging.......................................................................272
11.2.2 PSD from Auto-Covariance Function .............................................................274
11.2.2.1 Random Periodic Process...............................................................277
11.2.3 Wiener Representations and PSD....................................................................281
11.3 Spectral characteristics of standard processes...............................................................282
11.3.1 White Noise Process........................................................................................282
11.3.2 Spectral Density of ARMA Process: Colored Noise.......................................283
11.4 Cross-spectral density and coherence............................................................................287
11.5 Partial coherence............................................................................................................293
11.6 Spectral factorization.....................................................................................................295
11.7 Summary........................................................................................................................301
PART III ESTIMATION METHODS
Chapter 12 Introduction to Estimation........................................................................................305
12.1 Motivation......................................................................................................................305
12.2 A simple example: constant embedded in noise............................................................305
12.3 Definitions and terminology..........................................................................................307
12.3.1 Goodness of Estimators...................................................................................309
12.4 Types of estimation problems........................................................................................310
12.4.1 Signal Estimation.............................................................................................310
12.4.2 Parameter Estimation.......................................................................................312
12.4.3 State Estimation...............................................................................................313
12.4.4 Other Classifications........................................................................................313
12.5 Estimation methods .......................................................................................................314
12.6 Historical notes..............................................................................................................315
Chapter 13 Goodness of Estimators............................................................................................317
13.1 Introduction ...................................................................................................................317
13.2 Fisher information .........................................................................................................318
13.3 Bias................................................................................................................................322
13.4 Variance .........................................................................................................................322
13.4.1 Minimum Variance Unbiased Estimator .........................................................325
13.5 Efficiency ......................................................................................................................325
13.6 Sufficiency .....................................................................................................................326
13.7 Cramer-Rao’s inequality................................................................................................326
13.7.1 Best Linear Unbiased Estimator......................................................................331
13.8 Asymptotic bias.............................................................................................................332
13.9 Mean square error..........................................................................................................333
13.9.1 Minimum Mean-Square Estimator..................................................................334
13.10 Consistency....................................................................................................................334
Get Principles of System Identification now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
