book

Adaptive Filtering

by Alexander D. Poularikas

December 2017

Intermediate to advanced

363 pages

6h 39m

English

CRC Press

Read now

Unlock full access

1.1 Introduction1.1.1 Multiplication by a Constant and Addition and Subtraction1.1.1.1 Multiplication by a Constant1.1.1.2 Addition and Subtraction1.1.2 Unit Coordinate Vectors1.1.3 Inner Product1.1.4 Distance between Two Vectors1.1.5 Mean Value of a Vector1.1.6 Direction Cosines1.1.7 The Projection of a Vector1.1.8 Linear Transformations1.2 Linear Independence, Vector Spaces, and Basis Vectors1.2.1 Orthogonal Basis VectorsProblemsHints–Suggestions–Solutions

2.1 Introduction2.2 General Types of Matrices2.2.1 Diagonal, Identity, and Scalar Matrices2.2.2 Upper and Lower Triangular Matrices2.2.3 Symmetric and Exchange Matrices2.2.4 Toeplitz Matrix2.2.5 Hankel and Hermitian2.3 Matrix Operations2.4 Determinant of a Matrix2.4.1 Definition and Expansion of a Matrix2.4.2 Trace of a Matrix2.4.3 Inverse of a Matrix2.5 Linear Equations2.5.1 Square Matrices (n × n)2.5.2 Rectangular Matrices (n < m)2.5.3 Rectangular Matrices (m < n)2.5.4 Quadratic and Hermitian Forms2.6 Eigenvalues and Eigenvectors2.6.1 Eigenvectors2.6.2 Properties of Eigenvalues and EigenvectorsProblemsHints–Suggestions–Solutions
3.1 Discrete-Time Signals3.1.1 Time-Domain Representation of Basic Continuous and Discrete Signals3.2 Transform-Domain Representation of Discrete Signals3.2.1 Discrete-Time Fourier Transform3.2.2 The Discrete FT3.2.3 Properties of DFT3.3 The z-Transform3.4 Discrete-Time Systems3.4.1 Linearity and Shift Invariant3.4.2 Causality3.4.3 Stability3.4.4 Transform-Domain RepresentationProblemsHints–Suggestions–Solutions
4.1 Discrete Random Signals, Probability Distributions, and Averages of Random Variables4.1.1 Stationary and Ergodic Processes4.1.2 Averages of RV4.1.2.1 Mean Value4.1.2.2 Correlation4.1.2.3 Covariance4.2 Stationary Processes4.2.1 Autocorrelation Matrix4.2.2 Purely Random Process (White Noise)4.2.3 Random Walk4.3 Special Random Signals and pdf’s4.3.1 White Noise4.3.2 Gaussian Distribution (Normal Distribution)4.3.3 Exponential Distribution4.3.4 Lognormal Distribution4.3.5 Chi-Square Distribution4.4 Wiener–Khinchin Relations4.5 Filtering Random Processes4.6 Special Types of Random Processes4.6.1 Autoregressive Process4.7 Nonparametric Spectra Estimation4.7.1 Periodogram4.7.2 Correlogram4.7.3 Computation of Periodogram and Correlogram Using FFT4.7.4 General Remarks on the Periodogram4.7.4.1 Windowed Periodogram4.7.5 Proposed Book Modified Method for Better Frequency Resolution4.7.5.1 Using Transformation of the rv’s4.7.5.2 Blackman–Tukey Method4.7.6 Bartlett Periodogram4.7.7 The Welch Method4.7.8 Proposed Modified Welch Methods4.7.8.1 Modified Method Using Different Types of Overlapping4.7.8.2 Modified Welch Method Using Transformation of rv’sProblemsHints–Solutions–Suggestions
5.1 Introduction5.2 The LS Technique5.2.1 Linear LS5.2.2 LS Formulation5.2.3 Statistical Properties of LSEs5.2.4 The LS Approach5.2.5 Orthogonality Principle5.2.6 Corollary5.2.7 Projection Operator5.2.8 LS Finite Impulse Response Filter5.3 The Mean-Square Error5.3.1 The FIR Wiener Filter
5.4.1 Orthogonality Condition5.4.2 Normalized Performance Equation5.4.3 Canonical Form of the Error-Performance Surface5.5 Wiener Filtering Examples5.5.1 Minimum MSE5.5.2 Optimum Filter (wo)5.5.3 Linear PredictionProblemsAdditional ProblemsHints–Solutions–SuggestionsAdditional Problems
6.1 The Eigenvalues of the Correlation Matrix6.1.1 Karhunen–Loeve Transformation6.2 Geometrical Properties of the Error SurfaceProblemsHints–Solutions–Suggestions
7.1 One-Dimensional Gradient Search Method7.1.1 Gradient Search Algorithm7.1.2 Newton’s Method in Gradient Search7.2 Steepest Descent Algorithm7.2.1 Steepest Descent Algorithm Applied to Wiener Filter7.2.2 Stability (Convergence) of the Algorithm7.2.3 Transient Behavior of MSE7.2.4 Learning Curve7.3 Newton’s Method7.4 Solution of the Vector Difference EquationProblemsEdition ProblemsHints–Solutions–SuggestionsAdditional Problems
8.1 Introduction8.2 The LMS Algorithm8.3 Examples Using the LMS Algorithm8.4 Performance Analysis of the LMS Algorithm8.4.1 Learning Curve8.4.2 The Coefficient-Error or Weighted-Error Correlation Matrix8.4.3 Excess MSE and Misadjustment8.4.4 Stability8.4.5 The LMS and Steepest Descent Methods8.5 Complex Representation of the LMS AlgorithmProblemsHints–Solutions–Suggestions
9.1 The Normalized Least Mean-Square Algorithm9.2 Power Normalized LMS9.3 Self-Correcting LMS Filter9.4 The Sign-Error LMS Algorithm9.5 The NLMS Sign-Error Algorithm9.6 The Sign-Regressor LMS Algorithm9.7 Self-Correcting Sign-Regressor LMS Algorithm9.8 The Normalized Sign-Regressor LMS Algorithm9.9 The Sign–Sign LMS Algorithm
9.11 Variable Step-Size LMS9.12 The Leaky LMS Algorithm9.13 The Linearly Constrained LMS Algorithm9.14 The Least Mean Fourth Algorithm9.15 The Least Mean Mixed Norm LMS Algorithm9.16 Short-Length Signal of the LMS Algorithm9.17 The Transform Domain LMS Algorithm9.17.1 Convergence9.18 The Error Normalized Step-Size LMS Algorithm9.19 The Robust Variable Step-Size LMS Algorithm9.20 The Modified LMS Algorithm9.21 Momentum LMS9.22 The Block LMS Algorithm9.23 The Complex LMS Algorithm9.24 The Affine LMS Algorithm9.25 The Complex Affine LMS AlgorithmProblemsHints–Solutions–Suggestions
A1.1 Suggestions and Explanations for MATLAB UseA1.1.1 Creating a DirectoryA1.1.2 HelpA1.1.3 Save and LoadA1.1.4 MATLAB as CalculatorA1.1.5 Variable NamesA1.1.6 Complex NumbersA1.1.7 Array IndexingA1.1.8 Extracting and Inserting Numbers in ArraysA1.1.9 VectorizationA1.1.10 WindowingA1.1.11 MatricesA1.1.12 Producing a Periodic FunctionA1.1.13 Script FilesA1.1.14 FunctionsA1.1.15 Complex ExpressionsA1.1.16 AxesA1.1.17 2D GraphicsA1.1.18 3D PlotsA1.1.18.1 Mesh-Type FiguresA1.2 General Purpose CommandsA1.2.1 Managing Commands and FunctionsA1.2.2 Managing Variables and WorkplaceA1.2.3 Operators and Special CharactersA1.2.4 Control FlowA1.3 Elementary Matrices and Matrix ManipulationA1.3.1 Elementary Matrices and ArraysA1.3.2 Matrix ManipulationA1.4 Elementary Mathematical FunctionsA1.4.1 Elementary FunctionsA1.5 Numerical Linear AlgebraA1.5.1 Matrix AnalysisA1.6 Data AnalysisA1.6.1 Basic OperationsA1.6.2 Filtering and ConvolutionA1.6.3 Fourier TransformsA1.7 2D PlottingA1.7.1 2D Plots
A2.1 DefinitionsA2.2 Special MatricesA2.3 Matrix Operation and FormulasA2.4 Eigendecomposition of MatricesA2.5 Matrix ExpectationsA2.6 Differentiation of a Scalar Function with respect to a Vector
A3.1 Trigonometric IdentitiesA3.2 OrthogonalityA3.3 Summation of Trigonometric FormsA3.4 Summation FormulasA3.4.1 Finite Summation FormulasA3.4.2 Infinite Summation FormulasA3.5 Series ExpansionsA3.6 LogarithmsA3.7 Some Definite Integrals

Overview

This book covers the fundamentals of adaptive filtering, with a focus on the least mean square (LMS) adaptive filter. It discusses random variables, stochastic processes, vectors, matrices, determinants, discrete random signals, and probability distributions, while delivering a concise introduction to MATLAB®—complete with problems, computer experiments, and over 110 functions and script files. The text not only addresses the basics of the LMS adaptive filter algorithm but also explores the Wiener filter and its applications, details the steepest descent method, and develops the Newton’s algorithm.