Machine Learning, 2nd Edition

Appendix DThe Geometry of Linear Functions and Linear Classifiers

Linear Functions and Hyperplanes

Linear functions play an important role in the operation and design of several classification algorithms: for example, logistic regression, neural networks, support vector machines, and linear discriminant analysis. This appendix presents some geometric properties of linear functions which are used, explicitly or implicitly, in the design and description of these algorithms.

A linear function¹ is a function of the form

where is a -long vector of real parameters. For notational convenience or emphasis this may be written as

where .

Parameter vector can be thought of as encoding three geometrically meaningful quantities:²

• A magnitude, given by the length of the vector .
• A direction ...