Autonomous Learning Systems: From Data Streams to Knowledge in Real-time

8

Autonomous Learning Classifiers

8.1 Classifying Data Streams

One traditional approach to classifying data streams is the incremental classifier (Fung and Mangasariany, 2002). In the literature there are various classification frameworks that work in an incremental mode (per sample), for example, decision trees (Yuan and Shaw, 1995), neural network such as adaptive resonance theory, ART (Carpenter and Grossberg, 2003), incremental learning vector quantiser, iLVQ (Poirier and Ferrieux, 1991), probabilistic such as incremental versions of Bayesian classifiers (Schlimmer and Fisher, 1986), incremental Fisher LDA (Pang, Ozawa and Kasabov, 2004), and so on. It should be stressed, however, that the classifier structure in all incremental classifier methods mentioned above is fixed.

Incremental classifiers are inefficient with respect to the problem of the so-called drift and shift in the data density pattern. In machine learning by drift they refer to a modification of the concept over time that relates to a relatively smooth transition of the data distribution from one local region of the feature space to another (Widmer and Kubat, 1996). It is author's point of view that drift and shift have to be considered from the point of view of data density (not pdf but the density as described earlier in this book).

By shift they traditionally refer in machine learning literature to a more abrupt change such as the sudden appearance of a fault or an abrupt change of a regime of operation (Tsymbal, ...