Music Emotion Recognition

Saunder January 24, 2011 10:39 book

2

Overview of Emotion

Description and

Recognition

It has been claimed that music is one of the finest languages of emotion [252]. The

relationship between music and emotion has been studied by researchers from several

disciplines. Many researchers have suggested that music is an excellent medium for

studying emotion, because people tend to make judgments about music and their

affective responses to music [102,159]. The purpose of this chapter is to provide an

overview of the techniques developed for emotion description and emotion recogni-

tion. Specifically, this chapter is divided into two parts. The first part discusses the

emotion models that have been proposed by psychologists, so that the readers can

get a sense of how emotions are usually conceptualized. The second part describes

some well-known work on MER and provides the details of the basic components

of an MER system. The concepts introduced in this chapter are important to the

understanding of the remaining chapters of the book.

2.1 Emotion Description

The relationship between music and emotion has been well studied by psycholo-

gists for decades. The research problems faced by psychologists include whether the

everyday emotions are the same as emotions that are perceived in music, whether

music represents emotions (that are perceived by the listener) or induces emotions

(that are felt by the listener), how the musical, personal, and situational factors af-

fect emotion perception, and how we should conceptualize music emotion [159].

15

Saunder January 24, 2011 10:39 book

16  Music Emotion Recognition

As there has been rich literature on these research topics, below we focus on emotion

conceptualization alone, for it is closely related to MER.

In the study of emotion conceptualization, psychologists often utilize people’s

verbal reports of emotion responses [159]. For example, the celebrated paper of

Hevner [126] studied the relationship between music and emotion through experi-

ments in which subjects are asked to report the adjectives that came to their minds as

the most representative part of a music piece played. From these empirical studies,

a great variety of emotion models have been proposed, most of which belong to

one of the following two approaches to emotion conceptualization: the categorical

approach and the dimensional approach.

2.1.1 Categorical Approach

The categorical approach to emotion conceptualization considers that people ex-

perience emotions as categories that are distinct from each other. Essential to this

approach is the concept of basic emotions, that is, the idea that there are a lim-

ited number of innate and universal emotion categories, such as happiness, sadness,

anger, fear, disgust, and surprise, from which all other emotion classes can be de-

rived [22, 82, 154, 166, 186, 192, 253, 288]. Each “basic” emotion can be defined

functionally in terms of a key appraisal of goal-relevant events that have occurred

frequently during evolution. The basic emotions can be found in all cultures, and

they are often associated with distinct patterns of physiological changes or emotional

expressions. The notion of basic emotions is diversified; different researchers have

come up with different sets of basic emotions [299].

Another famous categorical approach to emotion conceptualization is Hevner’s

adjective checklist [126, 127]. Through experiments, eight clusters of affective ad-

jectives are discovered and laid out in a circle, as shown in Figure 2.1. The adjectives

within each cluster are similar, whereas the meaning of neighboring clusters varies in

a cumulative way until reaching a contrast in the opposite position. Hevner’s adjec-

tive checklist (proposed in 1935) was later refined and regrouped into ten groups by

Farnsworth in 1954 [87, 88] and into nine groups in 2003 by Schubert [285]. For

example, in [285], 46 affective adjectives are grouped according to their positions

on a two-dimensional emotion space. The resulting nine clusters and the associated

affective adjectives are shown in Table 2.1.

The major drawback of the categorical approach is that the number of primary

emotion classes is too small in comparison with the richness of music emotion per-

ceived by humans. Using a finer granularity, on other hand, does not necessarily

solve the problem because the language for describing emotions is inherently am-

biguous and varies from person to person [158]. Moreover, using a large number of

emotion classes could overwhelm the subjects and is impractical for psychological

studies [299].

Saunder January 24, 2011 10:39 book

Overview of Emotion Description and Recognition  17

Figure 2.1 Hevner’s eight clusters of affective terms [126, 127].

Table 2.1 The Nine Emotion Clusters Proposed by E. Schubert in 2003 [285]

Cluster Emotions in Each Cluster

1 Bright, cheerful, happy, joyous

2 Humorous, light, lyrical, merry, playful

3 Calm, delicate, graceful, quiet, relaxed, serene, soothing, tender,

tranquil

4 Dreamy, sentimental

5 Dark, depressing, gloomy, melancholy, mournful, sad, solemn

6 Heavy, majestic, sacred, serious, spiritual, vigorous

7 Tragic, yearning

8 Agitated, angry, restless, tense

9 Dramatic, exciting, exhilarated, passionate, sensational, soaring,

triumphant