O'Reilly logo

Music Emotion Recognition by Homer H. Chen, Yi-Hsuan Yang

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Saunder January 24, 2011 10:39 book
Contents
Preface ................................................................................................. xi
Abbreviations.................................................................................... xiii
1 Introduction.................................................................................1
1.1 Importance of Music Emotion Recognition ......................................1
1.2 Recognizing the Perceived Emotion of Music....................................4
1.3 Issues of Music Emotion Recognition ...............................................6
1.3.1 Ambiguity and Granularity of Emotion Description ............6
1.3.2 Heavy Cognitive Load of Emotion Annotation....................7
1.3.3 Subjectivity of Emotional Perception ...................................8
1.3.4 Semantic Gap between Low-Level Audio Signal
and High-Level Human Perception .....................................9
1.4 Summary.........................................................................................12
2 Overview of Emotion Description and Recognition...............15
2.1 Emotion Description.......................................................................15
2.1.1 Categorical Approach ........................................................16
2.1.2 Dimensional Approach......................................................18
2.1.3 Music Emotion Variation Detection .................................20
2.2 Emotion Recognition......................................................................21
2.2.1 Categorical Approach ........................................................22
2.2.1.1 Data Collection.................................................23
2.2.1.2 Data Preprocessing............................................25
2.2.1.3 Subjective Test..................................................26
2.2.1.4 Feature Extraction.............................................28
2.2.1.5 Model Training.................................................28
2.2.2 Dimensional Approach......................................................29
2.2.3 Music Emotion Variation Detection .................................31
2.3 Summary.........................................................................................32
v
Saunder January 24, 2011 10:39 book
vi Contents
3 Music Features ...........................................................................35
3.1 Energy Features...............................................................................36
3.2 Rhythm Features.............................................................................37
3.3 Temporal Features ..........................................................................42
3.4 Spectrum Features...........................................................................44
3.5 Harmony Features...........................................................................51
3.6 Summary.........................................................................................54
4 Dimensional MER by Regression .............................................55
4.1 Adopting the Dimensional Conceptualization
of Emotion......................................................................................55
4.2 VA Prediction .................................................................................57
4.2.1 Weighted Sum of Component Functions ..........................57
4.2.2 Fuzzy Approach.................................................................58
4.2.3 System Identification Approach (System ID).....................58
4.3 The Regression Approach................................................................59
4.3.1 Regression Theory .............................................................59
4.3.2 Problem Formulation ........................................................60
4.3.3 Regression Algorithms .......................................................60
4.3.3.1 Multiple Linear Regression................................60
4.3.3.2 -Support Vector Regression.............................61
4.3.3.3 AdaBoost Regression Tree (AdaBoost.RT) ........62
4.4 System Overview.............................................................................62
4.5 Implementation ..............................................................................63
4.5.1 Data Collection .................................................................63
4.5.2 Feature Extraction .............................................................65
4.5.3 Subjective Test...................................................................67
4.5.4 Regressor Training.............................................................67
4.6 Performance Evaluation ..................................................................68
4.6.1 Consistency Evaluation of the Ground Truth....................68
4.6.2 Data Transformation.........................................................70
4.6.3 Feature Selection................................................................71
4.6.4 Accuracy of Emotion Recognition .....................................74
4.6.5 Performance Evaluation for Music Emotion
Variation Detection...........................................................77
4.6.6 Performance Evaluation for Emotion Classification...........78
4.7 Summary.........................................................................................79
5 Ranking-Based Emotion Annotation and Model Training .....81
5.1 Motivation......................................................................................81
5.2 Ranking-Based Emotion Annotation...............................................82
Saunder January 24, 2011 10:39 book
Contents vii
5.3 Computational Model for Ranking Music
by Emotion .....................................................................................84
5.3.1 Learning-to-Rank ..............................................................85
5.3.2 Ranking Algorithms...........................................................85
5.3.2.1 RankSVM .........................................................85
5.3.2.2 ListNet..............................................................85
5.3.2.3 RBF-ListNet......................................................87
5.4 System Overview.............................................................................90
5.5 Implementation ..............................................................................90
5.5.1 Data Collection .................................................................92
5.5.2 Feature Extraction .............................................................95
5.6 Performance Evaluation ..................................................................96
5.6.1 Cognitive Load of Annotation ...........................................97
5.6.2 Accuracy of Emotion Recognition .....................................98
5.6.2.1 Comparison of Different Feature
Representations .................................................99
5.6.2.2 Comparison of Different Learning
Algorithms ......................................................100
5.6.2.3 Sensitivity Test ................................................102
5.6.3 Subjective Evaluation of the Prediction Result.................104
5.7 Discussion.....................................................................................104
5.8 Summary.......................................................................................105
6 Fuzzy Classification of Music Emotion ..................................107
6.1 Motivation....................................................................................107
6.2 Fuzzy Classification.......................................................................108
6.2.1 Fuzzy k-NN Classifier .....................................................108
6.2.2 Fuzzy Nearest-Mean Classifier.........................................109
6.3 System Overview...........................................................................112
6.4 Implementation ............................................................................113
6.4.1 Data Collection ...............................................................113
6.4.2 Feature Extraction and Feature Selection .........................113
6.5 Performance Evaluation ................................................................114
6.5.1 Accuracy of Emotion Classification..................................114
6.5.2 Music Emotion Variation Detection................................114
6.6 Summary.......................................................................................117
7 Personalized MER and Groupwise MER.................................119
7.1 Motivation....................................................................................119
7.2 Personalized MER.........................................................................121
7.3 Groupwise MER ...........................................................................122
Saunder January 24, 2011 10:39 book
viii Contents
7.4 Implementation ............................................................................124
7.4.1 Data Collection ...............................................................124
7.4.2 Personal Information Collection......................................126
7.4.3 Feature Extraction ...........................................................127
7.5 Performance Evaluation ................................................................128
7.5.1 Performance of the General Method................................128
7.5.2 Performance of GWMER................................................130
7.5.3 Performance of PMER.....................................................130
7.6 Summary.......................................................................................134
8 Two-Layer Personalization .....................................................135
8.1 Problem Formulation....................................................................135
8.2 Bag-of-Users Model ......................................................................136
8.3 Residual Modeling and Two-Layer Personalization Scheme ..........137
8.4 Performance Evaluation ................................................................139
8.5 Summary.......................................................................................143
9 Probability Music Emotion Distribution Prediction.............145
9.1 Motivation....................................................................................145
9.2 Problem Formulation....................................................................146
9.3 The KDE-Based Approach to Music Emotion
Distribution Prediction .................................................................148
9.3.1 Ground Truth Collection ................................................148
9.3.2 Regressor Training...........................................................150
9.3.2.1 ν-Support Vector Regression...........................151
9.3.2.2 Gaussian Process Regression............................151
9.3.3 Regressor Fusion..............................................................153
9.3.3.1 Weighted by Performance ...............................153
9.3.3.2 Optimization...................................................154
9.3.4 Output of Emotion Distribution .....................................156
9.4 Implementation ............................................................................157
9.4.1 Data Collection ...............................................................157
9.4.2 Feature Extraction ...........................................................157
9.5 Performance Evaluation ................................................................161
9.5.1 Comparison of Different Regression Algorithms..............161
9.5.2 Comparison of Different Distribution
Modeling Methods..........................................................162
9.5.3 Comparison of Different Feature Representations ...........165
9.5.4 Evaluation of Regressor Fusion ........................................166
9.6 Discussion.....................................................................................167
9.7 Summary.......................................................................................172
Saunder January 24, 2011 10:39 book
Contents ix
10 Lyrics Analysis and Its Application to MER ...........................173
10.1 Motivation..................................................................................173
10.2 Lyrics Feature Extraction.............................................................174
10.2.1 Uni-Gram ....................................................................175
10.2.2 Probabilistic Latent Semantic Analysis (PLSA) .............176
10.2.3 Bi-Gram .......................................................................177
10.3 Multimodal MER System ...........................................................179
10.4 Performance Evaluation ..............................................................181
10.4.1 Comparison of Multimodal Fusion Methods ...............181
10.4.2 Performance of PLSA Model........................................183
10.4.3 Performance of Bi-Gram Model...................................184
10.5 Summary.....................................................................................184
11 Chord Recognition and Its Application to MER.....................187
11.1 Chord Recognition .....................................................................187
11.1.1 Beat Tracking and PCP Extraction...............................188
11.1.2 Hidden Markov Model and N-Gram Model ................188
11.1.3 Chord Decoding...........................................................190
11.2 Chord Features............................................................................191
11.2.1 Longest Common Chord Subsequence.........................192
11.2.2 Chord Histogram .........................................................192
11.3 System Overview.........................................................................193
11.4 Performance Evaluation ..............................................................193
11.4.1 Evaluation of Chord Recognition System .....................193
11.4.2 Accuracy of Emotion Classification ..............................194
11.5 Summary.....................................................................................196
12 Genre Classification and Its Application to MER...................197
12.1 Motivation..................................................................................197
12.2 Two-Layer Music Emotion Classification ...................................198
12.3 Performance Evaluation ..............................................................199
12.3.1 Data Collection ............................................................199
12.3.2 Analysis of the Correlation between Genre
and Emotion.................................................................200
12.3.3 Evaluation of the Two-Layer Emotion
Classification Scheme ...................................................203
12.3.3.1 Computational Model................................203
12.3.3.2 Evaluation Measures...................................203
12.3.3.3 Results........................................................204
12.4 Summary.....................................................................................205

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required