基于脑电信号和瞬时情感强度标签的情感识别方法

doi:10.16180/j.cnki.issn1007-7820.2024.11.011

摘要/Abstract

摘要：

通过机器学习脑电图(Electroencephalogram, EEG)揭示人类大脑活动已成为探索人类内在情感状态的重要方案。由于情感状态变化是动态而非恒定,因此预测情感状态变化是情感识别领域的研究难题。文中提出了一种瞬时情感强度标签生成框架,通过让受试者观看视频来刺激并采集其瞬时情绪强度从而生成一组有监督标签,结合有监督标签与脑电特征生成3组半监督标签对应受试者的瞬时情感状态变化。使用脑电特征与多种机器学习方法分析4组标签对情感状态变化的适用性。通过支持向量机模型在有监督情感强度标签集上对两类、三类和四类情感强度取得了80.02%,54.76%和56.14%的分类精度。实验结果表明,有监督瞬时情感强度标签对不同受试者的脑电数据和情感状态变化更具普适性。

关键词: 机器学习, 脑电图, 情感状态, 情感识别, 标签生成, 瞬时情感强度标签, 普适性标签, 被试特异性标签

Abstract:

Revealing human brain activity through machine learning EEG(Electroencephalogram) has become an important scheme to explore the inner emotional state of humans. Because the change of emotion state is dynamic rather than constant, it is difficult to predict the change of emotion state in the field of emotion recognition. This study proposes a label generation framework for instantaneous emotion intensity. A set of supervised labels is generated by having subjects watch videos that stimulate and capture their instantaneous emotional intensity, and combine the supervised labels with EEG features to generate three sets of semi-supervised labels to correspond to the instantaneous emotional state changes of subjects. In this study, EEG features and various machine learning methods are used to analyze the applicability of four groups of labels to emotional state changes. The support vector machine model achieves 80.02%, 54.76% and 56.14% classification accuracy for two-class, three-class and four-class sentiment intensities on supervised label sets. The experimental results show that the supervised instantaneous emotion intensity labels are more universal for EEG data and emotional state changes across different subjects.

Key words: machine learning, electroencephalogram, emotional state, emotion recognition, label generation, instantaneous emotion intensity label, universal label, subject specific labels

中图分类号:

TP391

甘开宇, 尹钟. 基于脑电信号和瞬时情感强度标签的情感识别方法[J]. 电子科技, 2024, 37(11): 78-84.

GAN Kaiyu, YIN Zhong. Emotion Recognition Method Based on EEG and Instantaneous Emotion Intensity Label[J]. Electronic Science and Technology, 2024, 37(11): 78-84.

图/表 9

表1

图1

图2

图3

图4

图5

表2

表3

表4

参考文献 22

[1]	Maiorana E, Solé-Casals J, Campisi P. EEG signal preprocessing for biometric recognition[J]. Machine Vision & Applications, 2016, 27(8):1-10.
[2]	Han J, Zhang Z, Ren Z, et al. Emobed:Strengthening monomodal emotion recognition via training with crossmodal emotion embeddings[J]. IEEE Transactions on Affective Computing, 2019, 12(3):553-564.
[3]	Dolan R J. Emotion,cognition and behavior[J]. Science, 2002, 298(5596):1191-1194. doi: 10.1126/science.1076358 pmid: 12424363
[4]	Puk K M, Wan S, Rosenberger J, et al. Emotion recognition and EEG analysis using admm-based sparse group lasso[J]. IEEE Transactions on Affective Computing, 2019, 13(1):199-210.
[5]	Alarcäo S M, Fonseca M J. Emotions recognition using EEG signals:A survey[J]. IEEE Transactions on Affective Computing, 2019, 10(3):374-393.
[6]	Wang Y, Song W, Tao W, et al. A systematic review on affective computing: Emotion models, databases and recent advances[J]. Information Fusion, 2022, 83(7):19-52.
[7]	Huang Y J, Wu C Y, Wong M K, et al. Novel active comb-shaped dry electrode for EEG measurement in hairy site[J]. IEEE Transactions on Biomedical Engineering, 2015, 62(1):256-263.
[8]	Katsigiannis S, Ramzan N. DREAMER:A database for emotion recognition through EEG and ECG signals from wireless low-cost off-the-shelf devices[J]. IEEE Journal of Biomed Health Inform, 2018, 22(1):98-107.
[9]	张悦, 胡春燕. 基于有记忆递归神经网络的脑电特征情感识别研究[J]. 电子科技, 2020, 33(11):67-72.
	Zhang Yue, Hu Chunyan. Research on emotion recognition of EEG features based on the long short-term memory neural network[J]. Electronic Science and Technology, 2020, 33(11):67-72.
[10]	Zhang F, Li X C, Lim C P, et al. Deep emotional arousal network for multimodal sentiment analysis and emotion recognition[J]. Information Fusion, 2022, 88(12):296-304.
[11]	Alonso-Fernandez F, Raja K B, Raghavendra R, et al. Cross-sensor periocular biometrics in a global pandemic:Comparative benchmark and novel multialgorithmic approach[J]. Information Fusion, 2022, 83(7):110-130.
[12]	Wang Z, Wang Y X, Zhang J P, et al. Spatial-temporal feature fusion neural network for EEG-based emotion recognition[J]. IEEE Transactions on Instrumentation and Measurement, 2022, 71(4):1-12.
[13]	Wang Z, Wang Y X, Zhang J P, et al. Transformers for EEG-based emotion recognition:A hierarchical spatial information learning model[J]. IEEE Sensors Journal, 2022, 22(5):4359-4368.
[14]	Zhang Y, Liu H, Zhang D, et al. EEG-based emotion recognition with emotion localization via hierarchical self-attention[J]. IEEE Transactions on Affective Computing, 2022, 14(8):1-12.
[15]	Zheng W L, Zhu J Y, Peng Y, et al. EEG-based emotion classification using deep belief networks[C]. Chengdu:IEEE International Conference on Multimedia and Expo, 2014:1-6.
[16]	Kwak N, Choi C. Input feature selection by mutual information based on Parzen window[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2002, 24(12):1667-1671.
[17]	Aftanas L, Reva N, Varlamov A, et al. Analysis of evoked EEG synchronization and desynchronization in conditions of emotional activation in humans:Temporal and topographic characteristics[J]. Neuroscience Behavioral Physiology, 2004, 34(8):859-867
[18]	Li M, Lu B L. Emotion classification based on gamma-band EEG[C]. Minneapolis:Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 2009:1223-1226.
[19]	Shi L C, Jiao Y Y, Lu B L. Differential entropy feature for EEG-based vigilance estimation[C]. Osaka: The Thirty-fifth Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 2013:6627-6630.
[20]	Wang G Y, Yin Z, Zhao M Y, et al. Identification of human mental workload levels in a language comprehension task with imbalance neurophysiological data[J]. Computer Methods and Programs in Biomedicine, 2022, 224(9):107-119.
[21]	Zhang J, Yin Z, Wang R. Recognition of mental workload levels under complex human-machine collaboration by using physiological features and adaptive support vector machines[J]. IEEE Transactions on Human-Machine Systems, 2015, 45(2):200-214.
[22]	Rousseeuw P J. Silhouettes:A graphical aid to the interpretation and validation of cluster analysis[J]. Journal of Computational and Applied Mathematics, 1987(20):53-65.

视频编号	情绪类别标签	电影片段来源
1	积极	《人再囧途之泰囧》片段1
2	中立	《世界遗产在中国EP2》片段1
3	消极	《唐山大地震》片段1
4	消极	《一九四二》片段1
5	中立	《世界遗产在中国EP2》片段2
6	积极	《人再囧途之泰囧》片段2
7	消极	《一九四二》片段2
8	中立	《世界遗产在中国EP13》片段1
9	积极	《唐伯虎点秋香》片段1
10	积极	《越光宝盒》片段1
11	中立	《世界遗产在中国EP13》片段2
12	消极	《一九四二》片段3
13	中立	《世界遗产在中国EP30》片段1
14	积极	《越光宝盒》片段2
15	消极	《唐山大地震》片段2

方法	普适性标签集精度/F1分数	特异性标签集1 精度/F1分数	特异性标签集2 精度/F1分数	特异性标签集3 精度/F1分数
SVM	80.02/45.10	50.80/46.44	50.37/46.46	43.75/30.77
KNN	76.89/50.62	50.61/47.85	50.52/48.28	41.79/32.63
XGBoost	78.22/47.27	50.33/45.52	50.24/47.97	42.15/31.04
RF	79.53/46.32	51.37/46.25	50.97/46.22	44.98/29.18
MLP	77.97/47.77	50.29/48.29	50.26/48.66	64.35/27.29

方法	普适性标签集精度/F1分数	特异性标签集1 精度/F1分数	特异性标签集2 精度/F1分数	特异性标签集3 精度/F1分数
SVM	54.76/25.92	42.59/30.62	38.21/34.58	42.12/35.69
KNN	46.45/34.78	41.04/31.94	37.63/34.56	39.93/35.11
XGBoost	53.23/30.26	42.55/30.54	38.92/32.32	41.98/32.65
RF	53.83/28.93	42.47/29.61	38.00/31.00	42.20/34.36
MLP	51.26/29.89	42.94/29.42	38.75/30.24	40.26/30.99

方法	普适性标签集精度/F1分数	特异性标签集1 精度/F1分数	特异性标签集2 精度/F1分数	特异性标签集3 精度/F1分数
SVM	56.14/25.47	39.17/21.80	38.21/34.58	32.07/26.82
KNN	46.44/25.97	36.56/22.99	37.63/34.56	30.90/26.48
XGBoost	51.66/24.87	38.98/22.47	35.49/28.10	32.62/27.13
RF	53.96/21.27	39.29/21.08	37.95/33.06	31.90/25.15
MLP	52.98/20.49	37.82/22.66	34.08/22.76	31.44/22.39