基于CNNCIFG-Attention模型的文本情感分类

doi:10.16180/j.cnki.issn1007-7820.2022.02.008

Abstract

Abstract:

Neural networks are weak in text salient feature extraction and have relatively slow learning rate in processing Chinese text sentiment classification tasks. To solve this problem, this study proposes a hybrid network model based on attention mechanism. This study preprocesses the text corpus, uses the traditional convolutional neural network to extract the feature of the local information of the sample vector. Then, extracted features are input into the coupled input and forget gate network model to learn the connection between the preceding and following words and sentences. Subsequently, the attention mechanism layer is added to assign weights to deep-level text information to improve the intensity of the influence of important information on text sentiment classification. Finally, the proposed hybrid network model is tested on the crawled JD product review collection. The test results show that the accuracy of the proposed method reaches 92.13%, and the F-Score value is 92.06%, which proves the feasibility of the CNNCIFG-Attention model.

Key words: sentiment classification, hybrid network model, convolutional neural network, feature extraction, coupled input and forget gate network, attention model, weight distribution, accuracy, F-Score value

CLC Number:

TP391.1

LI Hui,WANG Yicheng. CNNCIFG-Attention Model for Text Sentiment Classifcation[J].Electronic Science and Technology, 2022, 35(2): 46-51.

Figures/Tables 8

Figure 1.

Figure 2.

Figure 3.

Table 1.

Table 2.

Table 3.

Figure 4.

Figure 5.

References 18

[1]	Dragoni M. A three-phase approach for exploiting opinion mining in computational advertising[J]. IEEE Intelligent Systems, 2017, 32(3):21-27. doi: 10.1109/MIS.2017.46
[2]	房乐楠, 何腾鹏, 刘宇红. 一种改进型PSO算法在SVM参数寻优中的应用[J]. 电子科技, 2018, 31(6):17-19.
	Fang Lenan, He Tengpeng, Liu Yuhong. Application of an improved PSO algorithm in SVM parameter optimization[J]. Electronic Science and Technology, 2018, 31(6):17-19.
[3]	Bilal M, Israr H, Shahid M, et al. Sentiment classification of Roman-Urdu opinions using Nave Bayesian, decision tree and KNN classification techniques[J]. Journal of King Saud University-Computer and Information Sciences, 2016, 28(3):330-344. doi: 10.1016/j.jksuci.2015.11.003
[4]	Huang M L, Qian Q, Zhu X Y. Encoding syntactic knowledge in neural networks for sentiment classification[J]. ACM Transactions on Information Systems, 2017, 35(3):1-27.
[5]	Poria S, Peng H Y, Hussain A, et al. Ensemble application of convolutional neural networks and multiple kernel learning for multimodal sentiment analysis[J]. Neurocomputing, 2017, 26(1):217-230.
[6]	Stojanovski D, Strezoski G, Madjarov G, et al. Deep neural network architecture for sentiment analysis and emotion identification of Twitter messages[J]. Multimedia Tools and Applications, 2018, 77(24):32213-32242. doi: 10.1007/s11042-018-6168-1
[7]	梁斌, 刘全, 徐进, 等. 基于多注意力卷积神经网络的特定目标情感分析[J]. 计算机研究与发展, 2017, 54(8):1724-1735.
	Liang Bin, Liu Quan, Xu Jin, et al. Aspect-based sentiment analysis based on multi-attention CNN[J]. Journal of Computer Research and Development, 2017, 54(8):1724-1735.
[8]	Rehman A U, Malik A K, Raza B, et al. A hybrid CNN-LSTM model for improving accuracy of movie reviews sentiment analysis[J]. Multimedia Tools and Applications, 2019, 78(18):26597-26613. doi: 10.1007/s11042-019-07788-7
[9]	Shuang K, Ren X T, Yang Q Q, et al. AELA-DLSTMs:attention-enabled and location-aware double LSTMs for aspect-level sentiment classification[J]. Neurocomputing, 2019, 33(4):25-34.
[10]	王伟, 王洪伟, 孟园. 协同过滤推荐算法研究:考虑在线评论情感倾向[J]. 系统工程理论与实践, 2014, 34(12):3238-3249.
	Wang Wei, Wang Hongwei, Meng Yuan. The collaborative filtering recommendation based on sentiment analysis of online reviews[J]. Systems Engineering-Theory & Practice, 2014, 34(12):3238-3249.
[11]	张平平. 卷积CIFG模型在影评情感分析中的应用[D]. 上海:上海师范大学, 2019.
	Zhang Pingping. Application of convolutional CIFG model in sentiment analysis of film review[D]. Shanghai:Shanghai Normal University, 2019.
[12]	Liu M F, Hu H J, Li L J, et al. Chinese image caption generation via visual attention and topic modeling[J]. IEEE Transactions on Cybernetics, 2020, 9(9):1-11.
[13]	Choi H, Cho K, Bengio Y. Fine-grained attention mechanism for neural machine translation[J]. Neurocomputing, 2018, 284(5):171-176. doi: 10.1016/j.neucom.2018.01.007
[14]	Zhou C T, Sun C L, Liu Z Y, et al. A C-LSTM neural network for text classification[J]. Computer Science, 2015, 1(4):39-44.
[15]	Li B Y, Zhou K M, Gao W, et al. Attention-based LSTM-CNNs for uncertainty identification on Chinese social media texts[C]. Shenzhen:Proceedings of the International Conference on Security,Pattern Analysis,and Cybernetics, 2017.
[16]	Zhang Y S, Jiang Y R, Tong Y X. Study of sentiment classification for Chinese microblog based on recurrent neural network[J]. Chinese Journal of Electronics, 2016, 25(4):601-607. doi: 10.1049/cje.v25.4
[17]	Dawn E H, Lakhmi C J. Innovations in machine learning[M]. Heidelberg:Springer, 2006.
[18]	Church W K. Word2vec[J]. Natural Language Engineering, 2017, 23(1):155-162. doi: 10.1017/S1351324916000334

数据集	样本情感属性	样本数量	句子平均长度
京东商品评论	积极	10 435	70
京东商品评论	消极	10 221	70
豆瓣电影评论	积极	12 942	240
豆瓣电影评论	消极	9 878	240

模型参数	取值
词向量维度	128
滤波器尺寸	3,4,5
滤波器个数	100,100,100
CIFG单元数量	128
Dropout率	0.5
Batch_size	64
迭代次数	20

数据集	模型	准确率	F-Score
京东商品评论	Naive Byes	85.26CNNCIFG	85.16CNNCIFG
	SVM	86.08CNNCIFG	86.18CNNCIFG
	CNN	87.31CNNCIFG	87.28CNNCIFG
	CNN-LSTM	89.14CNNCIFG	89.03CNNCIFG
	CNN-CIFG	90.02CNNCIFG	89.88CNNCIFG
	CNNCIFG-Attention	92.13CNNCIFG	92.06CNNCIFG
豆瓣电影评论	Naive Byes	85.11CNNCIFG	85.21CNNCIFG
	SVM	86.04CNNCIFG	86.14CNNCIFG
	CNN	87.20CNNCIFG	87.19CNNCIFG
	CNN-LSTM	88.00CNNCIFG	87.88CNNCIFG
	CNN-CIFG	88.52CNNCIFG	88.46CNNCIFG
	CNNCIFG-Attention	89.75CNNCIFG	89.68CNNCIFG

CNNCIFG-Attention Model for Text Sentiment Classifcation

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 8

References 18

Related Articles 15

Metrics

Comments

Recommended 10

[1]	WU Weijia,YANG Jian,YUAN Tianchen,SHAO Zhihui. Research on Track Structure Damage Identification Based on Support Vector Machine [J]. Electronic Science and Technology, 2022, 35(2): 27-33.
[2]	SHAO Zhihui,YANG Jian,YUAN Tianchen,WU Weijia. Sleeper Diseases Diagnosis Based on Permutation Entropy and Support Vector Machine [J]. Electronic Science and Technology, 2022, 35(2): 52-58.
[3]	ZHAN Yinwei,ZHU Baiwan,YANG Zhuo. Research and Application of Vehicle Face Recognition Algorithm [J]. Electronic Science and Technology, 2021, 34(8): 1-7.
[4]	LIAO Luming,ZHANG Wei. Batch Mixed Training Traffic Sign Recognition Based on Improved VGG16 Network [J]. Electronic Science and Technology, 2021, 34(8): 8-13.
[5]	SI Mingming,CHEN Wei,HU Chunyan,YIN Zhong. Fundus Blood Vessel Image Segmentation Combining Resnet50 and U-Net [J]. Electronic Science and Technology, 2021, 34(8): 19-24.
[6]	MA Lixin,DOU Chenfei,SONG Chencan,YANG Tianxiao. Insulator Nondestructive Testing Based on Feature Fusion CNN [J]. Electronic Science and Technology, 2021, 34(7): 26-30.
[7]	GU Wei,LI Feifei,CHEN Qiu. Pedestrian Detection Algorithm Based on Multiple Feature Fusion [J]. Electronic Science and Technology, 2021, 34(5): 29-34.
[8]	MENG Xiaojiao,ZHANG Shiwei,LI Xiaojian,LI Minyue,SONG Bingxin,LU Hongmin. An Evaluation Method of Vehicle-Mounted Communication Equipment Performance Based on BP Neural Network [J]. Electronic Science and Technology, 2021, 34(5): 24-28.
[9]	LIU Shu,SHAO Jie,ZHANG Yiting,ZHANG Shanzhang. ECG Classification Based on Bispectrum and Spectral Features [J]. Electronic Science and Technology, 2021, 34(5): 42-46.
[10]	ZHANG Jiapeng,LI Lin,ZHU Ye. A Review of Research on Decision-Making Method of Autonomous Vehicle Based on Reinforcement Learning [J]. Electronic Science and Technology, 2021, 34(5): 66-71.
[11]	SONG Zhangming,HE Huiyong,HUANG Yuejun. Study on On-Line Detection of Surface Defects of Flat Enameled Wire [J]. Electronic Science and Technology, 2021, 34(5): 72-78.
[12]	XUE Yongjie,JU Zhiyong. Fish Recognition Algorithm Based on Improved AlexNet [J]. Electronic Science and Technology, 2021, 34(4): 12-17.
[13]	GE Jing,LIU Zilong. The Algorithm Based on CNN and LSTM for Sleep Apnea Syndrome Detection [J]. Electronic Science and Technology, 2021, 34(2): 21-26.
[14]	ZHANG Can,CHEN Wei,YIN Zhong. Semantic Segmentation of Cervical Cell Image Based on Weak Supervision [J]. Electronic Science and Technology, 2021, 34(12): 68-74.
[15]	ZHANG Ying,LIU Zilong,WAN Wei. UAV Vehicle Target Detection Based on Faster R-CNN [J]. Electronic Science and Technology, 2021, 34(11): 11-20.