融入注意力网络的深度分解机推荐算法

doi:10.16180/j.cnki.issn1007-7820.2023.01.006

摘要/Abstract

摘要：

推荐系统能够在海量的信息中找到满足用户个性化需求的信息。随着深度学习的发展,深度学习也开始广泛被推荐系统所应用。CTR预估在推荐系统中起着重要作用,已被应用在个性化推荐、信息检索、在线广告等多个领域。针对推荐系统数据量大且稀疏的问题,文中将注意力网络和xDeepFM模型融合,提出了一种新的基于深度学习的CTR预估模型,即Atte-xDeepFM模型。该模型能够解决特征稀疏问题,有效学习特征之间的交互关系,且不需要手动提取特征工程中的有用信息。在Avazu数据集和Criteo数据集上进行的对比实验证明了文中提出的模型的有效性。与推荐系统CTR预估常用的算法模型对比,文中所提出的模型具有更好的推荐效果。

Abstract:

Recommender system can find information that satisfies the user's individual needs from huge amount of information. With the development of deep learning, deep learning has been widely applied in recommender systems. CTR prediction plays an important role in recommender system and has been widely used in many fields such as personalized recommendation, information retrieval, online advertising and so on. For the issue of large and sparse data in recommender system, this study merges xDeepFM model with attention network, and proposes a new CTR prediction model based on deep learning, which is called Atte-xDeepFM model. This model can solve the issue of feature scarcity, effectively learn the interactions relationship between features, and does not need to manually extract useful information in feature engineering. The comparative experiments on Avazu and Criteo data sets prove the effectiveness of the proposed model. Compared with the algorithm model commonly used in CTR prediction, the proposed model has better recommendation effect.

Key words: recommender system, deep learning, personalized recommendation, computational advertising, CTR prediction, factorization machine, attention network, feature sparsity

中图分类号:

TP393

邬彤,于莲芝. 融入注意力网络的深度分解机推荐算法[J]. 电子科技, 2023, 36(1): 38-43.

WU Tong,YU Lianzhi. The Recommendation Algorithm of Extreme Deep Factorization Machine Merged with Attention Network[J]. Electronic Science and Technology, 2023, 36(1): 38-43.

图/表 4

参考文献 29

[1]	项亮. 推荐系统实践[M]. 北京: 人民邮电出版社, 2012.
	Xiang Liang. Practice of recommender system[M]. Beijing: The People's Posts and Telecommunications Press, 2012.
[2]	Pavan K, Vairachilai S, Potluri S, et al. Recommender systems: Algorithms and applications[M]. Boca Raton: CRC Press, 2021.
[3]	Hsu C L. A multi-valued and sequential-labeled decision tree method for recommending sequential patterns in cold-start situations[J]. Applied Intelligence, 2021, 51(1):506-526. doi: 10.1007/s10489-020-01806-0
[4]	Wang Z, Yu Q, Shen C, et al. Feature selection in click-through rate prediction based on gradient boosting[M].Berlin:International Conference on Intelligent Data Engineering and Automated Learning, 2016.
[5]	Wen H, Zhang J, Lin Q, et al. Multi-level deep cascade trees for conversion rate prediction in recommendation System[C]. Honolulu: Proceedings of the Thirty-third AAAI Conference on Artificial Intelligence, 2019.
[6]	Ling X, Deng W, Gu C, et al. Model ensemble for click prediction in bing search ads[C]. Perth: Proceedings of the Twenty-sixth International Conference on World Wide Web Companion, 2019.
[7]	McMahan H B, Holt G, Scully D, et al. Ad click prediction a view from the trenches[C]. Chicago: Proceedings of the Nineteenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2013.
[8]	Ta A P. Factorization machines with follow- the- regularized- leader for CTR prediction in display advertising[C]. Santa Clara: Proceedings of the IEEE International Conference on Big Data, 2015.
[9]	Zhang S, Yao L, Sun A, et al. Deep learning based recommender system:A survey and new perspectives[C]. Como: Proceedings of the Eleventh ACM Conference on Recommender Systems, 2017.
[10]	Cheng H T, Koc L, Harmsen J, et al. Wide & Deep learning for recommender systems[C]. Boston: Proceedings of the First Workshop on Deep Learning for Recommender Systems, 2016.
[11]	Guo H, Tang R, Ye Y, et al. DeepFM: A factorization-machine based neural network for CTR prediction[C]. Melbourne: Proceedings of the Twenty-sixth International Joint Conference on Artifcial Intelligence, 2017.
[12]	Strub F, Gaudel R, Mary J. Hybrid recommender system based on autoencoders[C]. Boston: Proceedings of the First Workshop on Deep Learning for Recommender Systems, 2016.
[13]	Tahmasebi H, Ravanmehr R, Mohamadrezaei R. Social movie recommender system based on deep autoencoder network using Twitter data[J]. Neural Computing and Applications, 2021, 33(5):1607-1623. doi: 10.1007/s00521-020-05085-1
[14]	Gong Y, Zhang Q. Hashtag recommendation using Attention-based convolutional neural network[C]. New York: Proceedings of the Conference Division and the AI Journal Division, 2016.
[15]	Suglia A, Greco C, Musto C, et al. A deep architecture for content-based recommendations exploiting recurrent neural networks[C]. Bratislava: Proceedings of the Twenty-fifth Conference on User Modeling, Adaptation and Personalization, 2017.
[16]	许凤翔. 一种改进相似度的协同过滤算法实现[J]. 电子科技, 2020, 33(2):54-59.
	Xu Fengxiang. Implementation of a collaborative filtering algorithm based on improved similarity[J]. Electronic Science and Technology, 2020, 33(2):54-59.
[17]	卢佳伟, 陈玮, 尹钟. 融合TextRank算法的中文短文本相似度计算[J]. 电子科技, 2020, 33(10):51-56.
	Lu Jiawei, Chen Wei, Yin Zhong. Chinese short text similarity calculation based on TextRank algorithm[J]. Electronic Science and Technology, 2020, 33(10):51-56.
[18]	Gao L, Dai K, Gao L, et al. Expert knowledge recommendation systems based on conceptual similarity and space mapping[J]. Expert Systems with Applications, 2019, 13(6):242-251.
[19]	Behera D K, Da S M, Swetanisha S. Predicting users' preferences for movie recommender system using rstricted boltzmann machine[M]. Singapore: Springer Nature, 2019.
[20]	Rendle S. Factorization machines[C]. Sydney: Proceedings of the Tenth IEEE International Conference on Data Mining, 2010.
[21]	Juan Y, Zhuang Y, Chin W S, et al. Field-aware factorization machines for CTR prediction[C]. Boston: Proceedings of the Tenth ACM Conference on Recommender System, 2016.
[22]	Blondel M, Fujino A, Ueda N, et al. Higher-order factorization machines[C]. New York: Proceedings of the Advances in Neural Information Processing Systems, 2016.
[23]	Xiao J, Ye H, He X, et al. Attentional factorization machines:Learning the weight of feature interactions via attention networks[C]. Melbourne: Proceedings of the Twenty-sixth International Joint Conference on Artificial Intelligence, 2017.
[24]	Zhang W, Du T, Wang J. Deep learning over multi-field categorical data[C]. Springer: Proceedings of the European Conference on Information Retrieval, 2016.
[25]	Qu Y, Han C, Kan R, et al. Product-based neural networks for user response prediction[C]. Barcelona: Proceedings of the IEEE International Conference on Data Mining, 2016.
[26]	Wang R, Fu B, Fu G, et al. Deep & Cross network for Ad click predictions[C]. Halifax: Proceedings of the Twenty-third ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2017.
[27]	He X, Chua T S. Neural factorization machines for sparse predictive analytics[J]. Proceeding of the Fortieth International ACM SIGIR Conference on Research and Development in Information Retrieval, 2017.
[28]	Lian J, Zhou X, Zhang F, et al. xDeepFM:Combining explicit and implicit feature interactions for recommender systems[C]. London: Proceedings of the Twenty-fourth ACM SIGKDD International Conference, 2018.
[29]	Hu J, Shen L, Sun G, et al. Squeeze-and-excitation networks[C]. Salt Lake City: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018.

数据集	样本量	特征数(稀疏化)	所占空间
Avazu	41 MB	14 kB	5.6 GB
Criteo	45 MB	45 kB	15.2 GB

模型名称	Avazu		Criteo
模型名称	AUC	Logloss	AUC	Logloss
AFM	0.742 097	0.354 247	0.736 624	0.492 851
NFM	0.741 831	0.354 501	0.736 438	0.493 016
CIN	0.744 281	0.359 772	0.743 343	0.489 487
Atte-CIN	0.746 040	0.356 845	0.744 328	0.486 356
DeepFM	0.762 503	0.334 862	0.783 259	0.458 730
xDeepFM	0.761 042	0.341 024	0.771 216	0.463 894
Atte-xDeepFM	0.764 892	0.315 846	0.788 733	0.448 124