基于改进ExfuseNet模型的街景语义分割

doi:10.16180/j.cnki.issn1007-7820.2022.06.005

摘要/Abstract

摘要：

使用ExfuseNet模型进行街景语义分割时,由于街景图像背景复杂度较高,造成感兴趣类之间的面积占比与分布不均衡,特别是图像中面积占比低且密度低的感兴趣目标,越到网络深层越容易被错误分类,最终导致模型分割性能下降。为解决该问题,文中对ExfuseNet模型进行了改进。为了获取不同尺度的语义信息,在不增加模型参数量的条件下,多监督模块采用不同空洞率的带孔卷积。在下采样特征融合后,立刻采用随机丢弃层来减少模型参数量,提高泛化力。在主输出前采用CBAM注意力机制模块以便更高效地对感兴趣目标类的深度语义信息进行采样,并在多监督模块之后采用类平衡函数来改善数据集Camvid的类不平衡问题。实验结果表明,改进的ExfuseNet模型语义分割效果有明显提升,其均交并比提升到了68.32%,Pole类分类准确率提升到38.14%。

关键词: 街景图像, 多监督, 空洞率, 带孔卷积, 随机丢弃层, 泛化力, 注意力机制, 类平衡, 均交并比

Abstract:

When using the ExfuseNet model for streetscape semantic segmentation, due to the high background complexity of the street view image, the area ratio and distribution between the classes of interest are unbalanced. Interesting targets with low area and low density in the image are more likely to be misclassified as they go deeper into the network, which ultimately leads to the degradation of model segmentation performance. To solve this problem, an improved Exfusenet model is proposed. In order to obtain the semantic information of different scales without increasing the amount of model parameters, the multi-monitor module adopts atrous convolution with different rates. After the down-sampling features are fused, the random discarding layer is used immediately to reduce the amount of model parameters and improve the generalization ability. Before the main output, the CBAM attention mechanism module is used to sample the depth semantic information of the target class of interest more efficiently, and the class balance function is used after the multi-supervision module to improve the class imbalance problem of the data set Camvid. The experimental results show that the semantic segmentation effect of the improved ExfuseNet model has been significantly improved, MIOU has increased to 68.32%, and the classification accuracy rate of the Pole class has increased to 38.14%.

Key words: street view image, multiple supervision, dilated rate, dilated convolution, random drop layer, generalization, attention mechanism, class balance, mean intersection over union

中图分类号:

TP391.4

陈劲宏,陈玮,尹钟. 基于改进ExfuseNet模型的街景语义分割[J]. 电子科技, 2022, 35(6): 28-34.

CHEN Jinhong,CHEN Wei,YIN Zhong. Semantic Segmentation of Streetscape Based on Improved ExfuseNet[J]. Electronic Science and Technology, 2022, 35(6): 28-34.

图/表 13

图1

图2

图3

图4

图5

图6

图7

图8

表1

图9

图10

表2

表3

参考文献 20

[1]	Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(4):640-651. doi: 10.1109/TPAMI.2016.2572683 pmid: 27244717
[2]	Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition[J]. Computer Science, 2014, 15(56):3431-3440.
[3]	Badrinarayanan V, Kendall A, Cipolla R. Segnet: A deep convolution encoder-decoder architecture for scene segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12):2481-2495. doi: 10.1109/TPAMI.2016.2644615 pmid: 28060704
[4]	Ronneberger O, Fischer P, Brox T. U-net:Convolutional networks for biomedical image segmentation[C]. Munich: Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, 2015.
[5]	Zhao H, Shi J, Qi X, et al. Pyramid scene parsing network[C]. Honolulu: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017.
[6]	Peng C, Zhang X, Yu G, et al. Large kernal matters--improve semantic segmentation by global convolutional network[C]. Honolulu: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017.
[7]	Krizhevsky A, Sutskever I, Hinton G E. Imagenet classification with deep convolutional neural networks[J]. Advances in Neural Information Processing Systems, 2012, 25(1):1097-1105.
[8]	LécCun Y, Bottou L, Bengio Y, et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998, 86(11):2278-2324. doi: 10.1109/5.726791
[9]	Zhang Z, Zhang X, Peng C, et al. Exfuse:Enhancing feature fusion for semantic segmentation[C]. Munich: Proceedings of the European Conference on Computer Vision, 2018.
[10]	Lin T Y, Goyal P, Girshick R, et al. Focal loss for dense object detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 42(2):318-327. doi: 10.1109/TPAMI.2018.2858826
[11]	Rumelhart D E, Hinton G E, Williams R J. Learning representations by back-propagating errors[J]. Nature, 1986, 323(6):533-536. doi: 10.1038/323533a0
[12]	Srivastava N, Hinton G, Krizhevsky A, et al. Dropout: a simple way to prevent neural networks from overfitting[J]. The Journal of Machine Learning Research, 2014, 15(1):1929-1958.
[13]	Woo S, Park J, Lee J Y, et al. CBAM:Convolutional block attention module[C]. Munich: Proceedings of the European Conference on Computer Vision, 2018.
[14]	董波, 周燕, 王永雄. 基于渐进结构感受野和全局注意力的显著性检测[J]. 电子科技, 2021, 34(1):23-30.
	Dong Bo, Zhou Yan, Wang Yongxiong. Saliency detection by progressive structural receptive field and global attention[J]. Electronic Science and Technology, 2021, 34(1):23-30.
[15]	Fu J, Liu J, Tian H, et al. Dual attention network for scene segmentation[C]. Long Beach: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019.
[16]	Hu J, Shen L, Sun G. Squeeze-and-excitation networks[J]. Pattern Analysis and Machine Intelligence, 2019, 42(8):2011-2023. doi: 10.1109/TPAMI.2019.2913372
[17]	鲜开义, 杨利萍, 周仁彬, 等. 变电站巡检机器人道路语义分割方法及其应用[J]. 科学技术与工程, 2020, 20(15):6151-6157.
	Xian Kaiyi, Yang Liping, Zhou Renbin, et al. Road semantic segmentation method for substation inspection robot and it’s application[J]. Science Technology and Engineering, 2020, 20(15):6151-6157.
[18]	罗嗣卿, 张志超, 岳琪. 基于改进SEGNET模型的图像语义分割[J]. 计算机工程, 2020, 47(4):256-261.
	Luo Siqing, Zhang Zhichao, Yue Qi. Semantic segmentation of image based on improved SEGNET model[J]. Computer Engineering, 2020, 47(4):256-261.
[19]	宋辉. 基于深度学习多路径特征融合的图像语义分割[D]. 北京: 北京邮电大学, 2019.
	Song Hui. Multi-path feature fusion network based on deep learning for semantic image segmentation[D]. Beijing: Beijing University of Posts and Telecommunications, 2019.
[20]	曹峰梅, 田海杰, 付君, 等. 结合特征图切分的图像语义分割[J]. 中国图象图形学报, 2019, 24(3):464-473.
	Cao Fengmei, Tian Haijie, Fu Jun, et al. Feature map slice for semantic segmentation[J]. Journal of Image and Graphics, 2019, 24(3):464-473.

方法/评价指标	1号模型	2号模型	3号模型	4号模型	5号模型	6号模型	7号模型
Baseline	?	?	?	?	?	?	?
SS_Astous(d=1,6,12,18)				?	?	?	?
CBAM_attention		?	?	?	?		?
DANet_attention^[15]							?
SENet_attention^[16]						?
Dropout			?	?	?	?	?
Focal Loss					?	?	?
MIOU/%	62.9	60.7	64.2	63.9	68.3	67.9	63.2
Class_Pole_ACC/%	17.3	12.6	25.7	22.4	38.1	31.6	18.2

方法	年代	MIOU/%	Class_Pole_ACC/%
文献[17]	2020	0.561	未提及
SEGNET+GAN^[18]	2020	0.652	0.358
文献[19]	2019	0.662	0.373
文献[20]	2019	0.663	未提及
本文模型	2021	0.683	0.381

方法	帧率/frame·s^-1	模型大小/MB
ExfuseNet模型	4.08	293
改进模型	6.45	162