基于改进ExfuseNet模型的街景语义分割

doi:10.16180/j.cnki.issn1007-7820.2022.06.005

Abstract

Abstract:

When using the ExfuseNet model for streetscape semantic segmentation, due to the high background complexity of the street view image, the area ratio and distribution between the classes of interest are unbalanced. Interesting targets with low area and low density in the image are more likely to be misclassified as they go deeper into the network, which ultimately leads to the degradation of model segmentation performance. To solve this problem, an improved Exfusenet model is proposed. In order to obtain the semantic information of different scales without increasing the amount of model parameters, the multi-monitor module adopts atrous convolution with different rates. After the down-sampling features are fused, the random discarding layer is used immediately to reduce the amount of model parameters and improve the generalization ability. Before the main output, the CBAM attention mechanism module is used to sample the depth semantic information of the target class of interest more efficiently, and the class balance function is used after the multi-supervision module to improve the class imbalance problem of the data set Camvid. The experimental results show that the semantic segmentation effect of the improved ExfuseNet model has been significantly improved, MIOU has increased to 68.32%, and the classification accuracy rate of the Pole class has increased to 38.14%.

Key words: street view image, multiple supervision, dilated rate, dilated convolution, random drop layer, generalization, attention mechanism, class balance, mean intersection over union

CLC Number:

TP391.4

CHEN Jinhong,CHEN Wei,YIN Zhong. Semantic Segmentation of Streetscape Based on Improved ExfuseNet[J].Electronic Science and Technology, 2022, 35(6): 28-34.

Figures/Tables 13

Figure 1.

Figure 2.

Figure 3.

Figure 4.

Figure 5.

Figure 6.

Figure 7.

Figure 8.

Table 1.

Figure 9.

Figure 10.

Table 2.

Table 3.

References 20

[1]	Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(4):640-651. doi: 10.1109/TPAMI.2016.2572683 pmid: 27244717
[2]	Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition[J]. Computer Science, 2014, 15(56):3431-3440.
[3]	Badrinarayanan V, Kendall A, Cipolla R. Segnet: A deep convolution encoder-decoder architecture for scene segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12):2481-2495. doi: 10.1109/TPAMI.2016.2644615 pmid: 28060704
[4]	Ronneberger O, Fischer P, Brox T. U-net:Convolutional networks for biomedical image segmentation[C]. Munich: Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, 2015.
[5]	Zhao H, Shi J, Qi X, et al. Pyramid scene parsing network[C]. Honolulu: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017.
[6]	Peng C, Zhang X, Yu G, et al. Large kernal matters--improve semantic segmentation by global convolutional network[C]. Honolulu: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017.
[7]	Krizhevsky A, Sutskever I, Hinton G E. Imagenet classification with deep convolutional neural networks[J]. Advances in Neural Information Processing Systems, 2012, 25(1):1097-1105.
[8]	LécCun Y, Bottou L, Bengio Y, et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998, 86(11):2278-2324. doi: 10.1109/5.726791
[9]	Zhang Z, Zhang X, Peng C, et al. Exfuse:Enhancing feature fusion for semantic segmentation[C]. Munich: Proceedings of the European Conference on Computer Vision, 2018.
[10]	Lin T Y, Goyal P, Girshick R, et al. Focal loss for dense object detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 42(2):318-327. doi: 10.1109/TPAMI.2018.2858826
[11]	Rumelhart D E, Hinton G E, Williams R J. Learning representations by back-propagating errors[J]. Nature, 1986, 323(6):533-536. doi: 10.1038/323533a0
[12]	Srivastava N, Hinton G, Krizhevsky A, et al. Dropout: a simple way to prevent neural networks from overfitting[J]. The Journal of Machine Learning Research, 2014, 15(1):1929-1958.
[13]	Woo S, Park J, Lee J Y, et al. CBAM:Convolutional block attention module[C]. Munich: Proceedings of the European Conference on Computer Vision, 2018.
[14]	董波, 周燕, 王永雄. 基于渐进结构感受野和全局注意力的显著性检测[J]. 电子科技, 2021, 34(1):23-30.
	Dong Bo, Zhou Yan, Wang Yongxiong. Saliency detection by progressive structural receptive field and global attention[J]. Electronic Science and Technology, 2021, 34(1):23-30.
[15]	Fu J, Liu J, Tian H, et al. Dual attention network for scene segmentation[C]. Long Beach: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019.
[16]	Hu J, Shen L, Sun G. Squeeze-and-excitation networks[J]. Pattern Analysis and Machine Intelligence, 2019, 42(8):2011-2023. doi: 10.1109/TPAMI.2019.2913372
[17]	鲜开义, 杨利萍, 周仁彬, 等. 变电站巡检机器人道路语义分割方法及其应用[J]. 科学技术与工程, 2020, 20(15):6151-6157.
	Xian Kaiyi, Yang Liping, Zhou Renbin, et al. Road semantic segmentation method for substation inspection robot and it’s application[J]. Science Technology and Engineering, 2020, 20(15):6151-6157.
[18]	罗嗣卿, 张志超, 岳琪. 基于改进SEGNET模型的图像语义分割[J]. 计算机工程, 2020, 47(4):256-261.
	Luo Siqing, Zhang Zhichao, Yue Qi. Semantic segmentation of image based on improved SEGNET model[J]. Computer Engineering, 2020, 47(4):256-261.
[19]	宋辉. 基于深度学习多路径特征融合的图像语义分割[D]. 北京: 北京邮电大学, 2019.
	Song Hui. Multi-path feature fusion network based on deep learning for semantic image segmentation[D]. Beijing: Beijing University of Posts and Telecommunications, 2019.
[20]	曹峰梅, 田海杰, 付君, 等. 结合特征图切分的图像语义分割[J]. 中国图象图形学报, 2019, 24(3):464-473.
	Cao Fengmei, Tian Haijie, Fu Jun, et al. Feature map slice for semantic segmentation[J]. Journal of Image and Graphics, 2019, 24(3):464-473.

方法/评价指标	1号模型	2号模型	3号模型	4号模型	5号模型	6号模型	7号模型
Baseline	?	?	?	?	?	?	?
SS_Astous(d=1,6,12,18)				?	?	?	?
CBAM_attention		?	?	?	?		?
DANet_attention^[15]							?
SENet_attention^[16]						?
Dropout			?	?	?	?	?
Focal Loss					?	?	?
MIOU/%	62.9	60.7	64.2	63.9	68.3	67.9	63.2
Class_Pole_ACC/%	17.3	12.6	25.7	22.4	38.1	31.6	18.2

方法	年代	MIOU/%	Class_Pole_ACC/%
文献[17]	2020	0.561	未提及
SEGNET+GAN^[18]	2020	0.652	0.358
文献[19]	2019	0.662	0.373
文献[20]	2019	0.663	未提及
本文模型	2021	0.683	0.381

方法	帧率/frame·s^-1	模型大小/MB
ExfuseNet模型	4.08	293
改进模型	6.45	162

Semantic Segmentation of Streetscape Based on Improved ExfuseNet

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 13

References 20

Related Articles 8

Metrics

Comments

Recommended 10

[1]	Chaowei LIN,Feifei LI,Qiu CHEN. Globaland Local Scene Representation Method Based on Deep Convolutional Features [J]. Electronic Science and Technology, 2022, 35(4): 20-27.
[2]	Lingyu JI,Yongbin GAO,Chenglu ZHAO,Xianhua TANG,Kaicheng XU,Jiacheng XU. CTA Segmentation Algorithm of Abdominal Artery Based on 3D Fully Convolutional Network [J]. Electronic Science and Technology, 2022, 35(3): 38-44.
[3]	ZHANG Shisen,SUN Xiankun,YIN Ling,LI Shixi. Design of Text Title Generation Prototype System Based on Neural Network [J]. Electronic Science and Technology, 2021, 34(5): 35-41.
[4]	XUE Yongjie,JU Zhiyong. Fish Recognition Algorithm Based on Improved AlexNet [J]. Electronic Science and Technology, 2021, 34(4): 12-17.
[5]	LI Fupeng,FU Dongxiang. Sentiment Analysis Method of Financial Text Based on Transformer Encoder [J]. Electronic Science and Technology, 2020, 33(9): 10-15.
[6]	ZHENG Meng. Research on Intelligent English Translation Method Based on Improved Attention Mechanism Model [J]. Electronic Science and Technology, 2020, 33(11): 84-87.
[7]	HE Wenjie,LIU Jingbiao,PAN Mian,LÜ Shuaishuai. Natural Scene Text Recognition Algorithm Based on Attention-CTC [J]. Electronic Science and Technology, 2019, 32(12): 32-36.
[8]	CHENG Lingfei,HE Yang,ZHANG Peiling,LI Yan. Classification Performance of Compressing Dimensionality of Hidden Layer of Deep Neural Network [J]. Electronic Science and Technology, 2019, 32(1): 72-75.