A Multi-task Model to Detect Saliency and Edge using Hybrid Cost Function

Document Type : Computer Article

Authors

1 Graduated

2 Faculty of Electrical and Computer Engineering, Semnan University

Abstract

Detection of salient objects is done with the aim of identifying and segmenting prominent objects or areas in an image. Fully Convolutional Networks (FCNs) have shown their advantages in salient object detection; however, many previous works have focused on the accuracy of the prominent area without paying attention to its edge. This paper focuses on the complementarity between edge information and salient object one and added an edge recognition module to explicitly model edge information to maintain salient object boundaries. Our proposed network is trying to improve these two tasks simultaneously. The presence of objects with different scales in related datasets is another problem in this area. It requires an appropriate cost function to deal with the imbalance problem between background and foreground in images. So, we have used the hybrid cost function in the training phase, which is not sensitive to the scale of objects and can better manage the problem of spatial coherence and uniformly highlight salient areas without additional parameters. A Comparison of the quantitative and qualitative results obtained by the proposed method with other advanced methods in six widely used protrusion detection datasets shows that the proposed method has a good performance and can quickly identify prominent areas. In particular, according to the quantitative results, our method gets the best result on three widely used test datasets in terms of F-measure and MAE criteria, demonstrating the proposed method's efficiency.

Keywords

Main Subjects

References

[1] A. Borji et al., "Salient Object Detection: A Survey", Computational Visual Media, Vol. 5, 2019, pp. 117–150.
[2] W. Wang et al., "Salient Object Detection in the Deep Learning Era: An In-Depth Survey", arXiv: 1904.09146, 2019.
[3] J. Long, E. Shelhamer, and T. Darrell. “Fully Convolutional Networks for Semantic Segmentation”, IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp. 3431-3440.
[4] Y. Ji, H. Zhang, Z., Zhang and M. Liu, "CNN-Based Encoder-Decoder Networks for Salient Object Detection: A Comprehensive Review and Recent Advances, "Information Sciences", Vol. 546, 2021, pp. 835-857.
[5] N. Liu and J. Han. “DHSNet: Deep Hierarchical Saliency Network for Salient Object Detection”, IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 678-686.
[6] N. Liu, J. Han, and M.-H. Yang., “Picanet: Learning Pixel-wise Contextual Attention for Saliency Detection” , IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 3089-3098.
[7] L. Zhang et al., “A Bi-directional Message Passing Model for Salient Object Detection”, IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 1741-1750.
[8] P. Zhang et al., “Amulet: Aggregating Multi-level Convolutional Features for Salient Object Detection”, IEEE International Conference on Computer Vision, 2017, pp. 202-211.
[9] J.-X. Zhao et al., “EGNet: Edge Guidance Network for Salient Object Detection”, IEEE International Conference on Computer Vision, 2019, pp. 8779-8788.
[10] A.K. Gupta et al., "Salient Object Detection Techniques in Computer Vision—A Survey", Entropy, Vol. 22, NO. 10, 2020, pp. 1174.
[11] Y. Pang et al., "Multi-scale Interactive Network for Salient Object Detection", IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 9413-9422.
[12] A. Borji et al., "Salient Object Detection: A Benchmark", IEEE Transactions on Image Processing, Vol. 24, NO. 12, pp. 5706-5722.
[13] محمود معلم، علی‌اکبر پویان، "کشف ناهنجاری با استفاده از کد کننده خودکار مبتنی بر بلوک‌های LSTM"، مدل سازی در مهندسی، دوره 17، شماره 56، بهار 1398، صفحه 191-211.
[14] الهام پارسایی مهر، مهدی فرتاش و جواد اکبری ترکستانی، "بهبود استخراج ویژگی با استفاده از یک مدل یادگیری عمیق گروهی برای تشخیص موجودیت"، مدل سازی در مهندسی، دوره 20، شماره 69، تیر 1401، صفحه 103-112.
[15] راضیه راستگو و کوروش کیانی، "شناسایی چهره با استفاده از تنطیم دقیق شبکه های کانولوشنی عمیق و رویکرد یادگیری انتقالی"، مدل‌سازی در مهندسی، دوره 17، شماره 58 ، پائیز 1398، صفحه 103-111.
[16] S. He et al., “Delving into Salient Object Subitizing and Detection”, IEEE International Conference on Computer Vision, 2017, pp. 1059-1067.
[17] Z. Wu, L. Su, and Q. Huang, “Stacked Cross Refinement Network for Edge-aware Salient Object Detection”, IEEE/CVF International Conference on Computer Vision, 2019, pp. 7264-7273.
[18] L. Zhang et al., “Capsal: Leveraging Captioning to Boost Semantics for Salient Object Detection”, IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 6024-6033.
[19] W. Wang  et al., “Salient Object Detection with Pyramid Attention and Salient Edges”, IEEE Conference on Computer Vision and Pattern Recognition, 2019, pp. 1448-1457.
[20] Y. Zeng et al., “Joint Learning of Saliency Detection and Weakly Supervised Semantic Segmentation”,  IEEE/CVF International Conference on Computer Vision, 2019, pp. 7223-7233.
[21] R. Wu et al., “A Mutual Learning Method for Salient Object Detection with Intertwined Multi-superVision”, IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 8150-8159.
[22] J. Wei et al., “Label Decoupling Framework for Salient Object Detection”, IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 13025-13034.
[23] Qin, X., et al. “Basnet: Boundary-aware Salient Object Detection”, IEEE Conference on Computer Vision and Pattern Recognition, 2019, pp. 7479-7489.
[24] S. Xie and Z. Tu. “Holistically-nested Edge Detection”, IEEE international Conference on Computer Vision, 2015, pp. 1395-1403.
[25] Wang, L., et al., "Salient Object Detection with Recurrent Fully Convolutional Networks", IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 41, NO. 7, 2018, pp. 1734-1746.
[26] Luo, Z., et al. “Non-local Deep Features for Salient Object Detection”, IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 6609-6617.
[27] M. Feng, H. Lu, and E. Ding, “Attentive Feedback Network for Boundary-aware Salient Object Detection”, IEEE Conference on Computer Vision and Pattern Recognition, 2019, pp. 1623-1632.
[28] سجاد دهقان و محمدجواد فدائی اسلام، "بهبود تشخیص شئ برجسته با استفاده از ویژگی های چند مقیاسی در شبکه های عمیق"، چهارمین کنفرانس بین المللی محاسبات نرم، 1400.
[29] S. Gao et al., "Res2net: A New Multi-scale Backbone Architecture", IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 43, NO. 2, 2019, pp. 652-662.
[30] J.-J. Liu et al., “A Simple Pooling-based Design for Real-time Salient Object Detection”, IEEE Conference on Computer Vision and Pattern Recognition, 2019, pp. 3917-3926.
[31] T.-Y. Lin et al., “Feature Pyramid Networks for Object Detection”, IEEE Conference on Computer Vision and Pattern Recognition. 2017, pp. 2117-2125.
[32] K. He et al., “Deep Residual Learning for Image Recognition”, IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 770-778.
[33] L. Wang et al., “Learning to Detect Salient Objects with Image-level SuperVision”, IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 136-145.
[34] P. Arbelaez et al., "Contour Detection and Hierarchical Image Segmentation", IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 33, NO. 5, 2010, pp. 898-916.
[35] R. Mottaghi et al., “The Role of Context for Object Detection and Semantic Segmentation in the Wild”, IEEE Conference on Computer Vision and Pattern Recognition, 2014, pp. 891-898.
[36] G. Li et al., “Instance-level Salient Object Segmentation”, IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 2386-2395.  
[37] Q. Hou et al., “Deeply Supervised Salient Object Detection with Short Connections”, IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 3210-3212.
[38] T. Wang et al., “A Stagewise Refinement Model for Detecting Salient Objects in Images”, IEEE International Conference on Computer Vision, 2017, pp. 4019-4028.
[39] X. Zhan et al., “Progressive Attention Guided Recurrent Network for Salient Object Detection”, IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 714-722.
[40] T. Wang et al., “Detect Globally, Refine Locally: A Novel Approach to Saliency Detection”, IEEE Conference on Computer Vision and Pattern Recognition. 2018, pp. 3127-3135.
[41] Z. Wu, L. Su, and Q. Huang, “Cascaded Partial Decoder for Fast and Accurate Salient Object Detection”, IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 3907-3916.
[42] W. Wang et al., “An Iterative and Cooperative Top-down and Bottom-up Inference Network for Salient Object Detection”, IEEE Conference on Computer Vision and Pattern Recognition, 2019, pp. 5968-5977.
[43] J. Su et al., “Selectivity or Invariance: Boundary-aware Salient Object Detection”, IEEE International Conference on Computer Vision, 2019, pp. 3799-3808.
[44] Z. Deng et al., “R3net: Recurrent Residual Refinement Network for Saliency Detection”, International Joint Conference on Artificial Intelligence, 2018, pp. 684-690.
[45] V. Movahedi and J.H. Elder, “Design and Perceptual Validation of Performance Measures for Salient Object Segmentation”, IEEE Computer Society Conference on Computer Vision and Pattern Recognition-workshops, 2010, pp. 49-56.
[46] C. Yang et al., “Saliency Detection via Graph-based Manifold Ranking”, IEEE Conference on Computer Vision and Pattern Recognition, 2013, pp. 3166-3173.
[47] Q. Yan et al., “Hierarchical Saliency Detection”, IEEE Conference on Computer Vision and Pattern Recognition, 2013, pp. 1155-1162.
[48] Y. Li et al., “The Secrets of Salient Object Segmentation”, IEEE Conference on Computer Vision and Pattern Recognition, 2014, pp. 280-287.
[49] G. Li and Y. Yu. “Visual Saliency Based on Multiscale Deep Features”, IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp. 5455-5463.
Journal of Modeling in Engineering
Volume 20, Issue 71
November 2022
Pages 151-163

Files

History

  • Receive Date: 13 May 2022
  • Revise Date: 14 July 2022
  • Accept Date: 21 August 2022

Share

How to cite

Statistics