Detection and classification of asphalt pavement cracks using YOLOv5

Document Type : Civil Article

Authors

1 Department of Civil Engineering, Faculty of Engineering, Eqbal Lahori Institute of Higher Education

2 Department of civil engineering, Faculty of engineering. eqbal lahori institue of higher education

3 Department of computer engineering,Faculty of engineering,Eqbal lahori institute of higher education

Abstract

Automatic pavement crack detection is essential for assessing road maintenance and ensuring safe driving. Traditional crack detection has problems such as low efficiency and lack of complete detection. This study aims to solving the problems of traditional crack detection methods and using deep learning models. We proposed a method based on object detection algorithms for pavement crack detection and discussed the latest YOLOv5 series models for pavement crack detection while explaining the theoretical concepts. Finally, a crack detection model and effective pavement management is presented. The proposed model can determine the type, position and geometric characteristics of cracks accurately and at a higher speed in comparison with other methods. For this purpose, the images that had been taken from the asphalt of Mashhad roads were used to train and evaluate the model. Images were labeled for both linear crack and surface crack. Proposed model is developed using five YOLOv5 series algorithms and transfer learning and were evaluated for accuracy and speed of prediction. The models’ accuracy is between 77 to 98% and the prediction speed is between 17.4 to 105 milliseconds, which indicates the optimal performance of the models. The v5s model with 92.8% accuracy and a speed of 23.9 ms is selected as the final model for real prediction of cracks in one of the main thoroughfares of Mashhad. Based on the dimensions and the type of predicted crack and the use of the proposed decision tree, the maintenance approach for each part was determined.

Keywords

Main Subjects

References

[1] J. Tian, Y. Zhang, Y. Yao, X. Yao, C. Shi, and Y. Xu, "Road crack detection algorithm based on YOLOv3," in 2021 International Conference on Intelligent Transportation, Big Data & Smart City (ICITBS), 2021, pp. 39-42: IEEE.
[2] P. Rosa, "Automatic pavement crack detection and classification system," Transport. Res. Board, vol. 11,2012, pp. 57-65.
[3] H.-N. Nguyen, T.-Y. Kam, and P.-Y. Cheng, "An automatic approach for accurate edge detection of concrete crack utilizing 2D geometric features of crack," Journal of Signal Processing Systems, vol. 77, no. 3, 2014, pp. 221-240.
[4] Z. Yiyang, "The design of glass crack detection system based on image preprocessing technology," in 2014 IEEE 7th Joint International Information Technology and Artificial Intelligence Conference, 2014, pp. 39-42: IEEE.
[5] Y. Yi, J. Wang, W. Zhou, Y. Fang, J. Kong, and Y. Lu, "Joint graph optimization and projection learning for dimensionality reduction," Pattern Recognition, vol. 92, 2019, pp. 258-273.
[6] Y. Yi, Y. Chen, J. Wang, G. Lei, J. Dai, and H. Zhang, "Joint feature representation and classification via adaptive graph semi-supervised nonnegative matrix factorization," Signal Processing: Image Communication, vol. 89,2020, p. 115984.
[7] R. Rastgoo and K. Kiani, "Face recognition using fine-tuning of Deep Convolutional Neural Network and transfer learning," Journal of Modeling in Engineering, vol. 17, no. 58, 2019, pp. 103-111.
[8] M. Golshan, M. Teshnehlab, and A. Sharifi, "A Modified Brain Emotional Learning Model Inspired By Online Recurrent Memory Sequential Extreme Learning Machine Based On Neural Networks," Journal of Modeling in Engineering, 2022.
[9] E. Parsaeimehr, M. Fartash, and J. Akbari Torkestani, "An ensemble deep learning model to enhance feature representation for entity detection," Journal of Modeling in Engineering, vol. 20, no. 69, 2022.
[10] D. Ma, H. Fang, B. Xue, F. Wang, M. A. Msekh, and C. L. Chan, "Intelligent detection model based on a fully convolutional neural network for pavement cracks," Computer Modeling in Engineering & Sciences, vol. 123, no. 3, 2020, pp. 1267-1291.
[11] Y. Wang, K. Song, J. Liu, H. Dong, Y. Yan, and P. Jiang, "RENet: Rectangular convolution pyramid and edge enhancement network for salient object detection of pavement cracks," Measurement, vol. 170,2021, p. 108698.
[12] Y. Yi, J. Wang, W. Zhou, C. Zheng, J. Kong, and S. Qiao, "Non-negative matrix factorization with locality constrained adaptive graph," IEEE Transactions on circuits and systems for video technology, vol. 30, no. 2, 2019, pp. 427-441.
[13] Y. J. Cha, W. Choi, and O. Büyüköztürk, "Deep learning‐based crack damage detection using convolutional neural networks," Computer‐Aided Civil and Infrastructure Engineering, vol. 32, no. 5, 2017, pp. 361-378.
[14] F.-C. Chen and M. R. Jahanshahi, "NB-CNN: Deep learning-based crack detection using convolutional neural network and Naïve Bayes data fusion," IEEE Transactions on Industrial Electronics, vol. 65, no. 5, 2017, pp. 4392-4400.
[15] J. Long, E. Shelhamer, and T. Darrell, "Fully convolutional networks for semantic segmentation," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 3431-3440.
[16] X. Yang, H. Li, Y. Yu, X. Luo, T. Huang, and X. Yang, "Automatic pixel‐level crack detection and measurement using fully convolutional network," Computer‐Aided Civil and Infrastructure Engineering, vol. 33, no. 12, 2018, pp. 1090-1109.
[17] F. Yang, L. Zhang, S. Yu, D. Prokhorov, X. Mei, and H. Ling, "Feature pyramid and hierarchical boosting network for pavement crack detection," IEEE Transactions on Intelligent Transportation Systems, vol. 21, no. 4, 2019, pp. 1525-1535.
[18] H. Chen, H. Lin, and M. Yao, "Improving the efficiency of encoder-decoder architecture for pixel-level crack detection," IEEE Access, vol. 7, 2019, pp. 186657-186670.
[19] Z. Fan et al., "Ensemble of deep convolutional neural networks for automatic pavement crack detection and measurement," Coatings, vol. 10, no. 2, 2020, p. 152.
[20] X. Xiang, Y. Zhang, and A. El Saddik, "Pavement crack detection network based on pyramid structure and attention mechanism," IET Image Processing, vol. 14, no. 8, 2020, pp. 1580-1586.
[21] S. Zhou et al., "Automated detection and classification of spilled loads on freeways based on improved YOLO network," Machine Vision and Applications, vol. 32, no. 2, 2021, pp. 1-12.
[22] E. U. Rahman, Y. Zhang, S. Ahmad, H. I. Ahmad, and S. Jobaer, "Autonomous vision-based primary distribution systems porcelain insulators inspection using UAVs," Sensors, vol. 21, no. 3, 2021, p. 974.
[23] Q. Li, Q. Zou, D. Zhang, and Q. Mao, "FoSA: F* seed-growing approach for crack-line detection from pavement images," Image and Vision Computing, vol. 29, no. 12, 2011, pp. 861-872.
[24] Q. Li and X. Liu, "Novel approach to pavement image segmentation based on neighboring difference histogram method," in 2008 Congress on Image and Signal Processing, 2008, vol. 2, pp. 792-796: IEEE.
[25] F. Liu, G. Xu, Y. Yang, X. Niu, and Y. Pan, "Novel approach to pavement cracking automatic detection based on segment extending," in 2008 International Symposium on Knowledge Acquisition and Modeling, 2008, pp. 610-614: IEEE.
[26] M. Gavilán et al., "Adaptive road crack detection system by pavement classification," Sensors, vol. 11, no. 10, 2011, pp. 9628-9657.
[27] L. Q. M. Qingzhou, "Land-borne pavement rapid test and measurement," Journal of Transport Information and Safety, 2009.
[28] Y. Hu, C.-x. Zhao, and H.-n. Wang, "Automatic pavement crack detection using texture and shape descriptors," IETE Technical Review, vol. 27, no. 5, 2010, pp. 398-405.
[29] A. Cord and S. Chambon, "Automatic road defect detection by textural pattern recognition based on AdaBoost," Computer‐Aided Civil and Infrastructure Engineering, vol. 27, no. 4, 2012, pp. 244-259.
[30] Y. Shi, L. Cui, Z. Qi, F. Meng, and Z. Chen, "Automatic road crack detection using random structured forests," IEEE Transactions on Intelligent Transportation Systems, vol. 17, no. 12, 2016, pp. 3434-3445.
[31] N.-D. Hoang and Q.-L. Nguyen, "A novel method for asphalt pavement crack classification based on image processing and machine learning," Engineering with Computers, vol. 35, no. 2, 2019, pp. 487-498.
[32] H. Li, J. Zong, J. Nie, Z. Wu, and H. Han, "Pavement crack detection algorithm based on densely connected and deeply supervised network," IEEE Access, vol. 9, 2021, pp. 11835-11842.
[33] H. Maeda, Y. Sekimoto, T. Seto, T. Kashiyama, and H. Omata, "Road damage detection using deep neural networks with images captured through a smartphone," arXiv preprint arXiv:1801.09454, 2018.
[34] X. Wang and Z. Hu, "Grid-based pavement crack analysis using deep learning," in 2017 4th international conference on transportation information and safety (ICTIS), 2017, pp. 917-924: IEEE.
[35] B. Kim and S. Cho, "Automated vision-based detection of cracks on concrete surfaces using a deep learning technique," Sensors, vol. 18, no. 10, 2018, p. 3452.
[36] A. Zhang et al., "Automated pixel‐level pavement crack detection on 3D asphalt surfaces using a deep‐learning network," Computer‐Aided Civil and Infrastructure Engineering, vol. 32, no. 10, 2017, pp. 805-819.
[37] Y. Fei et al., "Pixel-level cracking detection on 3D asphalt pavement images through deep-learning-based CrackNet-V," IEEE Transactions on Intelligent Transportation Systems, vol. 21, no. 1, 2019, pp. 273-284.
[38] Q. Zou, Z. Zhang, Q. Li, X. Qi, Q. Wang, and S. Wang, "Deepcrack: Learning hierarchical convolutional features for crack detection," IEEE Transactions on Image Processing, vol. 28, no. 3, 2018, pp. 1498-1512.
[39] G. X. Hu, B. L. Hu, Z. Yang, L. Huang, and P. Li, "Pavement Crack Detection Method Based on Deep Learning Models," Wireless Communications and Mobile Computing, vol. 2021, 2021.
[40] Z.-Q. Zhao, P. Zheng, S.-t. Xu, and X. Wu, "Object detection with deep learning: A review," IEEE transactions on neural networks and learning systems, vol. 30, no. 11, 2019, pp. 3212-3232.
[41] Z. Zou, Z. Shi, Y. Guo, and J. Ye, "Object detection in 20 years: A survey," arXiv preprint arXiv:1905.05055, 2019.
[42] A. R. Pathak, M. Pandey, and S. Rautaray, "Application of deep learning for object detection," Procedia computer science, vol. 132, 2018, pp. 1706-1717.
[43] R. Girshick, J. Donahue, T. Darrell, and J. Malik, "Rich feature hierarchies for accurate object detection and semantic segmentation," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2014, pp. 580-587.
[44] J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, "You only look once: Unified, real-time object detection," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 779-788.
[45] R. Girshick, "Fast r-cnn," in Proceedings of the IEEE international conference on computer vision, 2015, pp. 1440-1448.
[46] S. Ren, K. He, R. Girshick, and J. Sun, "Faster r-cnn: Towards real-time object detection with region proposal networks," Advances in neural information processing systems, vol. 28, 2015.
[47] G. Jocher. (2020, 21 April ). Available: https://github.com/ultralytics/yolov5
[48] A. I. Magazine. (2020, 10 February). Guide to Yolov5 for Real-Time-Object Detection. Available: https://analyticsindiamag.com/yolov5/
[49] J. Yao, J. Qi, J. Zhang, H. Shao, J. Yang, and X. Li, "A real-time detection algorithm for Kiwifruit defects based on YOLOv5," Electronics, vol. 10, no. 14, 2021, p. 1711.
[50] J. Redmon and A. Farhadi, "Yolov3: An incremental improvement," arXiv preprint arXiv:1804.02767, 2018.
[51] A. Kuznetsova, T. Maleva, and V. Soloviev, "YOLOv5 versus YOLOv3 for apple detection," in Cyber-Physical Systems: Modelling and Intelligent Control: Springer, 2021, pp. 349-358.
[52] U. Nepal and H. Eslamiat, "Comparing YOLOv3, YOLOv4 and YOLOv5 for Autonomous Landing Spot Detection in Faulty UAVs," Sensors, vol. 22, no. 2, p. 464, 2022.
[53] Y. Fang, X. Guo, K. Chen, Z. Zhou, and Q. Ye, "Accurate and Automated Detection of Surface Knots on Sawn Timbers Using YOLO-V5 Model," BioResources, vol. 16, no. 3, 2021.
[54] F. Zhou, H. Zhao, and Z. Nie, "Safety helmet detection based on YOLOv5," in 2021 IEEE International Conference on Power Electronics, Computer Applications (ICPECA), 2021, pp. 6-11: IEEE.
[55] C.-Y. Wang, H.-Y. M. Liao, Y.-H. Wu, P.-Y. Chen, J.-W. Hsieh, and I.-H. Yeh, "CSPNet: A new backbone that can enhance learning capability of CNN," in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, 2020, pp. 390-391.
[56] T.-Y. Lin, P. Dollár, R. Girshick, K. He, B. Hariharan, and S. Belongie, "Feature pyramid networks for object detection," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 2117-2125.
[57] M. Tan, R. Pang, and Q. V. Le, "Efficientdet: Scalable and efficient object detection," in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 10781-10790.
[58] S. Liu, L. Qi, H. Qin, J. Shi, and J. Jia, "Path aggregation network for instance segmentation," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 8759-8768.
[59] B. Xu, N. Wang, T. Chen, and M. Li, "Empirical evaluation of rectified activations in convolutional network," arXiv preprint arXiv:1505.00853, 2015.
[60] J. Turian, J. Bergstra, and Y. Bengio, "Quadratic features and deep architectures for chunking," in Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Companion Volume: Short Papers, 2009, pp. 245-248.
[61] S. Ruder, "An overview of gradient descent optimization algorithms," arXiv preprint arXiv:1609.04747, 2016.
[62] D. P. Kingma and J. Ba, "Adam: A method for stochastic optimization," arXiv preprint arXiv:1412.6980, 2014.
[63] D. Thuan, "Evolution of yolo algorithm and yolov5: the state-of-the-art object detection algorithm," 2021.
[64] W. Wu et al., "Application of local fully Convolutional Neural Network combined with YOLO v5 algorithm in small target detection of remote sensing image," PloS one, vol. 16, no. 10, 2021, p. e0259283.
[65] S. Albelwi and A. Mahmood, "A framework for designing the architectures of deep convolutional neural networks," Entropy, vol. 19, no. 6,2017, p. 2427.
[66] D. Cires, U. Meier, J. Masci, L. Gambardella, and J. Schmidhuber, "High Performance Convolutional Neural Networks for Image Classification," in Proceedings of 22nd International Joint Conference on Artificial Intelligence, pp. 1237-1242.
[67] J. Snoek, H. Larochelle, and R. P. Adams, "Practical bayesian optimization of machine learning algorithms," Advances in neural information processing systems, vol. 25, 2012.
[68] L. Li, K. Jamieson, G. DeSalvo, A. Rostamizadeh, and A. Talwalkar, "Hyperband: A novel bandit-based approach to hyperparameter optimization," The Journal of Machine Learning Research, vol. 18, no. 1, 2017, pp. 6765-6816.
[69] K. Gopalakrishnan, S. K. Khaitan, A. Choudhary, and A. Agrawal, "Deep convolutional neural networks with transfer learning for computer vision-based data-driven pavement distress detection," Construction and building materials, vol. 157, 2017, pp. 322-330.
[70] K. Zhang, H. Cheng, and B. Zhang, "Unified approach to pavement crack and sealed crack detection using preclassification based on transfer learning," Journal of Computing in Civil Engineering, vol. 32, no. 2, 2018, p. 04018001.
[71] Y. Gao and K. M. Mosalam, "Deep transfer learning for image‐based structural damage recognition," Computer‐Aided Civil and Infrastructure Engineering, vol. 33, no. 9, 2018, pp. 748-768.
[72] K. Weiss, T. M. Khoshgoftaar, and D. Wang, "A survey of transfer learning," Journal of Big data, vol. 3, no. 1, 2016, pp. 1-40.
[73] M. Kasper-Eulaers, N. Hahn, S. Berger, T. Sebulonsen, Ø. Myrland, and P. E. Kummervold, "Detecting heavy goods vehicles in rest areas in winter conditions using YOLOv5," Algorithms, vol. 14, no. 4, 2021, p. 114.
[74] H. Zhang, M. Tian, G. Shao, J. Cheng, and J. Liu, "Target Detection of Forward-Looking Sonar Image Based on Improved YOLOv5," IEEE Access, vol. 10, 2022, pp. 18023-18034.
[75] Z. Chen et al., "Plant Disease Recognition Model Based on Improved YOLOv5," Agronomy, vol. 12, no. 2, 2022, p. 365.
[76] J. D. Kelleher, B. Mac Namee, and A. D'arcy, Fundamentals of machine learning for predictive data analytics: algorithms, worked examples, and case studies. MIT press, 2020.
[77] A. Burkov, The hundred-page machine learning book. Andriy Burkov Quebec City, QC, Canada, 2019.
[78] C. M. Bishop and N. M. Nasrabadi, Pattern recognition and machine learning (no. 4). Springer, 2006.
Journal of Modeling in Engineering
Volume 21, Issue 72
March 2023
Pages 175-193

Files

History

  • Receive Date: 17 June 2022
  • Revise Date: 01 August 2022
  • Accept Date: 18 September 2022

Share

How to cite

Statistics