|
Integration of deep learning and photogrammetry principles for the identification and geometric classification of potholes in a terrestrial image-based system
|
Mehdi Amirdehi   , Ali Hosseini naveh * |
|
|
|
Abstract: (55 Views) |
With the increase in traffic, road health has become more critical. Advances in artificial intelligence and robotic algorithms now enable more precise and effective monitoring of road conditions. This study aims to detect and geometrically classify potholes using a mobile phone. Since a mobile phone is equipped with a single camera and lacks a baseline length, the measurements were calculated relatively. The key innovation of this research lies in utilizing the YOLO neural network for pothole detection, along with determining their two-dimensional image coordinates, while the ORB SLAM 3 algorithm was employed to estimate the position and orientation of the camera. To determine the three-dimensional location of potholes, a video of the target route was recorded using the VIRec software installed on the mobile phone, capturing data acquisition timestamps at each moment. This video was then processed as input to the ORB SLAM 3 algorithm, which extracted the camera’s position and orientation for each frame, resulting in the creation of a projection matrix for every frame in the video. For pothole detection, the YOLO V8m neural network was used in conjunction with a tracking model that achieved 92% accuracy after being trained on datasets from urban streets across various countries. Once detected, the potholes were tracked across different frames by fitting ellipses to them, allowing their image coordinates to be obtained at different points in the video. Finally, using the camera’s position and orientation from the ORB SLAM 3 algorithm and the two-dimensional image coordinates of the potholes obtained from YOLO, triangulation and bundle adjustment were performed. This process enabled the relative estimation of the endpoints of the major and minor axes of each fitted ellipse. To evaluate the accuracy of the transformed points, the reprojection error parameter was used, indicating an error range of 1 to 3.5 pixels. For further quantitative analysis, relative areas were compared. Initially, the normalized area derived from the transformed points of each pothole was calculated and compared with its corresponding normalized area, obtained using the G1 Plus Sout multi-frequency receiver with an accuracy of 0.012 meters. The results showed a discrepancy of 2% to 5% between the computed pothole areas and the actual ground truth
|
Article number: 5 |
| Keywords: YOLO, Visual SLAM, Visual Odometry |
|
|
Full-Text [PDF 1345 kb]
(28 Downloads)
|
Type of Study: Research |
Subject:
Photo&RS
Received: 2024/12/1
|
|
|
|
|
|
|
| References |
1. P. Riya, K. Nakulraj, and A. Anusha, "Pothole Detection Methods," in 2018 3rd International Conference on Inventive Computation Technologies (ICICT), 2018: IEEE, pp. 120-123. [ DOI:10.1109/ICICT43934.2018.9034440] 2. Y.-M. Kim, Y.-G. Kim, S.-Y. Son, S.-Y. Lim, B.-Y. Choi, and D.-H. Choi, "Review of Recent Automated Pothole-Detection Methods," Applied Sciences, vol. 12, no. 11, 2022, doi: 10.3390/app12115320. [ DOI:10.3390/app12115320] 3. W. S. Qureshi et al., "An Exploration of Recent Intelligent Image Analysis Techniques for Visual Pavement Surface Condition Assessment," Sensors (Basel), vol. 22, no. 22, Nov 21 2022, doi: 10.3390/s22229019. [ DOI:10.3390/s22229019] 4. K. R and N. S, "Pothole and Object Detection for an Autonomous Vehicle Using YOLO," presented at the 2021 5th International Conference on Intelligent Computing and Control Systems (ICICCS), 2021. 5. S. Sen et al., "Pothole Detection System Using Object Detection through Dash Cam Video Feed," presented at the 2023 International Conference for Advancement in Technology (ICONAT), 2023. [ DOI:10.1109/ICONAT57137.2023.10080856] 6. M. Sathvik, G. Saranya, and S. Karpagaselvi, "An Intelligent Convolutional Neural Network based Potholes Detection using Yolo-V7," presented at the 2022 International Conference on Automation, Computing and Renewable Systems (ICACRS), 2022. [ DOI:10.1109/ICACRS55517.2022.10029263] 7. D. Chen, N. Chen, X. Zhang, and Y. Guan, "Real-Time Road Pothole Mapping Based on Vibration Analysis in Smart City," IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 15, pp. 6972-6984, 2022, doi: 10.1109/jstars.2022.3200147. [ DOI:10.1109/JSTARS.2022.3200147] 8. P. M. Harikrishnan and V. P. Gopi, "Vehicle Vibration Signal Processing for Road Surface Monitoring," IEEE Sensors Journal, vol. 17, no. 16, pp. 5192-5197, 2017, doi: 10.1109/jsen.2017.2719865. [ DOI:10.1109/JSEN.2017.2719865] 9. C. G. Harris and J. Pike, "3D positional integration from image sequences," Image and Vision Computing, vol. 6, no. 2, pp. 87-90, 1988. [ DOI:10.1016/0262-8856(88)90003-0] 10. H. C. Longuet-Higgins, "A computer algorithm for reconstructing a scene from two projections," Nature, vol. 293, no. 5828, pp. 133-135, 1981. [ DOI:10.1038/293133a0] 11. Aparna, Y. Bhatia, R. Rai, V. Gupta, N. Aggarwal, and A. Akula, "Convolutional neural networks based potholes detection using thermal imaging," Journal of King Saud University - Computer and Information Sciences, vol. 34, no. 3, pp. 578-588, 2022, doi: 10.1016/j.jksuci.2019.02.004. [ DOI:10.1016/j.jksuci.2019.02.004] 12. S. Gupta, P. Sharma, D. Sharma, V. Gupta, and N. Sambyal, "Detection and localization of potholes in thermal images using deep neural networks," Multimedia Tools and Applications, vol. 79, no. 35-36, pp. 26265-26284, 2020, doi: 10.1007/s11042-020-09293-8. [ DOI:10.1007/s11042-020-09293-8] 13. J. Terven, D.-M. Córdova-Esparza, and J.-A. Romero-González, "A comprehensive review of yolo architectures in computer vision: From yolov1 to yolov8 and yolo-nas," Machine Learning and Knowledge Extraction, vol. 5, no. 4, pp. 1680-1716, 2023. [ DOI:10.3390/make5040083] 14. M. Gao, X. Wang, S. Zhu, and P. Guan, "Detection and segmentation of cement concrete pavement pothole based on image processing technology," Mathematical Problems in Engineering, vol. 2020, no. 1, p. 1360832, 2020. [ DOI:10.1155/2020/1360832] 15. I. Scholl, T. Aach, T. M. Deserno, and T. Kuhlen, "Challenges of medical image processing," Computer science-Research and development, vol. 26, pp. 5-13, 2011. [ DOI:10.1007/s00450-010-0146-9] 16. Y. Bhatia, R. Rai, V. Gupta, N. Aggarwal, and A. Akula, "Convolutional neural networks based potholes detection using thermal imaging," Journal of King Saud University-Computer and Information Sciences, vol. 34, no. 3, pp. 578-588, 2022. [ DOI:10.1016/j.jksuci.2019.02.004] 17. S. Patra, A. I. Middya, and S. Roy, "PotSpot: Participatory sensing based monitoring system for pothole detection using deep learning," Multimedia Tools and Applications, vol. 80, no. 16, pp. 25171-25195, 2021. [ DOI:10.1007/s11042-021-10874-4] 18. R. Sathya and B. Saleena, "A framework for designing unsupervised pothole detection by integrating feature extraction using deep recurrent neural network," Wireless Personal Communications, vol. 126, no. 2, pp. 1241-1271, 2022. [ DOI:10.1007/s11277-022-09790-z] 19. R. Fan, U. Ozgunalp, Y. Wang, M. Liu, and I. Pitas, "Rethinking road surface 3-d reconstruction and pothole detection: From perspective transformation to disparity map segmentation," IEEE Transactions on Cybernetics, vol. 52, no. 7, pp. 5799-5808, 2021. [ DOI:10.1109/TCYB.2021.3060461] 20. P. Jiang, D. Ergu, F. Liu, Y. Cai, and B. Ma, "A Review of Yolo algorithm developments," Procedia computer science, vol. 199, pp. 1066-1073, 2022. [ DOI:10.1016/j.procs.2022.01.135] 21. C. Wan, Y. Pang, and S. Lan, "Overview of YOLO Object Detection Algorithm," International Journal of Computing and Information Technology, vol. 2, no. 1, pp. 11-11, 2022. [ DOI:10.56028/ijcit.1.2.11] 22. M. Nazarkevych and N. Oleksiv, "Оbject recognition system based on the Yolo model and database formation," Ukrainian Journal of Information Technology, vol. 6, pp. 120-126, 01/01 2024, doi: 10.23939/ujit2024.01.120. [ DOI:10.23939/ujit2024.01.120] 23. B. Karthika, M. Dharssinee, V. Reshma, R. Venkatesan, and R. Sujarani, "Object Detection Using YOLO-V8," in 2024 15th International Conference on Computing Communication and Networking Technologies (ICCCNT), 2024: IEEE, pp. 1-4. [ DOI:10.1109/ICCCNT61001.2024.10724411] 24. B. P, P. P, D. M, and V. S, "Unauthorized Drone Detection Using Deep Learning," International Journal for Research in Applied Science and Engineering Technology, vol. 12, no. 5, pp. 282-287, 2024, doi: 10.22214/ijraset.2024.61496. [ DOI:10.22214/ijraset.2024.61496] 25. F. Imanuel, S. K. Waruwu, A. Linardy, and A. M. Husein, "Literature review application of yolo algorithm for detection and tracking," Journal of Computer Networks, Architecture and High Performance Computing, vol. 6, no. 3, pp. 1378-1383, 2024. [ DOI:10.47709/cnahpc.v6i3.4374] 26. C. Altuntas, "Triangulation and time-of-flight based 3D digitisation techniques of cultural heritage structures," The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, vol. 43, pp. 825-830, 2021. [ DOI:10.5194/isprs-archives-XLIII-B2-2021-825-2021] 27. "pothoe dataset." https://roboflow.com/ (accessed. 28. W. Song, X. Xie, G. Li, and Z. Wang, "Flexible method to calibrate projector-camera systems with high accuracy," Electronics Letters, vol. 50, no. 23, pp. 1685-1687, 2014, doi: 10.1049/el.2014.1383. [ DOI:10.1049/el.2014.1383] 29. Z. Zhang, "A flexible new technique for camera calibration," IEEE Transactions on pattern analysis and machine intelligence, vol. 22, no. 11, pp. 1330-1334, 2000. [ DOI:10.1109/34.888718] |
|
| Send email to the article author |
|
|
|
| Add your comments about this article |
|
|
|
|
amirdehi M, hosseini naveh A. Integration of deep learning and photogrammetry principles for the identification and geometric classification of potholes in a terrestrial image-based system. JGST 2025; 14 (4) : 5 URL: http://jgst.issgeac.ir/article-1-1209-en.html
|
