- CONVR 2023 - Proceedings of the 23rd International Conference on Construction Applications of Virtual Reality
- Edited by Pietro Capone, Vito Getuli, Farzad Pour Rahimian, Nashwan Dawood, Alessandro Bruttini, Tommaso Sorbi
Utilizing 360-Degree Images for Synthetic Data Generation in Construction Scenarios
- Aqsa Sabir
- Rahat Hussain
- Syed Farhan Alam Zaidi
- Akeem Pedro
- Mehrtash Soltani
- Dongmin Lee
- Chansik Park
- © 2023 Author(s) |
- CC BY-NC 4.0
- DOI: 10.36253/979-12-215-0289-3.70
Computer vision-based safety monitoring requires machine learning models trained on generalized datasets covering various viewpoints, surface properties, and lighting conditions. However, capturing high-quality and extensive datasets for some construction scenarios is challenging at real job sites due to the risky nature of construction scenarios. Previous methods have proposed synthetic data generation techniques involving 2D background randomization with virtual objects in game-based engines. While there has been extensive work on utilizing 360-degree images for various purposes, no study has yet employed 360-degree images for generating synthetic data specifically tailored for construction sites. To improve the synthetic data generation process, this study proposes a 360-degree images-based synthetic data generation approach using Unity 3D game engine. The approach efficiently generates a sizable dataset with better dimensions and scaling, encompassing a range of camera positions with randomized lighting intensities. To check the effectiveness of our proposed method, we conducted a subjective evaluation, considering three key factors: object positioning, scaling in terms of object respective size, and the overall size of the generated dataset. The synthesized images illustrate the visual improvement in all three factors. By offering an improved data generation method for training safety-focused computer vision models, this research has the potential to significantly enhance the automation of the construction safety monitoring process, and hence, this method can bring substantial benefits to the construction industry by improving operational efficiency and reinforcing safety measures for workers
- Keywords:
- 360-Degree Images,
- Computer Vision,
- Synthetic Data Generation,
- Game Engine,
- Object Detection,
- Construction Safety Monitoring,
Chung Ang University, Korea (the Republic of) - ORCID: 0009-0006-5459-909X
Chung Ang University, Korea (the Republic of) - ORCID: 0000-0002-6909-5189
Chung Ang University, Korea (the Republic of) - ORCID: 0000-0003-2257-290X
Chung Ang University, Korea (the Republic of) - ORCID: 0000-0002-7884-5316
Chung Ang University, Korea (the Republic of) - ORCID: 0000-0002-5217-2010
Chung Ang University, Korea (the Republic of) - ORCID: 0000-0002-3176-5327
Chung Ang University, Korea (the Republic of) - ORCID: 0000-0003-2256-300X
- Assadzadeh, A., Arashpour, M., Brilakis, I., Ngo, T., & Konstantinou, E. (2022). Vision-based excavator pose estimation using synthetically generated datasets with domain randomization. Automation in Construction, 134, 104089. DOI: 10.1016/j.autcon.2021.104089
- Barrera-Animas, A. Y., & Davila Delgado, J. M. (2023). Generating real-world-like labelled synthetic datasets for construction site applications. Automation in Construction, 151, 104850. DOI: 10.1016/j.autcon.2023.104850
- Borkman, S., Crespi, A., Dhakad, S., Ganguly, S., Hogins, J., Jhang, Y.-C., Kamalzadeh, M., Li, B., Leal, S., Parisi, P., Romero, C., Smith, W., Thaman, A., Warren, S., & Yadav, N. (2021). Unity Perception: Generate Synthetic Data for Computer Vision. DOI: 10.48550/arXiv.2107.04259
- Choi, W., & Pyun, S. (2021). Synthetic Training Data Generation for Fault Detection Based on Deep Learning. Geophysics and Geophysical Exploration, 24(3), 89–97. DOI: 10.7582/GGE.2021.24.3.089
- Frolov, V., Faizov, B., Shakhuro, V., Sanzharov, V., Konushin, A., Galaktionov, V., & Voloboy, A. (2022). Image Synthesis Pipeline for CNN-Based Sensing Systems. Sensors, 22(6), 2080. DOI: 10.3390/s22062080
- Kim, A., Lee, K., Lee, S., Song, J., Kwon, S., & Chung, S. (2022). Synthetic Data and Computer-Vision-Based Automated Quality Inspection System for Reused Scaffolding. Applied Sciences, 12(19), 10097. DOI: 10.3390/app121910097
- Kim, J., Kim, D., Shah, J., & Lee, S. (2022). Training a Visual Scene Understanding Model Only with Synthetic Construction Images. Computing in Civil Engineering 2021, 221–229. DOI: 10.1061/9780784483893.028
- Lee, J. G., Asce, S. M., Hwang, J., Chi, S., Asce, M., & Seo, J. (2022). Synthetic Image Dataset Development for Vision-Based Construction Equipment Detection. Computing in Civil Engineering, 36(5). DOI: 10.1061/(ASCE)CP.1943-5487.0001035
- Li, Y., Wei, H., Han, Z., Jiang, N., Wang, W., & Huang, J. (2022). Computer Vision-Based Hazard Identification of Construction Site Using Visual Relationship Detection and Ontology. Buildings, 12(6), 857.
- Neuhausen, M., Herbers, P., & König, M. (2020). Using synthetic data to improve and evaluate the tracking performance of construction workers on site. Applied Sciences (Switzerland), 10(14), 4948. DOI: 10.3390/app10144948
- Oh, X., Loh, L., Foong, S., Koh, Z. B. A., Ng, K. L., Tan, P. K., Toh, P. L. P., & Tan, U. X. (2021). Initialisation of Autonomous Aircraft Visual Inspection Systems via CNN-Based Camera Pose Estimation. Proceedings - IEEE International Conference on Robotics and Automation, 2021-May, 11047–11053. DOI: 10.1109/ICRA48506.2021.9561575
- Rho, J., Park, M., & Lee, H.-S. (2020). Automated construction progress management using computer vision-based CNN model and BIM. Korean Journal of Construction Engineering and Management, 21(5), 11–19.
- Sami Ur Rehman, M., Shafiq, M. T., & Ullah, F. (2022). Automated Computer Vision-Based Construction Progress Monitoring: A Systematic Review. Buildings, 12(7), 1037.
- Siu, C., Wang, M., & Cheng, J. C. P. (2022). A framework for synthetic image generation and augmentation for improving automatic sewer pipe defect detection. Automation in Construction, 137, 104213. DOI: 10.1016/j.autcon.2022.104213
- Soltani, M. M., Zhu, Z., & Hammad, A. (2016). Automated annotation for visual recognition of construction resources using synthetic images. Automation in Construction, 62, 14–23. DOI: 10.1016/j.autcon.2015.10.002
- Sutjaritvorakul, T., Vierling, A., & Berns, K. (2020). Data-driven worker detection from load-view crane camera. Proceedings of the 37th International Symposium on Automation and Robotics in Construction, ISARC 2020: From Demonstration to Practical Use - To New Stage of Construction Robot, 864–871. DOI: 10.22260/isarc2020/0119
- Tohidifar, A., Fatemeh Saffari, S., & Kim, D. (2022). Fake it till you make it: Training Deep Neural Networks for Worker Detection using Synthetic Data. The 29th EG-ICE International Workshop on Intelligent Computing in Engineering, 386–396. DOI: 10.7146/AUL.455.C229
- Wei, Y., & Akinci, B. (2022). Synthetic Image Data Generation for Semantic Understanding in Everchanging Scenes Using BIM and Unreal Engine. Computing in Civil Engineering 2021, 934–941. DOI: 10.1061/9780784483893.115
- Wong, M. Z., Kunii, K., Baylis, M., Ong, W. H., Kroupa, P., & Koller, S. (2019). Synthetic dataset generation for object-to-model deep learning in industrial applications. PeerJ Computer Science, 2019(10), e222. DOI: 10.7717/peerj-cs.222
- Zhang, J., Fukuda, T., & Yabuki, N. (2022). Automatic generation of synthetic datasets from a city digital twin for use in the instance segmentation of building facades. Journal of Computational Design and Engineering, 9(5), 1737–1755. DOI: 10.1093/jcde/qwac086
Chapter Information
Chapter Title
Utilizing 360-Degree Images for Synthetic Data Generation in Construction Scenarios
Authors
Aqsa Sabir, Rahat Hussain, Syed Farhan Alam Zaidi, Akeem Pedro, Mehrtash Soltani, Dongmin Lee, Chansik Park
DOI
10.36253/979-12-215-0289-3.70
Peer Reviewed
Publication Year
2023
Copyright Information
© 2023 Author(s)
Content License
Metadata License
Bibliographic Information
Book Title
CONVR 2023 - Proceedings of the 23rd International Conference on Construction Applications of Virtual Reality
Book Subtitle
Managing the Digital Transformation of Construction Industry
Editors
Pietro Capone, Vito Getuli, Farzad Pour Rahimian, Nashwan Dawood, Alessandro Bruttini, Tommaso Sorbi
Peer Reviewed
Publication Year
2023
Copyright Information
© 2023 Author(s)
Content License
Metadata License
Publisher Name
Firenze University Press
DOI
10.36253/979-12-215-0289-3
eISBN (pdf)
979-12-215-0289-3
eISBN (xml)
979-12-215-0257-2
Series Title
Proceedings e report
Series ISSN
2704-601X
Series E-ISSN
2704-5846