Contained in:
Book Chapter

Improving Sense-Making for Construction Planning Tasks Using Visual and Haptic Stimuli in Virtual Reality Environments

  • Ivan Mutis
  • Marina Oberemok
  • Nishanth Purushotham

Design documents, drawings, and specifications are visual representations that are fundamental and prevalent in today’s construction engineering practice. Construction specialties (e.g., structural, mechanical) rely on these visual representations to express and draw meaning during collaborations. Construction engineering and management (CEM) students must acquire the knowledge, skills, and abilities — a key example of which is perceptual competence —for interpreting visual representations to facilitate efficient task execution, such as planning. Empowering learners with new technology using robust real-world immersion and interactive features is a significant step towards this target. The presented research explores new human-machine interactions to determine the best way for CEM students to learn through the combined senses of sight and touch. The approach merges visual and haptic interactions within an immersive environment to enhance perception and reasoning skills. The research demonstrates how CEM learners interact with and interpret the meanings of information within a planning task. It explores how VR and haptic technology augment the ability to recognize meanings — a new type of representational competency — for improved interpretation of information related to components with respect to engineering disciplines and sub-systems in a CEM, and investigates learners’ problem-solving ability by using perception-rich enhanced virtual reality (VR) and haptic affordances

  • Keywords:
  • haptic cues,
  • human-computer-interaction,
  • design interpretations,
+ Show More

Ivan Mutis

Illinois Institute of Technology, United States - ORCID: 0000-0003-2707-2701

Marina Oberemok

Illinois Institute of Technology, United States - ORCID: 0000-0003-0695-6305

Nishanth Purushotham

Illinois Institute of Technology, United States - ORCID: 0000-0002-8962-1596

  1. Adami, P., Rodrigues, P. B., Woods, P. J., Becerik-Gerber, B., Soibelman, L., Copur-Gencturk, Y., & Lucas, G. (2022). Impact of VR-Based Training on Human–Robot Interaction for Remote Operating Construction Robots. Journal of Computing in Civil Engineering, 36(3), 04022006. DOI: 10.1061/(ASCE)CP.1943-5487.0001016
  2. Adilkhanov, A., Rubagotti, M., & Kappassov, Z. (2022). Haptic Devices: Wearability-Based Taxonomy and Literature Review. IEEE Access, 10, 91923-91947. DOI: 10.1109/access.2022.3202986
  3. Alakhawand, N., Frier, W., & Lepora, N. F. (2022). Mapping Mid-Air Haptics With a Low-Cost Tactile Robot. IEEE Robotics and Automation Letters, 7, 7873-7880.
  4. Antonenko, P., & Mutis, I. (2017). Using Unmanned Aerial Systems to Bring STEM Field Experiences to the Classroom National Association for Research in Science Teaching (NARST), San Antonio, TX. http://www.narst.org/annualconference/2017conference.cfm
  5. Antonenko, P. D., & Mutis, I. (2017). Empowering learning through remote visualizations using unmanned aerial systems: Perspectives of education and industry experts 2017 American Educational Research Association, San Antonio, TX. http://www.aera.net/Publications
  6. Bluteau, J., Coquillart, S., Payan, Y., & Gentaz, E. (2008). Haptic Guidance Improves the Visuo-Manual Tracking of Trajectories. PLOS ONE, 3(3), e1775. DOI: 10.1371/journal.pone.0001775
  7. Christiand, & Yoon, J. (2011). Assembly simulations in virtual environments with optimized haptic path and sequence. Robotics and Computer-Integrated Manufacturing, 27(2), 306-317. DOI: 10.1016/j.rcim.2010.07.015
  8. Chryssa, T., & Julie-Ann, S. (2020). From Video-Conferencing to Holoportation and Haptics: How Emerging Technologies Can Enhance Presence in Online Education? DOI: 10.1007/978-981-15-0618-5_16
  9. Coffey, M., & Pierson, A. (2022, 2022). Collaborative Teleoperation with Haptic Feedback for Collision-Free Navigation of Ground Robots.
  10. Cooper, N., Milella, F., Pinto, C., Cant, I., White, M., & Meyer, G. (2018). The effects of substitute multisensory feedback on task performance and the sense of presence in a virtual reality environment. PLOS ONE, 13(2), e0191846. DOI: 10.1371/journal.pone.0191846
  11. Enriquez, M., Maclean, K., & Chita, C. (2006, 2006). Haptic phonemes Basic - Building Blocks of Haptic Communication.
  12. Enriquez, M. J., & MacLean, K. E. (2003, 22-23 March 2003). The hapticon editor: a tool in support of haptic communication research. 11th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, 2003. HAPTICS 2003. Proceedings.,
  13. Feygin, D., Keehner, M., & Tendick, R. (2002). Haptic guidance: experimental evaluation of a haptic training method for a perceptual motor skill.
  14. Fulkerson, M. (2020). Touch (E. N. Zalta, Ed. Summer 2020 ed.). Metaphysics Research Lab, Stanford University. https://plato.stanford.edu/archives/sum2020/entries/touch/
  15. Hatzfeld, C., Kern, T. A., Opitz, T., Neupert, C., Kassner, S., Matysek, M., Rausch, J., Meckel, O., Rettig, A., Sindlinger, S., & Haus, H. (2015). Engineering Haptic Devices - Table of Contents and Sample Pages.
  16. Hu, W., & Zhang, X. (2012, 2012). A Rapid Development Method of Virtual Assembly Experiments Based on 3D Game Engine.
  17. Huang, K., Chitrakar, D., Rydén, F., & Chizeck, H. J. (2019). Evaluation of haptic guidance virtual fixtures and 3D visualization methods in telemanipulation—a user study. Intelligent Service Robotics, 12(4), 289-301. DOI: 10.1007/s11370-019-00283-w
  18. James, J., R, B., & Neamtu, G. (2019). Design of a bi-manual haptic interface for skill acquisition in surface mount device soldering. Soldering & Surface Mount Technology, 31. DOI: 10.1108/SSMT-01-2018-0001
  19. Jong, T. d. (2014). Emerging Representation Technologies for Problem Solving. In J. M. Spector, M. D. Merrill, J. Elen, & M. J. Bishop (Eds.), Handbook of Research on Educational Communications and Technology (pp. 809-816). Springer New York. DOI: 10.1007/978-1-4614-3185-5_65
  20. Jose, J., Unnikrishnan, R., Marshall, D., & Bhavani, R. R. (2016, 2016). Haptics enhanced multi-tool virtual interfaces for training carpentry skills.
  21. Kortum, P. (2008). HCI beyond the GUI: Design for haptic, speech, olfactory, and other nontraditional interfaces. Elsevier.
  22. Kreimeier, J., Hammer, S., Friedmann, D., Karg, P., Bühner, C., Bankel, L., & Götzelmann, T. (2019, 2019). Evaluation of different types of haptic feedback influencing the task-based presence and performance in virtual reality.
  23. Li, Z., Wang, J., Anwar, M. S., & Zheng, Z. (2020). An efficient method for generating assembly precedence constraints on 3D models based on a block sequence structure. Computer-Aided Design, 118, 102773. DOI: 10.1016/j.cad.2019.102773
  24. Liu, X., Dodds, G., Hinds, B. K., & McCartney, J. (2003). Virtual DesignWorks: Designing 3D CAD Models Via Touch Interaction. ASME 2003 International Mechanical Engineering Congress and Exposition,
  25. Luo, J., Yang, C., Su, H., & Liu, C. (2019). A Robot Learning Method with Physiological Interface for Teleoperation Systems. Applied Sciences, 9(10), 2099. DOI: 10.3390/app9102099
  26. Manchanda, N., Jha, S., & Mukherjee, S.-h. (2017). Overview: Human-Computer Interaction an Globally Uses Technique in Society.
  27. Mastrolembo Ventura, S., Castronovo, F., Nikolić, D., & Ciribini, A. L. C. (2022). Implementation of virtual reality in construction education: a content-analysis based literature review. Journal of Information Technology in Construction, 27, 705-731. DOI: 10.36680/j.itcon.2022.035
  28. Medellín-Castillo, H., Gonzalez-Badillo, G., Govea, E., Espinosa Castañeda, R., & Gallegos-Nieto, E. (2015). Development of Haptic-Enabled Virtual Reality Applications for Engineering, Medicine and Art. DOI: 10.1115/IMECE2015-52770
  29. Mugge, W., Kuling, I. A., Brenner, E., & Smeets, J. B. J. (2016). Haptic Guidance Needs to Be Intuitive Not Just Informative to Improve Human Motor Accuracy. PLOS ONE, 11(3), e0150912. DOI: 10.1371/journal.pone.0150912
  30. Müller, T. (2020). Designing with Haptic Feedback Umeå University]. http://umu.diva-portal.org/smash/get/diva2:1445032/FULLTEXT01.pdf
  31. Mutis, I. (2014). Enhancing spatial and temporal cognitive ability in construction education through augmented reality and artificial visualizations. The International Conference for Computing in Civil and Building Engineering (Icccbe) and CIB-W78, 2014, Orlando Fl.
  32. Mutis, I. (2015). Enhancing spatial and temporal cognitive ability in construction education through the effect of artificial visualizations. 2015 ASCE International Workshop on Computing in Civil Engineering, Austin, Tx.
  33. Mutis, I. (2018a). Spatial-temporal cognitive ability: Coupling representations to situations and contexts for coordinating activities in the construction project environment. In Transforming engineering education: Innovative computer-mediated learning technologies (pp. 5-24).
  34. Mutis, I. (2018b). Spatial-Temporal Cognitive Ability: coupling representations to situations and contexts for coordinating activities in the construction project environment. In I. Mutis, R. Fruchter, & C. Menassa (Eds.), Transforming Engineering Education through Innovative Computer Mediated Learning Technologies (pp. 14 ). ASCE, American Society of Civil Engineers.
  35. NSF, N. S. F. (2020). STEM Education for the Future: a visioning report. National Science Foundation.
  36. OED. (2020). Oxford English dictionary. https://www.google.com/search?q=haptics+definition&rlz=1C5CHFA_enUS1021US1021&oq=haptics+&aqs=chrome.0.35i39j69i57j35i39j0i131i433i512j0i512j69i60l3.5443j1j4&sourceid=chrome&ie=UTF-8
  37. Pavlik, R. A., Vance, J. M., & Luecke, G. R. (2013). Interacting With a Large Virtual Environment by Combining a Ground-Based Haptic Device and a Mobile Robot Base.
  38. Prabhakaran, A., Mahamadu, A.-M., & Mahdjoubi, L. (2022). Understanding the challenges of immersive technology use in the architecture and construction industry: A systematic review. Automation in Construction, 137, 104228. DOI: 10.1016/j.autcon.2022.104228
  39. Rahimian, F. P., & Ibrahim, R. (2011). Impacts of VR 3D sketching on novice designers’ spatial cognition in collaborative conceptual architectural design. Design Studies, 32, 255-291.
  40. Ranjith, R., Akshay, N., Unnikrishnan, R., & Bhavani, R. R. (2014, 2014). Do It Yourself Educational Kits for Vocational Education and Training.
  41. Sanfilippo, F., Blazauskas, T., Salvietti, G., Ramos, I., Vert, S., Radianti, J., Majchrzak, T. A., & Oliveira, D. (2022). A Perspective Review on Integrating VR/AR with Haptics into STEM Education for Multi-Sensory Learning. Robotics, 11(2), 41. DOI: 10.3390/robotics11020041
  42. Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12(2), 257-285. DOI: 10.1016/0364-0213(88)90023-7
  43. Sweller, J., van Merriënboer, J. J. G., & Paas, F. (2019). Cognitive Architecture and Instructional Design: 20 Years Later. Educational Psychology Review, 31(2), 261-292. DOI: 10.1007/s10648-019-09465-5
  44. Takahashi, C., Diedrichsen, J., & Watt, S. J. (2009). Integration of vision and haptics during tool use. Journal of Vision, 9(6), 3-3. DOI: 10.1167/9.6.3
  45. Teranishi, A., Korres, G., Park, W., & Eid, M. A. (2018). Combining Full and Partial Haptic Guidance Improves Handwriting Skills Development. IEEE Transactions on Haptics, 11, 509-517.
  46. Tran, C., Smith, B., & Buschkuehl, M. (2017). Support of mathematical thinking through embodied cognition: Nondigital and digital approaches. Cognitive Research: Principles and Implications, 2(1). DOI: 10.1186/s41235-017-0053-8
  47. Tytler, R. (2020). STEM Education for the Twenty-First Century. In (pp. 21-43). DOI: 10.1007/978-3-030-52229-2_3
  48. Williams Ii, R. L., Chen, M.-Y., & Seaton, J. M. (2001). Haptics-Augmented High School Physics Tutorials. International Journal of Virtual Reality, 5(1), 167-184. DOI: 10.20870/ijvr.2001.5.1.2678
  49. Williams, N. L., Li, J., & Lin, M. C. (2023). A Framework for Active Haptic Guidance Using Robotic Haptic Proxies. arXiv preprint arXiv:2301.05311.
  50. Yeh, S.-C., Hwang, W.-Y., Wang, J.-L., & Zhan, S.-Y. (2013). Study of co-located and distant collaboration with symbolic support via a haptics-enhanced virtual reality task. Interactive Learning Environments, 21, 184 - 198.
  51. Yuan, M. L., Ong, S. K., & Nee, A. Y. C. (2008). Augmented reality for assembly guidance using a virtual interactive tool. International Journal of Production Research, 46, 1745 - 1767.
PDF
  • Publication Year: 2023
  • Pages: 142-154

XML
  • Publication Year: 2023

Chapter Information

Chapter Title

Improving Sense-Making for Construction Planning Tasks Using Visual and Haptic Stimuli in Virtual Reality Environments

Authors

Ivan Mutis, Marina Oberemok, Nishanth Purushotham

DOI

10.36253/979-12-215-0289-3.14

Peer Reviewed

Publication Year

2023

Copyright Information

© 2023 Author(s)

Content License

CC BY-NC 4.0

Metadata License

CC0 1.0

Bibliographic Information

Book Title

CONVR 2023 - Proceedings of the 23rd International Conference on Construction Applications of Virtual Reality

Book Subtitle

Managing the Digital Transformation of Construction Industry

Editors

Pietro Capone, Vito Getuli, Farzad Pour Rahimian, Nashwan Dawood, Alessandro Bruttini, Tommaso Sorbi

Peer Reviewed

Publication Year

2023

Copyright Information

© 2023 Author(s)

Content License

CC BY-NC 4.0

Metadata License

CC0 1.0

Publisher Name

Firenze University Press

DOI

10.36253/979-12-215-0289-3

eISBN (pdf)

979-12-215-0289-3

eISBN (xml)

979-12-215-0257-2

Series Title

Proceedings e report

Series ISSN

2704-601X

Series E-ISSN

2704-5846

117

Fulltext
downloads

112

Views

Export Citation

1,346

Open Access Books

in the Catalogue

2,262

Book Chapters

3,790,127

Fulltext
downloads

4,420

Authors

from 923 Research Institutions

of 65 Nations

65

scientific boards

from 348 Research Institutions

of 43 Nations

1,248

Referees

from 381 Research Institutions

of 38 Nations