The reality of research

Students use virtual reality to enhance the future of surveying engineering education in first-of-its-kind project

It’s a new technology.

It’s a collaboration among students in a variety of majors.

It provides research experience.

And it’s only happening at Penn State Wilkes-Barre, where students are using cutting-edge mapping technology and exploring the world of virtual reality to research and enhance surveying engineering education. The Penn State Wilkes-Barre project marks the first known implementation of immersive and interactive virtual reality (VR) in the surveying engineering curriculum in the United States.

This interdisciplinary research project provides the opportunity for participating students to have their research published. Working closely with faculty gives students an opportunity to build on knowledge learned in the classroom while gaining valuable and marketable real-world skills.

In 2019, Assistant Professor of Surveying Engineering Dimitrios Bolkas and Assistant Teaching Professor of Engineering Jeffrey Chiampi received funding from the Schreyer Institute for Teaching Excellence, campus Chancellor Endowments, and the office of Engineering Technology and Commonwealth Engineering. The funding was awarded to create a VR computer laboratory and conduct research on how to use immersive technologies to augment and strengthen surveying engineering education.

“Virtual reality and immersive technologies are continuously gaining ground in the surveying profession and in the AEC [architecture, engineering, and construction] industry,” said Bolkas. “The project has two main sub-projects. The first is to replicate surveying laboratories in immersive and interactive virtual reality, and the second is to create immersive 360° videos to improve laboratory instruction.”

Majors and talents come together

Several students worked on the project during the 2019-2020 academic year, including Jason and Luke Kepner, twin brothers majoring in telecommunications; David Neilson, who is majoring in surveying engineering; John Chapman and Vincent Pavill, who earned their degrees in surveying engineering in May 2020; Seth St Clair and Brandon Fino, who earned their degrees in information sciences and technology in May 2020; and Joe Fioti and Donovan Gaffney, who are majoring in computer science.

Chapman and Pavill used laser scanners and unmanned aerial systems to create realistic virtual environments of the Wilkes-Barre campus. Fioti and Gaffney used a video-game engine to program the functionality of the virtual surveying laboratories, while Fino and St Clair created models and refined the virtual environment.

The resulting VR surveying laboratories were first utilized during the Fall 2019 semester in a first-year surveying course. The laboratories are currently being improved with insights gained from this initial trial.

“In the VR surveying labs, students are able to complete surveying tasks completely virtually in a realistic, immersive and interactive environment, using and handling virtual instruments resembling their real-world activities,” Bolkas said. Currently, the VR surveying labs allow students to “walk” throughout the campus and conduct a basic leveling lab in virtual reality.

“The VR surveying labs are used with first-year engineering students to enhance their skills with equipment and help them connect theory and practice. By changing terrain and other lab parameters, we can create different surveying scenarios, and students can exercise their skills and critical thinking. The VR labs can also be useful when outdoor labs are cancelled due to rain and snow,” Bolkas explained.

A challenge in surveying instruction is that students need to learn how to operate sophisticated instruments and follow specific field procedures to achieve certain accuracy thresholds. “Students often have a very difficult time thinking spatially, so when we are explaining the labs before we go outside, they cannot really imagine what they are supposed to do,” Bolkas said.

To address this, the Kepners and Neilson worked on creating a series of immersive 360° videos on how to use surveying instruments and how to conduct basic surveying tasks such as leveling and traversing. The videos are now being used by the surveying program at Penn State Wilkes-Barre.

The immersive videos help students be better prepared for the surveying lab and also allow professors to instruct more efficiently. Students view the videos before conducting the labs and also have the videos available during the labs to answer some of their questions.

In the future, the team would like to include additional surveying tasks and environments that are sometimes not physically accessible due to traveling costs, safety or liability.

“Surveyors are integral to constructing highways, buildings, and tunnels. However, as instructors we cannot access such physical sites due to liability, and we cannot replicate these environments on campus. Utilizing virtual reality can help us expand students’ skills and experience in these applications,” Bolkas said.

The project has provided research and publication opportunities for students and faculty regarding the effectiveness of VR as a pedagogical tool in a program like surveying. So far, the ongoing research has been published in conference proceedings from the American Society for Engineering Education (ASEE) and the International Society for Photogrammetry and Remote Sensing (ISPRS); as a journal article with the International Journal of Image and Data Fusion; and as multiple workshops and presentations at conferences organized by the Pennsylvania Society of Land Surveyors (PSLS) and the Surveying and Geomatics Educators Society (SaGES).

Jason Kepner first got involved in the project after participating in the Multi-Campus Research Experience for Undergraduates (MC-REU) during summer 2019. “This is an exciting opportunity to advance my career,” he stated enthusiastically. “I think it will also show prospective and current students that anything is possible. It’s not that hard to get yourself out there and really build a name for yourself while you’re in college.”

Several of the participating students have found internships and jobs at leading companies locally and nationally due to their involvement in the project. Students who worked on the project over the summer received monetary compensation for their research, aiding them in financing their college career. The research also provides the opportunity for faculty to advance their career through promotion, tenure, or additional research funding.

Penn State Wilkes-Barre offers the only four-year accredited surveying engineering program in Pennsylvania, one of fewer than 30 such programs in the entire nation. The program is one of nine bachelor’s and four associate degree programs offered at the Wilkes-Barre campus.