Embry-Riddle Utilizes AR for Flight Training

A team of Simulation Science, Games and Animation (SSGA) students got to work this summer in Prescott, Arizona, developing augmented-reality (AR) holograms that will eventually play a part in training the next generation of aviators at Embry-Riddle Aeronautical University.

The project was funded by an Undergraduate Research Institute (URI) grant, empowering Prescott Campus students to create virtual CRJ-700 models to be used in College of Aviation classrooms, using HoloLens 2 goggles.

“The holograms they have made will be a wonderful tool to start using AR for teaching, practicing of skills and, ultimately, testing,” said Dr. Michelle Hight, assistant professor of Aeronautical Science and Fixed Wing program chair.

The holograms developed by the SSGA team work like this: A student wearing the headset can step through “slides” inside interactive virtual training modules, giving them access to view schematics and information about the CRJ-700 aircraft in 3D. Another type of hologram model in development could also let them look around the flight deck, manipulating the buttons, switches and controls as they would in real life — the HoloLens 2 can even “see” their hands.

Some of the other holograms under development by the SSGA team depict the landing gear operation and the external fuel panel.

“We encouraged the team to consider the reasons for using augmented reality rather than virtual reality on the project,” associate professor and SSGA Program Chair Derek Fisher explained. “They determined that while virtual reality might provide a more immersive alternate experience, AR provides opportunities to focus on interacting with the computer-generated jet while collaborating in the familiar environment of the classroom, with real-life colleagues.”

The holograms created by the SSGA team for the AR environment have informational labels and can be rotated and resized by the user, giving students the opportunity to do a unique sort of virtual hands-on study, becoming familiar with where controls and important parts of the aircraft are located. Specifically, the module the SSGA students completed has a detailed schematic of the exhaust and radar danger zones. When it’s time for quizzing, the parts’ labels can be removed, and students will be asked to identify either location of parts or schematics to test their knowledge.

“This is the first step down a very good road for us as a university, potentially toward an entire class in AR,” Hight added. “Even if students had to attend class from home, this brings the jet to life. Every student who has a HoloLens 2 could join in 3D, standing or sitting in a virtual classroom, interacting with hologram learning tools rather than a flat screen. This is a timely pedagogical development for a world evolving and responding to a global pandemic.”

Over the summer, a team of five students worked on the AR prototype over a five-week period. Next, the team will improve the software so that multiple users can interact with the same jet simulation simultaneously. After testing the new prototype, the group will seek funding to expand the program to a full-class experience.

Some advanced forms of the software have even already been used in flight deck procedures training, allowing students to participate in virtual tutorials. The ongoing development of the technology, though, is providing students with valuable learning opportunities.

“Participating on this project has helped me decide that I want to do virtual or augmented reality simulations, either for military or police, as my career,” said Evan LaBate, a senior SSGA major. “I believe this technology can be used to help the military simulate live firefights, or even help with Air Force, in giving pilots updated heads-up displays.”