Using augmented reality to see under a patient's skin without the scalpel

University of Alberta scientists created an AR technology, that allows their new system called ProjectDR, to display medical images such as CT scans and MRI data to be displayed directly on a patient’s body in a way that moves as the patient does.

“We wanted to create a system that would show clinicians a patient’s internal anatomy within the context of the body,” explained Ian Watts, a computing science graduate student and the developer of ProjectDR.

ProjectDR's technology includes a motion-tracking system using infrared cameras, markers on the patient's body and a projector to display the images. Watts says that to be able to have the images track properly on patients’ bodies as they move, he built a custom software to get all of the components working simultaneously.

"There are lots of applications for this technology, including in teaching, physiotherapy, laparoscopic surgery and even surgical planning," said Watts, who developed the technology with fellow graduate student Michael Fiest.

ProjectorDR also can present segmented images – for example, only the lungs or only the blood vessels—depending on what a clinician is interested in seeing.

Now, Watts is working on refining ProjectDR to improve the system’s automatic calibration and to add components such as depth sensors. The next steps are testing the program’s viability in a clinical setting, explains Pierre Boulanger, a Department of Computing Science professor. "Soon, we'll deploy ProjectDR in an operating room in a surgical simulation laboratory to test the pros and cons in real-life surgical applications," he says.

The research team is “also doing pilot studies to test the usability of the system for teaching chiropractic and physical therapy procedures." adds Greg Kawchuk, a co-supervisor on the project from the Faculty of Rehabilitation Medicine. Once these pilot studies are complete, the team expects deployment of the system in real surgical pilot studies will quickly follow.