UT Dallas Team Creating Virtual Reality Patient

The University of Texas at Dallas’ Center for Modeling and Simulation and Virtual Humans and Synthetic Societies Lab are developing an emotive “Virtual Reality Patient,”, in conjunction with Southwestern Medical Foundation, that medical students will be able to use to improve their patient communication skills – and the Virtual Reality Patient (VRP) research team just received a $200,000 grant from the foundation it will use to create a platform that will replicate medical interviews with the help of virtual patients and caregivers.

Dr. Marjorie Zielke, who leads the team and both UT facilities, says, “We hope to develop one of the first augmented or virtual reality-based conversational digital patients right here in our lab.” Zielke said the platform will offer high-quality simulations, known as emotive Virtual Reality Patients, which can exhibit medical symptoms to help medical students improve their verbal and nonverbal communication skills.

The virtual humans will complement other training methods, too, and ideally possess a lifelike ability to have both a conversation and convey emotion — something Zielke said is particularly important in the interview process, since patients express some things nonverbally.

First, Zielke’s team will compile data on body language, facial cues and other physiological information to develop a “natural language interface” capable of responsive and realistic communication. Virtual humans have always been a major focus for the center that “has long been interested in creating virtual robots that can either work in tandem, or in some cases, replace the need for medical mannequins often used in educational scenarios,” says Ziele. “The advantage of a training simulation is its potential to physically emulate what symptoms the patient is presenting.”

Given the lab’s past work on game-based medical simulations featuring stroke patients, Zielke said her team has a rich backlog of data regarding stroke-specific dialogue and symptoms they can use as their first case in this new project. “We’ve been working on this project for quite a while, and we would really like this to be a stake in the ground for developing world-class research on virtual patients in Texas.”