Ryan Hill describes a simulation-based training prototype called the Client Assessment Practice Simulation (CAPS), being developed for research purposes at the University of Central Florida.

Ryan Hill, PhD candidate at the University of Central Florida, discusses the Institute for Simulation and Training’s prototype simulation development program and the benefits to students and the University.

At the University of Central Florida, partnerships between colleges within the school and the university's Institution for Simulation and Training lead to the introduction of innovative simulation-based training prototypes in fields that are just getting their feet wet in technology-driven simulation. The UCF College of Health and Public Affairs' School of Social Work was interested in simulation as an additional way to help prepare social work students for experiences in the field while still in their degree programs and after graduation. As a doctoral student and graduate researcher in the university's Modeling and Simulation program, I was approached to be a research project lead to work on simulation-based training for the School of Social Work. From a student's perspective, the structure of UCF's interdisciplinary Modeling and Simulation graduate program allows students to undertake such projects in healthcare, military, safety, and other domains.

The Motivation Behind Social Work Simulation

In an effort to stay abreast of the latest educational technology, the UCF School of Social Work has been looking into simulation. While still relatively light in literature compared to the field of medicine, the use of simulation in social work education is an area of great interest. The general feeling is that many of the gains in education and training by the medical field, especially when it comes to interacting with people to discover information and make accurate conclusions about aspects of someone's life, can also be made in the field of social work.

The majority of simulation literature in the field of social work involved the use of standardized clients to help train and evaluate social workers through a live simulation technique known as the Objective Structured Clinical Examination, OSCE for short. The word "live" means that an OSCE is done using real actors presenting a standardized case to train and evaluate learners on their social work field skills. Research in the use of OSCEs in social work shows that the technique holds promise to train and evaluate social workers (Logie et al. 2013), who are often a critical component in recognizing the need for care and also arranging the ability for a patient (referred to as "clients" by social workers) to obtain the care they need.

Live simulation requires multiple parties to be present for a significant length of time. The coordination of multiple parties, including educators, standardized clients, students and technology specialist if delivered on line, takes time and money. The standardized nature of the experience could even be harmed due to the exhaustion of the educators or standardized clients. These downsides provide pressure to seek out alternative ways to achieve the same or similar training outcomes that have fewer costs.

Using virtual humans and clever platforms to deliver training is one way to try to avoid some of the downsides of live simulation. Creating constructive conversational agents can potentially enable social work students to practice and be evaluated on their skills, avoiding cost and student and faculty disruption. Once the tools are created to generate virtual clients, creating new scenarios that represent high-severity, low-frequency cases can enable experiences that are difficult to arrange with both live simulation and actual field practice.

Using virtual clients to conquer some of the challenges of training social workers sounds like a pretty rosy solution. Just build a few pretend people and watch as your students become expert interviewers and investigators in their spare time using 3D environments and conversational agents that can be cheaply created and arranged for any training scenario. Unfortunately, as you have discerned, it isn't as easy as it sounds. Conversational agents are difficult to create cheaply or quickly, and using top-of-the-line technology might be more cost-prohibitive than using live actors.

That leaves us with questions that are worth answering. Can the fidelity of an interaction with a virtual client enable a strong enough improvement in skills to justify the investment? Can virtual client simulation-based training be strong enough to use to make up for deficiencies in field experience during a social work program? Can the use of virtual clients lead to a reliable and valid evaluation method for social work instructors? After all, a social work student can't experience everything during their educational program; when it comes to the effective use of social work skills, a single low-frequency, high-severity scenario can sometimes mean the difference between life and death for a client.

The Client Assessment Practice Simulation (CAPS)

At UCF, we're approaching these questions proactively in an attempt to make a positive impact in the world of social work education. The CAPS prototype will explore how effective simulation-based training is for skill development and evaluation purposes.

The learner's goal in CAPS is to develop a "biopsychosocial" profile for a virtual client, plan next steps and conduct a student self-reflection of his/her approach during the client's assessment. Since the ability to research on the fly and rephrase complex or confusing client statements were key skills subject matter experts identified, certain interactions will require the student to take additional action, i.e. referencing some basic documentation and putting in short answers for how they would rephrase something the virtual client said.

To accomplish those goals, the CAPS prototype utilizes a 3D environment with a single virtual client, a homeless veteran named Fred who acts as a conversational agent that interacts with the learner. The prototype will record all student interactions with the virtual client, visible to an instructor and also include a few game-like elements so that students can see the time and effort they put into using the prototype. Most importantly, CAPS uses an open entry style of input. This means that learners freely type in the questions or statements they think of for the virtual client. There is currently no multiple-choice option for CAPS, meaning students must think on their feet and listen actively to answers. This essential feature of CAPS is no small feat and required the development of an entire conversational model capable of handling input based on an extensive literature review.

After reviewing Dr. Benjamin Lok and Brent Rossen's literature on a technique called Human-centered Distributed Conversational Modeling (Rossen & Lok 2012), it was decided that developing that technique could be a good choice that fit nicely with some of the project's constraints and resources. This method of conversational modeling basically uses crowd sourcing to quickly enable model growth. All of the testers' idiosyncrasies and approaches to the conversations are recorded and potentially incorporated into the model, and instead of a few experts devoting a large amount of time on a model, one can potentially obtain a more robust model more quickly with much less effort by any particular individual. Dr. Lok's Virtual Experience Research Group developed a tool to enable this type of modeling, called the Virtual People Factory, and gave permission to use the tool.

The CAPS prototype is on target to be fully assembled during the Fall 2015 semester. While we will be unable to answer all of the important questions about the effectiveness of the use of virtual clients in simulation-based training for social work education, the CAPS team hopes to lay the foundation for future research in this area. It is our hope that the School of Social Work, the Institute for Simulation and Training, and perhaps other partners can continue researching and expand the ways that healthcare and social work education can be made more accessible, adaptable, and effective. UCF’s programs, especially their Modeling and Simulation program, are well-suited to promote this kind of multidisciplinary partnership.

Graduate Program Experience

One of the primary reasons that the Client Assessment Practice Simulation project exists is due to UCF’s Modeling and Simulation PhD program and its interdisciplinary nature. The Institute of Simulation and Training is made up of multi-disciplinary faculty and students. My interest in applying my modeling and simulation techniques from entertainment game design and production for a social purpose led me to IST and a degree in instructional design.

The great thing about the graduate program, is its multi-disciplinary nature. It was the perfect program for my interests, while at the same time it was a great program for those who wanted to study modeling and simulation for the purposes of operations research, or those who wanted to explore the electrical engineering or computer engineering aspects of creating simulators. Students in my cohorts specialized in usability, human factors engineering and human-computer interaction, which were strong components of my own program.

Research opportunities existed within IST, who actively and regularly partners with other colleges, the military, industry and other government agencies, that compliments the diverse and interdisciplinary nature of the graduate program.

About the Author

Ryan Hill is a graduate researcher at the university’s Institute for Simulation and Training and a doctoral student in the University of Central Florida’s Modeling and Simulation PhD program projected to graduate next near. He has earned a Master’s in the same subject as well as in Game Design, and has a keen interest in the design and evaluation of games and simulations that incorporate virtual humans to create novel learning experiences in a variety of fields.

REFERENCES

1. Rossen, B. and Lok, B. 2012. A crowdsourcing method to develop human conversational agents. International Journal of Human-Computer Studies. 70:4. 301-319.
2. Logie, C., Bogo, M., Regehr, C., and Regehr, G. 2013. A critical appraisal of the use of standardized client simulation in social work education. Journal of Social Work Education. Volume 49. 66-80.