Debriefing Trends and Challenges

HTE correspondent Marty Kauchak highlights the efforts of the simulation and training industry, and healthcare learning audiences to more effectively integrate debriefing into training scenarios and other instruction.

Since the mid-1990s, this author has watched the military and civil aviation sectors refine their efforts to record and evaluate efforts by crews and individuals to complete scenarios in a variety of training settings. The investments by these high-risk operational sectors to obtain high-fidelity after action reviews (the military term) and debriefings (the preferred label in the aviation community) of training activities, are being matched by the quickening pace of activity in the healthcare sector. Healthcare learners, their instructors and other stakeholders, are using debriefings (the term also used by healthcare associations and practitioners) to enhance learning and, more significant, support quality programs.

Industry Survey

In one project, Mursion is partnering with Duke University Medical Center to test the effectiveness of virtual simulated patient training (VSP). Brentt Brown, the company’s director of Sales and Marketing, told the author this July 13, “A control group is receiving traditional, standardized patient training on how to deliver negative, diagnostic news to family members of a patient. The treatment group undergoes the same exercise within our virtual simulator.”

Debriefing is an essential part of the virtual training experience. In the current scenario, the doctor delivers the negative diagnostic news to the family that their patient/loved one’s wrong kidney was removed.

The control group’s debriefing process consists of the standardized patient being provided a rubric on how the healthcare provider-family interaction should progress. “In the virtual simulation with the simulated patient our avatar which is controlled by a real live actor, gives feedback to the learner,” the industry expert said and added, “In both the control group and treatment group, debriefing is done almost, identically the same way. For the study we have the same live actor playing the traditional, standardized patient and playing the virtual simulated patient.”

Brown noted the healthcare sector’s desire for more rigorous debriefing from existing virtual simulation platforms. “They are looking for the capacity to receive immediate feedback from the avatar that includes information on the quality of the interpersonal interaction. Many also want the simulation to be recorded, tagged and played back by a faculty member. There are two levels of feedback, then: one is by the live actor to give feedback on how the learner did, and the opportunity for faculty to extend the conversation in a deeper way either through written feedback, a one-on-one coaching session or an after action review with other students.”

Mursion is bringing to bear its technology competencies in adjacent sectors for its current and prospective healthcare business clients. “Our simulations are very easily recorded, where you get a split screen of the learner on one side, and the avatar on the other. The whole interaction can then be played back in snippets for debriefing,” Brown pointed out.

And while company also places an importance on the role of the live avatar to give immediate feedback, it also allows faculty, the hospital trainer or another individual the opportunity to record and conduct debriefing either on a one-on-one setting or a group setting with those who participated in the simulation.

According to Helen Woldeab, a marketing strategist at B-Line Medical, “Our customers’ requirements are disarmingly simple: They want to record simulations and clinical events seamlessly, with minimal workflow disruption and maximum impact. The recordings must be able to combine multiple camera angles, audio, patient simulators and / or medical device data with minimal lag, maximum quality and optimum security.”

Woldeab spoke from the experience of B-Line Medical’s accomplishments to design, develop and provide the SimCapture and LiveCapture Systems for debriefing for the education, simulation and clinical health care environments – with success. Installed in over 450 healthcare institutions in 25 countries, B-Line’s systems have been advancing simulation and healthcare education for over a decade. The Washington, DC-based community expert explained SimCapture and LiveCapture provide web-based video capture and debriefing capabilities and are the most flexible and intuitive platforms in the industry. “SimCapture enhances and automates simulations and OSCE [Objective Structured Clinical Examinations] management by seamlessly combining high quality video capture, simulator data integration, debriefing and assessment, center management and robust reporting,” she said and added, “LiveCapture is the ‘qualitative eye’ of a hospital quality improvement program and can be used to significantly reduce costly errors while simultaneously improving efficiencies and reimbursement rates. The LiveCapture platform records live clinical events and offers secure streaming, monitoring, debriefing and integrated checklists.”

B-Line reportedly provides its debriefing product customers with returns on investment beyond training. The SimCapture and LiveCapture Systems are used to enhance debriefing opportunities for individual and team performance during simulation-based education (both lab and in situ) and for Quality Improvement programs linked to patient safety, risk management and clinical research.

According to Woldeab, the impact of LiveCapture itself has been two-fold: the mere presence of video-recording and the potential for random audit of this footage, can have a dramatic effect on clinical behaviors, which is likely linked to the Hawthorn and Sentinel Effects. “But more importantly, the videos have contributed to the adoption of a Safety Culture when used in a non-punitive and best practice recognizing manner,” she added.

VirtaMed is a third, representative company providing debriefing capabilities with its products.

The Zurich, Switzerland-based company provides high fidelity virtual reality training solutions in the fields of urology, gynecology and arthroscopy.  Kelly Wettengel, the company’s Product Manager, pointed out, “We pride ourselves on having the most realistic simulators available in the market today and specialize in combing the rubber model, which provides realistic feedback with virtual reality where we can show any pathology, anatomy and complication. We strive to continually update our simulators with new patients and cases and we've recently expanded our arthroscopy line to include a complex teaching module for ACL reconstruction.” ACL reconstruction is one of the most common surgeries in the knee and the company’s ArthroS™ is the first-ever simulator for the entire procedure. The company will also add a hip module to our Arthroscopy simulator in the upcoming months, with continuing expansion of the Arthroscopy simulation line to ankle in the near future. Wettengel added, “This hip module will be the first to allow for access training and safe fluoroscopy can be practiced on this simulator, something not available anywhere else on the market.”

VirtaMed recently expanded its gynecology line to include fertility procedures with the development of the embryo transfer simulator which was developed in partnership with the American Society for Reproductive Medicine, which shows how important virtual reality training has become in medical training. Wettengel continued, “Our entire simulation line includes knee and shoulder arthroscopy, urology procedures including TURB, TURP and laser treatment for BPH, hysteroscopy, and IUD placement.”

With respect to debriefing capabilities, all of VirtaMed’s simulation modules assess competency using expert-defined metrics. “These metrics are specific to each simulator and procedure and cover topics such as patient safety, accuracy and efficiency,” the company’s product manager explained. The simulator stores this data by user for every procedure, along with screen shots of key procedure steps and a video of the entire procedure. This allows for a rapid assessment of each trainee's progression or a comparison between colleagues and skill levels. Data can also be compared across trainees of the same training level to identify which trainees are above and below the desired skills level. “Proctors can then refer to the graphs to ensure that each trainee is learning at the desired pace, identify areas that need improvement and then use the procedure images and videos as debriefing tools to provide focused tips to the user,” Wettengel said and continued, “Finally, our simulators come with suggested courses created by subject matter experts around the world to allow for standardized training.  Alternatively, proctors and create customized courses to address the specific needs for the trainees on a case-by-case basis.”

VirtaMed’s simulation products’ debriefing capabilities are dynamic, with Wettengel pointing out her company’s simulators are always evolving to incorporate findings from the latest research in instructional design, as well as client requests. “We have an educational team who are constantly researching trends in simulated medical education and adult learning theory to devise new ways to improve the educational value of our simulators. We actively pursue feedback from our users and work with them to provide solutions to their training needs,” she concluded.

Putting it Together – an End User’s Perspective

Anna Lerant, the managing director of the Simulation and Interprofessional Simulation Center (SIEC) at the University of Mississippi Medical Center, placed into perspective the importance of debriefing in the continuum of learning, when she pointed out that her center requires all scenario-based simulation to consist of four phases: pre-briefing (orientation, fiction contract); the scenario itself; structured debriefing; and evaluation (by trainees and by the instructors at a minimum).

With respect to the third phase, “the preferred structure for debriefing is “GAS”: “G” (gaps) - emotional decompression, discovery of gaps in self-perception of performance: “A” (analysis)- events and behaviors, which may or may not involve use of recorded video replay; and “S” (summary) – where trainees share the ‘take home’ messages they gleamed from the scenario and debriefing,” she explained.

SIEC’s Debriefing Technology Breakdown

The Jackson, Mississippi-based community expert lifted the veil on debriefing technologies at SIEC, noting the facility has two simulation “stages” with control rooms and adjoining debriefing rooms. Lerant continued, “Both locations are equipped with a home-made recording system consisting of analogue PTZ (pan, tilt, zoom) cameras, directional microphones, an iMac with Elgato software, a video mixer for ‘picture in picture’ or overlay of vital signs, and the integrated ‘voice of god’ function.” Further, the recording can be observed on the debriefing room’s 60-in. [1.5m.] TV in real time, and/or replayed during debriefing.

SIEC’s system reportedly has several advantages to commercially available systems: there is no digital conversion delay causing sound-picture asynchrony; the sound quality is excellent, therefore communication during scenarios can be clearly heard; the resulting movie files are of tolerable size; if vital signs are overlaid, both actions and the vital sign readings are visible.

“Our center mandates audiovisual recording of scenarios only, if the scenario is part of competency assessment or summative evaluation,” Lerant said, and continued, “For learning sessions and/or formative assessments it is at the discretion of the course director and teaching faculty whether they use video recording in their debriefing.”

If video recording is not used, the participants are encouraged to take notes on a large post-it board, which is taken to the debriefing room and used as a record of actions during the Analysis phase.

For all scenarios the center also constructs a specific checklist that allows the observers to assess whether the observable learning objectives of the scenario were met.

Technology Gaps and Opportunities

After establishing the debriefing technology baseline for her center and its capabilities, Lerant described current shortfalls in state-of-the-art simulation & training equipment offerings which preclude the devices from providing yet, higher fidelity debriefings.

According to the community expert, there is a lot of emphasis placed on the number of cameras and image quality, while less attention is given to sound quality. She continued, “Medical simulation has background noises coming from certain directions (for example, hissing of the oxygen flow meter). Commercial solutions usually involve a PZT (lately HD [high definition]) camera-microphone combo, which makes it impossible to filter out the background noise. The result is great pictures with poor sound quality, which makes the recording less valuable during debriefing.”

Most commercial recording systems are observed to be designed to “capture and replay”. The managing director added, “However, there is another, crucial function: real time observation by instructors in the control room and/or observing participants in the debriefing room. They must receive an “in-synch” sound and visual input, without digital delay.”

Additionally, some manikins emit sensor signals (pulse examined, head tilted), which may or may not be necessary for the annotation of the scenario. “Ideally, the user should be able to choose what manikin input should be ‘bookmarked’ and which should not,” Lerant said.

Debriefing Sector Forecast

For its part, B-Line believes that there is a renewed focus on debriefing and its use across one’s healthcare career. This is said to be particularly noticeable in nursing, which is likely driven both by the conclusions drawn from the NCSBN (National Council of State Boards of Nursing) Simulation Study and the NLN (National League for Nursing) Debriefing Across the Curriculum papers. This has led to increasing discussions around debriefing styles and their efficacy, skills training, audit and quality assurance.

Woldeab added, “There is also a growing interest in the use of systems for the ‘direct observation’ element of the Milestones Projects in post-graduate medical education. It is simply impossible for an Attending [physician] to directly observe all of their Resident’s simulation-based and clinical work without video. So the question we must consider now is: How do we help them to do that? B-line believes that our LiveCapture System will be central to addressing these considerations and look forward to the rise of video-based audit & education within the clinical environment.”

To position its debriefing portfolio ot meet these expectations, B-Line is currently developing the next iteration of its code base, which will enable the firm to introduce many more user-friendly features in the coming years. “Our aim is to disrupt ourselves and to ensure that our technology is less hardware dependent and more affordable as the demand grows. We’re also examining the development of supportive materials, which will enhance the user experience and ensure that our system is used more widely and more appropriately by all of our customers,” Woldeab emphasized.

Peter Schenk, PhD, VirtaMed’s senior Product Manager, looked 12-24 months out on the business horizon and observed there is a huge push to go beyond the currently available face validity of simulation. He added, “VirtaMed's virtual reality simulation is in a unique position to offer both visually and haptically accurate feedback, and face and construct validity have been established for the vast majority of our simulators. Further validation studies are currently being conducted at institutions in the US and Europe.”

Dr. Schenk also focused on “the basic question”: what differentiates an excellent surgeon from a good surgeon, and said it is still not answered. “We believe that all the objective data that can be captured on a virtual reality simulator combined with advanced data processing methods will help to understand and quantify the differences,” the company executive said.  “Simply looking at the rapid developments in the gaming industry and all the new VR applications, we believe there is plenty room for simulation in healthcare to become much stronger than today,” he concluded.