Group editor Marty Kauchak provides an update on one successful pilot project which is advancing the business case to invest in AR technologies for US Navy learning audiences.

An early and successful US Navy-industry collaborative effort is advancing the state-of-the-art for augmented reality (AR) training in the sea service. The initiative is progressing under a Navy Small Business Innovative Research (SBIR) contract award and is focused on maintenance personnel. The outcomes from this investment hold the potential to allow the Navy, and indeed, other military services and departments, to expand AR to advance competencies in other individual and unit skills.

Aligning with High-Level Trends

This feature article provides a snapshot of the brisk pace of activity occurring in an Office of Naval Research-funded SBIR contract titled “Mixed reality Adaptive Scaffolding Tool for Enhancing Readiness (MASTER) and Refinement and Extension of MASTER (RE-MASTER)”.

The efforts to date of the SBIR contract’s awardee, Design Interactive Inc. (DI), are in lock step with overarching trends in the defense learning sector. S&T companies can now rarely “go it alone” on SBIR and other US DoD contract vehicles. The collaborative effort for this contract is of special note. On the military side of the ledger, in addition to ONR, DI’s service team mates include Naval Education and Training Command (NETC), CNA (Center for Naval Analyses), Program Executive Office Aircraft Carriers, Naval Sea Systems Command and USS Gerald R. Ford (CVN-78). DI’s industry teammates include Huntington Ingalls Industries, CACI and Cape Henry Associates.

A key differentiator making this SBIR contract successful to date, beyond command sponsorship and funding, has been the involvement and guidance of the technology topic point of contact (TPOC), Ray Perez, PhD. Victoria Claypoole, PhD, Research Associate II in DI’s eXtended Reality (XR) Division, reflected that ONR’s Perez has been very involved throughout the development and testing process. “We do not often get TPOCs who take such a strong interest in their SBIRs, so we are very thankful to have the support and interest from Dr. Perez,” she added.

As MS&T and the Halldale Group’s editorial focus expands into S&T developments in adjacent high-risk industries, another recurring theme is the “cross pollination”, or migration, of technology capabilities and thrusts across communities. In this case, DI’s solutions are also being used to develop AR training for US Army Tactical Combat Care courses. Transferring AR from a maintenance application under this Navy SBIR effort to a medical context “is actually quite easy,” pointed out Claypoole. Dr. Claypoole further explained, “This is because we, at DI, have established a sound pedagogical model for XR instruction that was developed by integrating theories of human cognition, instructional science, and adult skill acquisition. This pedagogy details what type of content to present and how to present this contextualized, AR information across levels of learner proficiency, from novice to expert.” Components of this pedagogy were validated during CNA’s evaluation. The entire model is reported to be currently under review at the International Journal of Artificial Intelligence in Education.”

Delving further into DI’s pedagogy, Claypoole noted the model has strong empirical backing and provides applicable directive in developing scenario elements, such as stressors and feedback, within AR. She continued, “We believe, that if you have a sound pedagogical model as your starting framework and truly understand if the task you are trying to train is suitable for AR (e.g., installs high psychomotor skills, is delivered at the point of need, or develops crucial spatial knowledge), then instantiating the course content is really no more than ‘window dressing.’

How to present information on symptoms related to a failed network versus symptoms on a casualty injury is really no different in the sense that you are spatially contextualizing both sets of information and embodying psychomotor skills through the in-scenario formative evaluations,” and emphasized, “The point being, you do not need a different set of organizing principles when developing AR content for different sectors. There are best practices, similar to every other training medium, and if followed, result in the expected learning gains.”

Early ROIs

Beyond applicability of AR as a learning enabler across different military occupational fields, other outcomes achieved by SBIR contract awardee Design Interactive, Inc. and its teammates are further raising the AR technology baseline to a higher level.

Claypoole observed the validation portion of this evaluation sought to determine if DI’s AR scenarios resulted in immediate learning gains – the result of which indicated that an increase in 32% was demonstrated. Additionally, CNA examined aspects of sailor adoption to determine if these service members were comfortable receiving training in this medium and if they had issues with a system collecting their performance data instead of a human. The community expert noted, “Responses to this evaluation, coupled with the demonstrated learning gains, indicated that sailors are not only comfortable with this new, emerging form of training, but they are also learning in it.”

Further, PEO Aircraft Carriers’ leadership introduced DI to a number of industry leaders from Huntington Ingalls Industries, CACI, and Cape Henry Associates, which allowed the contract awardee to secure access to the sailors onboard Ford. Dr. Claypoole noted without their support and assistance, DI’s AR training scenarios would not be as optimized for the sailors as they currently are. “Which brings us to our last area of support: the sailors themselves. The sailors we worked with on the CVN 78 were integral in the development and execution of the AR training scenarios as they provided feedback and support. The aforementioned validation and examination with CNA were conducted with actual novice maintainers onboard CVN 78. This would not have been possible without the sailors’ support, interest, and willingness to participate.”

A vital US Navy partner in this SBIR was USS Gerald R. Ford (CVN-78). The ship provided maintenance personnel across occupational specialties to support the SBIR's activities. Image credit: US Navy/Mass Communication Specialist 2nd Class Ruben Reed.

The DI-led team cast a wide net for types of maintainers to support this SBIR pilot project, some of whom from USS Gerald R. Ford include Combat Data Systems and Tactical Networks Division (CS 5) - MCMS Maintainers and PAWDS Maintainers (Engineering Non-Nuclear). Of added significance, select individuals are scheduled this September to be put through the paces of DI’s AR scenarios in the service’s new Carrier-Advanced Reconfigurable Training System (C-ARTS), which is stationed at Norfolk Naval Base, Virginia. Initially developed by Cape Henry Associates, C-ARTS is bringing a frontline Navy strategy to deliver high velocity learning to the point of need.

Sailor Performance Data

A vital, high-level, service perspective on the state of AR and its applicability in this SBIR project was provided by Michele Harrison, Branch Head for Strategy, Future Capabilities (Innovation/R&D), Analytics & Training Effectiveness, NETC.

From a service-wide perspective, Harrison observed her command’s interest in the use of AR for training is in its potential to collect sailor performance data. “This can provide NETC the capability to assess its training effectiveness in meeting fleet requirements and readiness, and to make any needed adjustments in its schoolhouse training,” she said and added, “Training effectiveness assessment data is currently gathered through surveys that have minimal response rates. The capability to collect data automatically through AR is a significant improvement.”

Ms. Harrison then pointed out assessing training through an AR application allows sailors to: demonstrate what they have learned through hands-on applications; maintain situational awareness, which makes it safer to use in a shipboard environment than virtual reality; and unlike paper-and-pencil assessments, the data is automatically collected (minimizing labor) and can be sent electronically for analysis.

From NETC’s viewpoint, this SBIR pilot test further demonstrated that it was possible to have an augmented reality-goggle application to request answers to questions about procedures and automatically collect the responses. The application collected data on the correctness of responses and how long it took sailors to respond. The NETC official continued, “The AR application can potentially be used for assessing training on land or at sea. Additionally, newer goggle versions may provide greater assessment capability. NETC’s hope is to analyze this additional assessment capability in the near future.”

“Ready for Prime Time”

MS&T continues to report on the quick pace of developments in AR and adjacent technology sectors in military organizations around the globe – which led to another appropriate question for DI’s Claypoole, is AR a mature learning technology, ready to meet training readiness challenges and gaps? Claypoole responded, “We believe that the current AR technology is in-fact, ‘ready for prime time’ – which is exciting because the giants in the field are going to keep pushing the envelope and developing even better versions of what currently exists. AR has been around for decades, but only recently have we seen options for commercialized use, which has allowed small businesses like us to break into this sector and use the technology for education and training.”

Concurrently, the supported software platforms have also evolved in recent years – providing software engineers with the opportunity to use off-the-shelf software applications for which to develop AR content. The subject matter expert again emphasized, “There is a plethora of research that shows AR, if developed correctly, can result in learning gains. So, the technology is ready, the software applications exist, and AR training has been proven successful. We are now entering a new phase – mass adoption. It will be up to the DoD to determine how quickly AR is holistically adopted across the services for formalized training.”

On MS&T’s Watch List

This instance of a Navy-sponsored and funded AR-enabled pilot project is elevating the technology baseline for learning in a focused occupational group, maintenance personnel – and has potential to support other learners. Of added interest is the concurrent development of C-ARTS, which is promises to further expand the art of learning across the Navy’s carrier force. MS&T and the Halldale Group website – - will continue to keep pace with and report on developments, in these learning spaces.

A Learning Technology Multiplier for US Navy Carrier Crews

One of the most significant US Navy training projects in development is Carrier-Advanced Reconfigurable Training Systems (C-ARTS). The project is designed to immerse the crews of this sea service’s carriers into next generation learning experiences. Group editor Marty Kauchak reports.

Expandable, Mobile and Reconfigurable

Crew members from USS Gerald R Ford (CVN-78) are using the Training Extension Package, which simulates the Primary Flight Control and catapult functions of an aircraft carrier. This learning technology enable the above catapult officers to learn/practice the procedures to launch aircraft from Ford. Image credit: Cape Henry Associates, Inc.

C-ARTS’ genesis took shape following I/ITSEC 2016 and subsequent discussions. The project’s initial, major milestone was the award of a contract to Cape Henry Associates, Inc. (CHA) by US Navy PMS 378 (PEO Carriers New Acquisition Program) for the construction, outfitting and operations of C-ARTS units to support USS Gerald R. Ford (CVN-78)-class training requirements associated with Contractor Furnished Equipment (CFE) systems. Neal Jefferis, C-ARTS Operations Manager, and a CHA employee supporting the CVN 78-class training efforts, recalled this early effort was led by Mr. Kelly Schneider, the PMS 378 Manpower Personnel and Training Manager. He continued, “With over 30 new systems and with throughput projects limited during the earlier years of the CVN 78-class’ service life, the decision to mandate a reconfigurable training system approach to providing training when and as required was a critical component. The result was the 53-ft [16m]-long, expandable, mobile and reconfigurable training unit called C-ARTS.”

Early on, the Navy’s requirements included the forward-looking decision to address the learning modalities of today’s sailors. Jefferis explained this strategy included making the C-ARTS units technologically advanced with a Learning Management System (LMS) capable of working with eXtended Reality (XR) technologies (Augmented Reality (AR), Virtual Reality (VR), Mixed Reality (MR)), video, adaptive manufacturing (3-D manufacturing) and similar technologies increasingly used in training systems today [see accompanying MS&T 3/2020 article “New Realities for US Navy Training”]. As significant, beyond this technology mix, “it was also an intentional design concept from the government customers that training would always be instructor-led even as various computer technologies were used to enrich the learning experience.”

C-ARTS Unit #3 (background) and #4 (foreground) are shown onsite at Naval Station Norfolk, Virginia. Image credit: Cape Henry Associates, Inc.

C-ARTS was established as a two-unit pair, an advanced Electronic Classroom and a Maintenance Laboratory. The C-ARTS Operations Manager explained, “This enables the students to perform a variety of skills-related tasks using actual equipment and or training enhancements or augmentations leveraging the XR, video and adaptive manufacturing technologies,” and emphasized, “The combination of the Electronic and Maintenance Laboratory pairs of units with the instructor-led training and the technology applications, provides the CVN 78-class sailors a robust and dynamic blended training approach supporting the variety of learning modalities of today’s sailors.”

A number of companies and government resources worked with PMS 378 and CHA. These included Huntington Ingalls Industries’ Newport News Shipyard (HII-NNS) and Technical Services Division’ Fleet Support Group (HII-FSG), CACI, the Naval Air Warfare Center Training Systems Division (NAWCTSD) and the Naval Information Warfare Center (NIWC). Jefferis further noted, each organization has contributed to the overall robust, technically advanced curriculum of the C-ARTS Courses and technologies.

Advances in Capability, Affordability & Application

As the C-ARTS technologies become more mature and affordable, the Instructional System Designers (ISDs) who author the Curriculum Content are learning about the possibilities to innovate. The C-ARTS Operations Manager pointed out that at the same time that “Finding and purchasing the appropriate software and hardware for the training task and then finding as many reasonable uses for those software and hardware applications are critical to being fiscally responsible”. Continuing, he told us “A small paradox occurs where the ISDs are not ready to commit to the use of a software or hardware that is not already acquired and validated in its applications. However, the … shorter and less expensive developmental process continues to expand the opportunity to make the decision to buy and apply them in new efforts.”

C-ARTS Expansion

Additional C-ARTS units are expected to be procured to support CVN 78-class ships; the Pre-Commissioning Unit John F Kennedy (CVN 79); and the future USS Enterprise (CVN 80) and USS Doris Miller (CVN 81). The industry manager concluded, “As these ships are added to the fleet over the next decade, the need to retain and expand the flexible, advanced and reconfigurable Training System that is C-ARTS will remain. The technology within C-ARTS will be updated as similar technology advances and is updated in other government and industry applications, but the C-ARTS units themselves are designed to accommodate such changes.”