IDE Solution Uses Big Data to Improve CBT Exams

• Astrom’s new IDE application can help airlines and training centers resolve common challenges
• The company’s IDE big data solution was developed using IRT theory

Astrom Training Solutions has released its new Intelligent Data Engine (IDE), which uses advanced big data analysis to evaluate exam results. The main purpose of the IDE application is to help airlines and training centers resolve common challenges including how to relieve instructors from spending a large amount of time assigning difficulty levels to each question, how to calculate a pilot’s true theory ability based on their clustered scores, and much more.  

Most exam systems use Classical Test Theory (CTT) to analyze exam results. CTT is an approach that is based on mathematics – primarily averages, proportions, and correlations. It is more than 100 years old, but is still used quite often for exam evaluation. CTT is very simple and easy to understand, which makes it convenient for working directly with content authors to evaluate and diagnose tests, however, CTT does not provide a deeper understand of exam structure and is not optimum for improving examinations.
 
In Contrast, Item Response Theory (IRT) is a highly advanced approach for analyzing tests. Moreover, IRT is used not just to analyze exams; it is a complete psychometric framework that changes how question banks are developed, exam formats are designed, exams are delivered, and scores are produced. There are many benefits to the IRT approach that justify its complexity, and this is why most leading examination authors in the world now utilize IRT.
 
 Astrom Training Solutions’ IDE Big Data solution was developed using IRT theory. IRT theory is predominantly based on the Rasch Model. However, our Big Data team has collaborated with a distinguished statistics professor to publish new research in this area based on Bayesian theory. Our IDE algorithms have been developed combining both the Rasch model and more recent advanced Bayesian algorithms to reach the examination analysi. Astrom’s IDE application now brings this analytical capability to aviation training managers.