Estimation of item parameters and examinees’ mastery probability in each domain of the Korean Medical Licensing Examination using a deterministic inputs, noisy “and” gate (DINA) model

Abstract

Purpose: The deterministic inputs, noisy “and” gate (DINA) model is a promising statistical method for providing useful diagnosticinformation about students’ level of achievement, as educators often want to receive diagnostic information on how examinees did oneach content strand, which is referred to as a diagnostic profile. The purpose of this paper was to classify examinees of the Korean Medical Licensing Examination (KMLE) in different content domains using the DINA model. <br>Methods: This paper analyzed data from the KMLE, with 360 items and 3,259 examinees. An application study was conducted to estimate examinees’ parameters and item characteristics. The guessing and slipping parameters of each item were estimated, and statisticalanalysis was conducted using the DINA model. <br>Results: The output table shows examples of some items that can be used to check item quality. The probabilities of mastery of eachcontent domain were also estimated, indicating the mastery profile of each examinee. The classification accuracy and consistency for 8content domains ranged from 0.849 to 0.972 and from 0.839 to 0.994, respectively. As a result, the classification reliability of the diagnostic classification model was very high for the 8 content domains of the KMLE. <br>Conclusion: This mastery profile can provide useful diagnostic information for each examinee in terms of each content domain of theKMLE. Individual mastery profiles allow educators and examinees to understand which domain(s) should be improved in order tomaster all domains in the KMLE. In addition, all items showed reasonable results in terms of item parameters.