TY - JOUR
T1 - The promise and challenge of including multimedia items in medical licensure examinations
T2 - Some insights from an empirical trial
AU - Shen, Linjun
AU - Li, Feiming
AU - Wattleworth, Roberta
AU - Filipetto, Frank
N1 - Copyright:
Copyright 2017 Elsevier B.V., All rights reserved.
PY - 2010/10
Y1 - 2010/10
N2 - BACKGROUND: The Comprehensive Osteopathic Medical Licensing Examination conducted a trial of multimedia items in the 2008-2009 Level 3 testing cycle to determine (1) if multimedia items were able to test additional elements of medical knowledge and skills and (2) how to develop effective multimedia items. METHOD: Forty-four content-matched multimedia and text multiple-choice items were randomly delivered to Level 3 candidates. Logistic regression and paired-samples t tests were used for pairwise and group-level comparisons, respectively. RESULTS: Nine pairs showed significant differences in either difficulty or/and discrimination. Content analysis found that, if text narrations were less direct, multimedia materials could make items easier. When textbook terminologies were replaced by multimedia presentations, multimedia items could become more difficult. Moreover, a multimedia item was found not uniformly difficult for candidates at different ability levels, possibly because multimedia and text items tested different elements of a same concept. CONCLUSIONS: Multimedia items may be capable of measuring some constructs different from what text items can measure. Effective multimedia items with reasonable psychometric properties can be intentionally developed.
AB - BACKGROUND: The Comprehensive Osteopathic Medical Licensing Examination conducted a trial of multimedia items in the 2008-2009 Level 3 testing cycle to determine (1) if multimedia items were able to test additional elements of medical knowledge and skills and (2) how to develop effective multimedia items. METHOD: Forty-four content-matched multimedia and text multiple-choice items were randomly delivered to Level 3 candidates. Logistic regression and paired-samples t tests were used for pairwise and group-level comparisons, respectively. RESULTS: Nine pairs showed significant differences in either difficulty or/and discrimination. Content analysis found that, if text narrations were less direct, multimedia materials could make items easier. When textbook terminologies were replaced by multimedia presentations, multimedia items could become more difficult. Moreover, a multimedia item was found not uniformly difficult for candidates at different ability levels, possibly because multimedia and text items tested different elements of a same concept. CONCLUSIONS: Multimedia items may be capable of measuring some constructs different from what text items can measure. Effective multimedia items with reasonable psychometric properties can be intentionally developed.
UR - http://www.scopus.com/inward/record.url?scp=77958506271&partnerID=8YFLogxK
U2 - 10.1097/ACM.0b013e3181ed3c80
DO - 10.1097/ACM.0b013e3181ed3c80
M3 - Article
C2 - 20881705
AN - SCOPUS:77958506271
SN - 1040-2446
VL - 85
SP - S56-S59
JO - Academic Medicine
JF - Academic Medicine
IS - 10 SUPPL.
ER -