TY - JOUR
T1 - Using Kane's Validity Framework to Compare an Integrated and Single-Skill Objective Structured Clinical Examination
AU - Lim, Angelina S.
AU - Abeyaratne, Carmen
AU - Reeve, Emily
AU - Desforges, Katherine
AU - Malone, Daniel Thomas
N1 - Publisher Copyright:
© 2024 The Author(s)
PY - 2024/9
Y1 - 2024/9
N2 - Objective: The aim of this study was to compare the validity of an integrated objective structured clinical examination (OSCE) station assessing both oral and written components with that of an OSCE station assessing 1 single skill (oral only), both targeted at assessing taking a best possible medication history. Methods: A convergent mixed-methods design that used the 4 inferences of Kane's validity framework (scoring, generalization, extrapolation, and implications) as a scaffold to integrate qualitative data (post-OSCE reflections) and quantitative data (assessment grades and categories of medication errors) was applied. Results: In 2022, 216 students completed the OSCE station with the oral component alone, while in 2023, 254 students completed the integrated (oral and written) OSCE station. Students in 2023 performed significantly better, with a median score of 88% vs 80% in 2022. There was a greater proportion of commission errors in the integrated assessment (20.4% vs 15.3%), but fewer omission errors (29.9% vs 31.8%) and patient profile errors (5.1% vs 69.4%). Student reflections revealed that conversations were rushed in the integrated assessment, with a greater focus on written formatting, but an appreciation for the authenticity and structured format of the integrated OSCE compared with the single-skill OSCE alone. Conclusion: Students completing the integrated OSCE (with oral and written components) had fewer patient profile and medication omission errors than students who completed the oral-only OSCE. Considering Kane's validity framework, there was a stronger argument for the more authentic integrated OSCE in terms of the inferences of extrapolation and implications.
AB - Objective: The aim of this study was to compare the validity of an integrated objective structured clinical examination (OSCE) station assessing both oral and written components with that of an OSCE station assessing 1 single skill (oral only), both targeted at assessing taking a best possible medication history. Methods: A convergent mixed-methods design that used the 4 inferences of Kane's validity framework (scoring, generalization, extrapolation, and implications) as a scaffold to integrate qualitative data (post-OSCE reflections) and quantitative data (assessment grades and categories of medication errors) was applied. Results: In 2022, 216 students completed the OSCE station with the oral component alone, while in 2023, 254 students completed the integrated (oral and written) OSCE station. Students in 2023 performed significantly better, with a median score of 88% vs 80% in 2022. There was a greater proportion of commission errors in the integrated assessment (20.4% vs 15.3%), but fewer omission errors (29.9% vs 31.8%) and patient profile errors (5.1% vs 69.4%). Student reflections revealed that conversations were rushed in the integrated assessment, with a greater focus on written formatting, but an appreciation for the authenticity and structured format of the integrated OSCE compared with the single-skill OSCE alone. Conclusion: Students completing the integrated OSCE (with oral and written components) had fewer patient profile and medication omission errors than students who completed the oral-only OSCE. Considering Kane's validity framework, there was a stronger argument for the more authentic integrated OSCE in terms of the inferences of extrapolation and implications.
KW - Best possible medication history
KW - Kane's validity framework
KW - Medication reconciliation
KW - Objective structured clinical examinations
UR - http://www.scopus.com/inward/record.url?scp=85200110502&partnerID=8YFLogxK
U2 - 10.1016/j.ajpe.2024.100756
DO - 10.1016/j.ajpe.2024.100756
M3 - Article
AN - SCOPUS:85200110502
SN - 0002-9459
VL - 88
JO - American Journal of Pharmaceutical Education
JF - American Journal of Pharmaceutical Education
IS - 9
M1 - 100756
ER -