Abstract
Introduction
Although a thoughtfully designed Objective Structured Clinical Examination (OSCE) is a robust and valid assessment tool, debate exists about the effectiveness of OSCEs to authentically mirror real-life scenarios. Limited studies have explored the extrapolation inference in Kane Validity’s Framework and explored how assessment relates to real life practice. This study aims to compare the performance of students in an OSCE to their performance in real-life, and explore factors affecting performance.
Methods
A sequential mixed methods approach was used. Mystery shoppers visited pharmacy students on their community pharmacy placement and simulated the same case scenario students were given a recent infectious diseases OSCE. Students were marked with the same rubrics and these marks were compared with their OSCE score. The mystery shopping visit was then revealed to all the students and all students were asked to participate in a semi-structured interview.
Results
115 visits were conducted across Monash Australia and Malaysia campuses, and 36 follow-up interviews were completed. The mean mystery shopping score was 36.2% compared to the actual OSCE score of81.1%. About 93% of students scored lower in the mystery shopping encounters compared to their OSCE (p<0.001). Only 3.5% did better, while 3.5% scored the same. In the interviews, students reflected that a real life patient is easier to manage than in an OSCE; stating in real-life they could speak more freely, real life patients were more open to different treatment options, and the presence of work colleagues eased their nervousness.
Discussion
Students scored lower on placement than in OSCEs even though they reflected that it is easier to manage a patient in real-life; outlining challenges to replicate a real-life pharmacy situation in an OSCE. Whilst OSCEs are useful for testing process type skills, clinical problem solving may be best assessed in a workplace environment.
Although a thoughtfully designed Objective Structured Clinical Examination (OSCE) is a robust and valid assessment tool, debate exists about the effectiveness of OSCEs to authentically mirror real-life scenarios. Limited studies have explored the extrapolation inference in Kane Validity’s Framework and explored how assessment relates to real life practice. This study aims to compare the performance of students in an OSCE to their performance in real-life, and explore factors affecting performance.
Methods
A sequential mixed methods approach was used. Mystery shoppers visited pharmacy students on their community pharmacy placement and simulated the same case scenario students were given a recent infectious diseases OSCE. Students were marked with the same rubrics and these marks were compared with their OSCE score. The mystery shopping visit was then revealed to all the students and all students were asked to participate in a semi-structured interview.
Results
115 visits were conducted across Monash Australia and Malaysia campuses, and 36 follow-up interviews were completed. The mean mystery shopping score was 36.2% compared to the actual OSCE score of81.1%. About 93% of students scored lower in the mystery shopping encounters compared to their OSCE (p<0.001). Only 3.5% did better, while 3.5% scored the same. In the interviews, students reflected that a real life patient is easier to manage than in an OSCE; stating in real-life they could speak more freely, real life patients were more open to different treatment options, and the presence of work colleagues eased their nervousness.
Discussion
Students scored lower on placement than in OSCEs even though they reflected that it is easier to manage a patient in real-life; outlining challenges to replicate a real-life pharmacy situation in an OSCE. Whilst OSCEs are useful for testing process type skills, clinical problem solving may be best assessed in a workplace environment.
| Original language | English |
|---|---|
| Pages | 78-78 |
| Number of pages | 1 |
| Publication status | Published - 2023 |
| Event | Australian & New Zealand Association for Health Professional Educators Conference 2023 - Gold Coast Convention Centre, Gold Coast, Australia Duration: 26 Jun 2023 → 29 Jun 2023 https://www.anzahpe.org/conference-2023 https://az659834.vo.msecnd.net/eventsairaueprod/production-eventstudio-public/b1d5ac398f764c84901658b420071c27 |
Conference
| Conference | Australian & New Zealand Association for Health Professional Educators Conference 2023 |
|---|---|
| Abbreviated title | ANZAHPE 2023 |
| Country/Territory | Australia |
| City | Gold Coast |
| Period | 26/06/23 → 29/06/23 |
| Other | Turning Tides: Navigating the Opportunities |
| Internet address |