Comparing pharmacy student performance in simulated Objective Structured Clinical Exam (OSCE) with work-integrated learning contexts

Research output: Contribution to conferenceAbstract

Abstract

Background: Although a thoughtfully designed Objective Structured Clinical Examination(OSCE) is a robust and valid assessment tool, debate exists about the effectiveness of OSCEs to authentically mirror real-life scenarios. Limited studies have explored the extrapolation inference in Kane Validity’s Framework.

Summary of work: A sequential mixed methods approach was used. Mystery shoppers visited pharmacy students on their community pharmacy placement and simulated the same case scenario students were given a recent infectious diseases OSCE. Students were marked with the same rubrics and these marks were compared with their OSCE score. The mystery shopping visit was then revealed to all the students and all students were asked to participate in a semi structured interview.

Results: Overall, 92 mystery shopper (Work Based Assessment (WBA)) visits with students were conducted and 36 follow-up interviews were completed. The median WBA score was39.2% lower compared to the OSCE score (p < 0.001). Interviews revealed students knew they did not perform as well in the WBA compared to their OSCE, but reflected that they still need OSCEs to prepare them to manage a real-life patient.

Discussion: Many students related their performance to how they perceived their role in OSCEs versus WBAs, and that OSCEs allowed them more autonomy to manage the patient as oppose to an unfamiliar workplace.

Conclusions: OSCEs performance did not correlate to real life performance in this instance; however, students still valued having an OSCE before a WBA.

Take-home messages: Students scored lower on placement than in OSCEs even though they reflected that it is easier to manage a patient in real-life; outlining challenges to replicate a real life pharmacy situation in an OSCE, and challenges to testing extrapolation. Whilst OSCEs are useful for testing process type skills, clinical problem solving may be best assessed in a workplace environment.
Original languageEnglish
Number of pages1
Publication statusPublished - 2024
EventOttawa Conference 2024: Assessment: Near and far - Melbourne Convention and Exhibition Centre, Melbourne, Australia
Duration: 24 Feb 202428 Feb 2024
http://ottawa2024.au/
https://ottawa2024.au/Microsite/content.html (Abstracts)

Conference

ConferenceOttawa Conference 2024
Abbreviated titleOttawa 2024
Country/TerritoryAustralia
CityMelbourne
Period24/02/2428/02/24
OtherOttawa Conference on the Assessment of Competence in Medical and the Health Care Professions 2024
Internet address

Cite this