Challenges of conducting a multi-institutional predictive validity study.

Irene Tatjana Lichtwark, Selina Thi Tran, Timothy William Friedman, Dulce Lay, Marilyn Annette Mercer, Margaret Hay

Research output: Contribution to conferenceAbstract


Background and Aims
This Australian multi-centre study investigates the predictive validity of the Undergraduate Medicine and Health Sciences Admission Test (UMAT) and other admissions criteria on medical course performance. Compilation of admissions and assessment data across 11 universities presented a significant challenge. This presentation aims to share solutions to these challenges so that others undertaking multi-centre studies may benefit from our experiences.

Deans of 12 Australian and New Zealand universities using the UMAT in selection of medical students were invited to join the study. Eleven institutions participated. The lead institution prepared the content for submission to each institution’s research ethics committee, which increased engagement with the study. A QualticsTM survey was used to obtain details of admissions and course-related assessment data. Few universities used the template, instead submitting data in a range of formats making standardisation of data onerous. Difficulties arose from inconsistencies with variable format, large variability in assessment data format, and varying levels of engagement with the project across participating universities.

Data were de-identified and merged into a single file consisting of N=10471 medical students across 11 universities. Admissions and within course assessment data for 7 commencing and 2 completed medical student cohorts are included. This represents the most comprehensive database of medical student admissions and within course performance data ever established in Australia and New Zealand.

Conclusions / Recommendations
Across institutions, the course admissions and assessment data varied considerably in quantity, quality and format. This was a major challenge for our project. However, there are wider implications at an institutional level for quality assurance and scholarship.
Acknowledgment: The UMAT Consortium for funding the research project.
Original languageEnglish
Number of pages1
Publication statusPublished - 24 Apr 2015
EventAustralasian Student Selection for the Health Professions Conference - Exhibition Centre, Melbourne, Australia
Duration: 8 Apr 201510 Apr 2015


ConferenceAustralasian Student Selection for the Health Professions Conference
Abbreviated titleSSHPC

Cite this