Psychometric evaluation of a direct observation of procedural skills assessment tool for ultrasound-guided regional anaesthesia

M J Watson, D M Wong, R Kluger, A Chuan, Michael D Herrick, I Ng, Damian Castanelli, L Lin, A K Lansdown, M J Barrington

Research output: Contribution to journalArticleResearchpeer-review

28 Citations (Scopus)


Assessment tools must be investigated for reliability, validity and feasibility before being implemented. In 2013, the Australian and New Zealand College of Anaesthetists introduced workplace-based assessments, including a direct observation of a procedural skills assessment tool. The objective of this study was to evaluate the psychometric properties of this assessment tool for ultrasound-guided regional anaesthesia. Six experts assessed 30 video-recorded trainee performances of ultrasound-guided regional anaesthesia. Inter-rater reliability, assessed using absolute agreement intraclass correlation coefficients, varied from 0.10 to 0.49 for the nine individual nine-point scale items, and was 0.25 for a total score of all items. Internal consistency was measured by correlation between total score and overall performance scale item (r = 0.68, p <0.001). Construct validity was demonstrated by the total score correlating with trainee experience (r = 0.51, p = 0.004). The mean time taken to complete assessments was 6 min 35 s. ? 2014 The Association of Anaesthetists of Great Britain and Ireland.
Original languageEnglish
Pages (from-to)604 - 612
Number of pages9
Issue number6
Publication statusPublished - 2014

Cite this