Evaluating antimicrobial therapy: How reliable are remote assessors?

Menino Osbert Cotta, Tim Spelman, Caroline Chen, Rodney S. James, Danny Liew, Karin A. Thursky, Kirsty L. Buising, Caroline Marshall

Research output: Contribution to journalArticleResearchpeer-review

6 Citations (Scopus)


Introduction Assessing the quality of antimicrobial prescribing provides hospitals with a means of targeting and measuring the impact of antimicrobial stewardship interventions. There are limited data available on the reliability of these assessments among different types of hospital assessors deployed away from the bedside (ie remotely). Importantly, it is unclear if assessors inexperienced in clinical infectious diseases can reliably evaluate the quality of antimicrobial prescriptions. This study sought to determine the reliability of assessments made by remote hospital assessors with different levels of clinical infectious diseases experience. These assessments were based on (1) concordance with national prescribing guidelines and (2) ‘overall appropriateness’. Methods 180 prescriptions were assessed for ‘concordance with guidelines’ and ‘overall appropriateness’ at the bedside (ie locally). Prescription data were then given to fifteen remote assessors. These assessors were blinded to local assessments. Inter-rater reliability was calculated using Fleiss' kappa statistics. Results Higher levels of agreements were achieved for ‘concordance with guidelines’ assessments. Local and remote antimicrobial management teams had the highest level of agreement and this improved when looking at antimicrobial treatment used to treat respiratory tract infections (kappa score = 0.67). Reliability in assessments was moderate for local pharmacist assessments and fair to slight for local infection control assessments. Conclusions There is scope to develop tools that will improve assessment reliability of antimicrobial therapy among remote assessors. Clinical pharmacists provide reliability comparable to infectious diseases experts, however, infection control practitioners may require further education and training to improve reliability in assessments.

Original languageEnglish
Pages (from-to)3-10
Number of pages8
JournalInfection, Disease and Health
Issue number1
Publication statusPublished - 1 May 2016
Externally publishedYes


  • Antimicrobial stewardship
  • Antimicrobials
  • Appropriateness
  • Guideline concordance
  • Inter-rater reliability

Cite this