Instant data analysis: Conducting usability evaluations in a day

When designing a usability evaluation, key decisions must be made regarding methods and techniques for data collection and analysis. Although there is a strong body of research within human-computer interaction regarding the appropriate choices of data collection methods and techniques much less research has been conducted examining and comparing methods and techniques for analyzing the collected data. This paper presents a data analysis technique which allows usability evaluations to be conducted, analyzed and documented in a day; Instant Data Analysis (IDA). The use of this technique is exemplified through a usability evaluation of a software product for a large hospital for which traditional video data analysis and Instant Data Analysis were applied independently through a controlled experiment. Among our key findings, the experiment revealed that in only 10% of the time required to do the video data analysis, Instant Data Analysis identified 85% of the critical usability problems in the system being evaluated. At the same time, the noise of unique usability problems usually characterizing video data analysis was significantly reduced.