Testing a Framework for Reliable Classification of Usability Problems

Abstract

The User Action Framework (UAF) is a knowledge base of usability issues and concepts structured to provide a framework and method for classifying usability problems identified during usability evaluation. The UAF is essentially a hierarchical structure of usability attributes that users traverse as a decision structure, selecting the most appropriate classification category and sub-category at each level of the hierarchy. The cumulative set of category choices along the classification path is taken as a sequence of usability attributes that determines a complete classification description of the usability problem in question. The UAF itself has been the subject of usability evaluation and is the product of an extensive, iterative design process. In this paper we report on the reliability of the UAF by measuring the agreement of 10 experienced usability practitioners as they classify 15 different usability problems.