The Case for Independent Software Usability Testing: Lessons Learned from a Successful Intervention

Abstract

This report presents the lessons learned from a software usability test for an external customer. An initial evaluation with naive users revealed problems in the user interface and that the customer's objectives were not being met. After initial resistance to making changes in the software, the customer decided to delay release of its product to implement some of the recommendations and changed the focus of initial release to experienced users. The results of a second evaluation conducted on the revised product with experienced users were positive. Several lessons can be learned from the above evaluation: (1) Usability evaluation should be incorporated earlier in the software development cycle to minimize resistance to changes in a hardened user interface; (2) Organizations should have an independent usability evaluation of software products to avoid the temptation to overlook problems to release the product; (3) Multiple categories of dependent measures should be employed in usability testing because subjective measurement is not always consonant with user performance; and (4) Even though usability testing at the later stages of development may not impact software changes, it is useful to point out areas where training is needed to overcome deficiencies in the software.