Effectiveness of Various Automated Readability Measures for the Competitive Evaluation of User Documentation

Abstract

I examined samples from a number of companys' user publications using several automated reading measures and a graphics/text ratio. The goal was to answer two questions: Were there reliable differences in writing style among the competitors? If so, were these differences related to their rank position in published surveys of user satisfaction with documentation? Of the measures included in the study, only the Cloudiness Count had any significant relationship to rank position in the surveys. A second evaluation, focused on the components of the Cloudiness Count, indicated that both of its components (passive voice and ‘empty’ words - a type of infrequent word) contributed equally to its effectiveness. This is consistent with psycholinguistic research that indicates that it is harder for people to extract the meaning from a passive sentence relative to its active counterpart, and that word frequency is the variable with the most influence on the speed of lexical access.