The measurement of information systems effectiveness

Abstract

Information system effectiveness is an important phenomenon for both researchers and practitioners. Despite widespread interest, and the importance of the uses, there have been no efforts to validate Computerworld's Premier 100 rankings of information system effectiveness. This paper uses structural equation modeling in an attempt to validate the measuring instrument used to derive the Computerworld rankings.Alternative models for the measuring instrument are proposed. Using a reflective model, the findings raise doubts as to the reliability of the rankings, and both content validity and construct validity are also suspect. The reliability and validity are problematic because multiple indicators of the same construct must be homogeneous for it to make sense to combine them into a composite index. A solution to this problem is to represent information system effectiveness as a multidimensional construct, that is part of a causal model. Based on previous research in the area, suggestions are offered to improve the measuring instrument.