Multiple meta-assessment measures of a quality process: toward institutional effectiveness

Abstract

Purpose The purpose of this study is to share quality process experience from a US comprehensive university to use both direct (participation rate and assessment quality) and indirect assessment measures (assessment survey) to evaluate the quality process. Design/methodology/approach A mixed method design was used to evaluate the quality process. The qualitative design used thematic analysis to find the common themes from the assessment survey. The quantitative design included reliability and internal consistency for the survey and inter-rater reliability for the peer review. Findings Findings indicated that participation rate and assessment quality improved over the years. Faculty provided positive feedback about assessment website, the peer review process and feedback but negative perspectives on the assessment management software and heavy workload on assessment activities. The researcher shared the actions made based on three-year assessment results, especially how leaders have used it to align with planning and budgeting to close the institutional effectiveness loop. Research limitations/implications This research had two limitations. The inter-rater reliability of the peer review was not high and the number of peer review reports in two years were so different, so the results could not be checked for statistical significance. Practical implications The meta-assessment results could have important implications for other universities to improve the quality process. Originality/value There have been limited studies on using both direct and indirect assessment measures to evaluate the quality process. This study uses multiple measures to provide accurate results for administrators’ decision-making in resource allocation.