Diabetologists' Judgments of Diabetic Control: Reliability and Mathematical Simulation

Abstract

In study 1, laboratory and supervised blood or urine test data from actual cases were used to develop patient profiles. Seven diabetologists from the same institution rated the diabetic control of 125 profiles on a four-point scale (1 = poor, 2 = fair, 3 = good, 4 = excellent). Six of the 7 diabetologists demonstrated adequate intra- and interrater reliability. Study 2 assessed the reliability of judgments of diabetic control made by diabetologists working in two different settings. There were 9 raters from institution 1 and8 from institution 2. The impact of the amount and type of information on judgment reliability was evaluated by developing two types of profiles. The test form contained only laboratory and supervised blood or urine test data similar to that utilized in study 1. The history form contained this information as well as other descriptive data typically available to diabetologists. The 17 diabetologists rated 125 anonymous profiles on each of twoseparate occasions ∼ 1 wk apart. On one occasion they rated profiles presented on the test form. On the other occasion they rated profiles presented on the history form. As in study 1, the diabetologist raters demonstrated adequate intra- and interrater reliability. Intrarater reliability was somewhat better when rating test form profiles compared with history form profiles. Reliability was not higher within than between institutions. Ananalysis of the relative contribution of different diabetes control indices to the diabetologists' judgments indicated that HbA1 influenced raters' judgmentsat both institutions more than any other single variable. However, raters utilized other information as well, particularly the urine test and fasting blood glucose results. Use of a cholesterol measure by institution 2 raters, but not by institution 1 raters, was theonly clearly discrepant use of test form data by the diabetologists working in two different settings. A mathematical prediction equation was developed that accurately simulated the diabetologists' judgments. The robustness of this equation was tested and demonstrated on a cross-validation sample of the cases utilized in study 1.