Childhood Adenotonsillectomy Trial

6.10.2 Tracking of QA/QC Data

The CP/QCM will track two types of data: data from the actual scored records and data generated during QA exercises. Using actual scored data, the overall mean RDI, sleep stage values and arousal indexes will be tracked for each scorer. If average values differ by > 15% for any given scorer, those records will be reviewed by the CP/QCM. The SRC Director will determine whether re-training and re-standardization are required. Using data from the QA exercises, levels of agreement will be determined among scorers and trends tracked over time. Any scorer identified to deviate excessively (>10% from the consensus statement) on 3 consecutive exercises will be “re-trained.” “Re-training” will be considered successful if review of at least 5 additional studies demonstrates no deviation from scoring protocol, and the subsequent QA exercises show no deviations in performance compared to scoring assignments made by the other scorers.

At monthly QA meetings, the statistics summarizing inter-and intra-scorer variability will be reviewed by the SRC staff. They will identify any explanations for differences. If differences between scorers can not be explained by real differences in the studies assigned to any given scorer in any given time period, scorers noted to score differently will score together, concentrating on the areas where differences were noted. The QCM will assign a senior scorer to review consecutive records and reinstruct the scorer. Subsequent scoring will be monitored until conformity is demonstrated.