Liability and validity of the Appropriateness Evaluation Protocol in Turkey

Objective. To assess the inter-rater reliability between nurses and the convergent validity of the Appropriateness Evaluation Protocol (AEP) in the Turkish context. Methods. Two nurses applied the original AEP concurrently to a random subsample of 335 patient-days in internal medicine, general surgery, and gynaecology departments at a university hospital and a government teaching hospital, as a part of a larger study. Inter-rater reliability was tested by calculating overall agreement and specific agreements between nurse reviewers' AEP assessments. Validity was tested by comparing the assessments of the nurses based on the AEP with the implicit judgements of five expert physicians on a random subsample of 818 patient-days. Sensitivity, specificity, positive and negative predictive values of the AEP were calculated. Reliability and validity were also evaluated by the κ statistic. Results. In the reliability test, there was a high level of agreement between the two independent raters applying the AEP in the three departments studied: Overall agreement = 90.7-97.6%; specific inappropriate agreement = 69.1-92.3%; specific appropriate agreement = 88.3-96.6%. In validity testing, the AEP had a sensitivity of 0.83-0.97, specificity of 0.62-0.80, and positive and negative predictive values of 0.84-0.88 and 0.73-0.95 respectively. Kappa coefficients in internal medicine and gynaecology indicated almost perfect agreement in reliability testing and moderate agreement in validity testing. In general surgery, the κ coefficients showed substantial agreement in both tests. Conclusion. These results indicate that the AEP is a reliable and valid instrument to assess appropriateness of patent-days in Turkey.