Measuring Integrated Care – Methodological Reflections from Monitoring and Evaluation Process of the PHC Plus Pilot Program in Poland

Abstract

Introduction: Integrated care is an important strategy for increasing health system performance. Despite its growing significance, detailed evidence on the measurement properties of integrated care instruments remains vague and limited. This article aims to present the Monitoring and Evaluation Framework (M&EF) used in the pilot program of coordinated primary care in Poland. It can serve as an example of Monitoring and Evaluation (M&E) concept for the countries taking their first steps in the care integration implementation or establishing PHC reforms. This article belongs to the series of publications entitled: “Highway to hell or stairway to heaven – evaluating complex integrated care models – lessons from the Primary Healthcare Plus pilot program (PHC Plus) in Poland”. Methods: The M&EF of the PHC Plus was based on the Theory of Change within which a shift was to take place in the following 4 aspects: (1) change in health outcomes among participating patients; (2) change in health care experience among participating patients; (3) change in the fragmentation of care provided for chronic diseases among participating patients; (4) change in overall spending on health services for the patients in the PHC Plus facilities. Data for the M&EF of the PHC Plus came from two main sources: the National Health Fund (national insurer) database and the results survey questionnaires. Results: Based on the established M&EF of PHC Plus it was possible to monitor and evaluate the change in patients’ health outcomes, health experience and health literacy, fragmentation of care index, and overall spending on healthcare services offered in the pilot. Some of the analyzes planned could not be carried out due errors in data reporting systems, the lack of data of adequate quality or the pandemic. However, M&E implementation process provided many insightful information supporting broader PHC reform in Poland. Inclusive process of information and data sharing, discussions with the country health stakeholders and solid analytical background allowed for better informed policy making and scaling up the pilot. Conclusion: M&E process was put in place to help identify interventions, processes and approaches that could be scaled up and implemented at the country level. This approach resulted in broader understanding and acceptance of the proposed reforms in the PHC in Poland, following PHC Plus pilot implementation. Tools and approaches available and used to evaluate care integration in the PHC setting may not fully respond to the PHC and care integration system characteristic and country capacity. Therefore, revised M&E approach should be an integral part of the health policy interventions and further development of the PHC. Care integration specific measurement tools should be considered. Integration is a complex, multidimensional concept that requires measurement at multiple levels, including the patient level, the provider (professional) level, organizational and systems level. The M&E system of care integration requires use of multidimensional approach, complexed data systems, but also active sharing process of key findings. A phased approach is recommended to help researchers define clearly where they are in the research process and show progress on the outcomes’ achievement. Some simplification of the M&EF used in the PHC Plus is recommended to increase its sustainability without loosing key analytics.