Can concordance between actual care received and a pathway map be measured on a population level in Ontario? A pilot study

Surgical Oncology


Can concordance between actual care received and a pathway map be measured on a population level in Ontario? A pilot study


K. Forster, MD, PhD*, K. Tsang, MHSc, BASc*, S. Li, BSc*, L. Ieraci, MSc*, P. Murray, MSc, PhD*, K.J. Woltman, MA*, D. Chmelnitsky, MBA, BMR(PT)*, C.M.B. Holloway, MD, PhD*, E.D. Kennedy, MD, PhD*,§


doi: http://dx.doi.org/10.3747/co.27.5349


ABSTRACT

Background

Clinical pathways are associated with improved adherence to clinical guidelines; however, most studies have evaluated pathways for a single intervention at a single institution. The objective of the present study was to develop and evaluate a method of measuring concordance with a population-based clinical pathway map to determine if that method could be feasible for assessing overall health system performance.

Methods

Patients with stage ii or iii colon cancer diagnosed in 2010 were identified, and clinical data were obtained through linkages to administrative databases. Pathway concordance was defined a priori based on receipt of key elements of the Ontario Health (Cancer Care Ontario) colorectal pathway maps. For stages ii and iii colon cancer alike, concordance was reported as the proportion of patients receiving care that followed the predefined key elements of the pathway map. Regression analysis was used to identify predictors of concordant care.

Results

Our study identified 816 patients with stage ii and 800 patients with stage iii colon cancer. Of the patients with stage ii disease, 70% (n = 571) received concordant care. Of the patients with stage iii disease, results showed high concordance for all key elements except receipt of chemotherapy, leading to an overall concordance rate of 39% for that cohort.

Conclusions

Our method of measuring concordance was feasible on a population-based level, but future studies to validate it and to develop more sophisticated methods to measure concordance in larger cohorts and various disease sites are necessary. Measurement of clinical pathway concordance on a population-based level has the potential to be a useful tool for assessing system performance.

KEYWORDS: Clinical pathways, concordance, quality improvement, system performance, colon cancer

INTRODUCTION

Clinical pathways are structured multidisciplinary care plans that outline key actions in the care of patients with a specific health condition1. Increasingly, pathways have been used in clinical practice because they have been shown to improve the quality of care and to promote desirable clinical outcomes. Based on the results of twenty-seven studies that compared standardized clinical pathways with usual care, a Cochrane systematic review found that reduced in-hospital complications and improved documentation can result when clinical pathways are used2. Clinical pathways have also been associated with adherence to clinical practice guidelines, reduction in repetition of diagnostic tests, increased patient satisfaction, and survival3,4. To date, however, most studies have evaluated pathways for a single intervention at a single institution, and no standardized methods have been developed to measure the effectiveness of clinical pathways5.

More recently, Ontario Health (Cancer Care Ontario) [oh(cco)], the government agency responsible for cancer care in the province of Ontario, developed clinical pathway maps for 11 cancers, including breast, prostate, lung, and colorectal cancer (https://www.cancercareontario.ca/en/pathway-maps). All of the oh(cco) pathway maps apply to the entire province rather than to a single institution and include the continuum of care from diagnosis to follow-up rather than a single intervention6. Those attributes of the oh(cco) pathway maps provide a unique opportunity to explore strategies to measure concordance of actual care received with care recommended by the pathway map. The resulting concordance measure could be used to assess overall system performance and to facilitate quality improvement by identifying unwarranted variation so that targeted quality improvement strategies could be implemented.

Currently, no method to measure concordance along a population-based clinical pathway has been established; thus, the overall objective of the present study was to develop and evaluate a method of measuring concordance along population-based clinical pathways, providing proof-of-concept that such an approach to assessing overall health system performance would be feasible.

METHODS

Study Design and Population

For the purposes of the present study, we used a pre-established cohort of colon cancer patients with sufficient duration of follow-up to assess overall survival. We selected that cohort because it was of sufficient sample size and because survival rates and mean survival times allowed for a meaningful comparison of survivors and non-survivors within the timeframe of the study. We specifically included only stage ii and stage iii colon cancer because the pathway maps for those two groups were felt to be the most clearly specified. The original cohort included all patients from the Ontario Cancer Registry who were diagnosed with colorectal cancer (International Classification of Diseases, 10th revision, codes C18–C21) between 1 January 2010 and 31 December 2010. The Ontario Cancer Registry is a patient-specific population-based cancer registry that includes staging and pathology data for all newly diagnosed cases7. Patients were excluded if they did not have a valid health card number, had non-diagnostic pathology, had more than 1 primary tumour, or resided outside of Ontario.

Using health card numbers, the final cohort was linked with 5 health utilization databases to obtain patient-level clinical data, including screening with fecal occult blood testing (fobt), diagnostic imaging, surgical treatment, adjuvant chemotherapy, emergency room visits, re-admission to hospital, outpatient clinic visits, and death.

The ohip (Ontario Health Insurance Plan) administrative database provides information about physician services provided on a fee-for-service basis. The ohip database was used to abstract data for screening by fobt, colonoscopy, abdominal computed tomography (ct), and consultation with a medical oncologist. Screening by fobt was defined according to a previously validated algorithm that categorizes patients into 4 groups based on exposure to fobt during the 5-year period before diagnosis8:

  • ■ None (no record of fobt)

  • ■ Pre-diagnostic (only 1 fobt performed within 9 months of diagnosis)

  • ■ Repeated (2 or more fobts, with the most recent fobt being performed within 9 months of diagnosis, and a 2nd being performed 12–24 months before the most recent fobt)

  • ■ Sporadic (all other cases)

The National Ambulatory Care Reporting System is a national database that collects administrative, clinical, and service-specific ambulatory and outpatient data. The National Ambulatory Care Reporting System was used to abstract data for chemotherapy treatment and emergency room visits.

The Discharge Abstract Database is a national database containing information about admissions to acute-care institutions. It was used to abstract data for details pertaining to surgical treatment and the Charlson comorbidity index9.

The Postal Code Conversion File is a database created by Statistics Canada that assigns a range of socioeconomic variables to standard geographic areas based on postal code10. All socioeconomic variables—including urban–rural status, immigration tertile, and neighbourhood median income quintile—were obtained from that database.

The Registered Persons Database maintained by the Ontario Ministry of Health and Long-Term Care is a population based registry of all deaths in the province. The Registered Persons Database was used to abstract data for patient deaths and to calculate survival rates for the study.

Pathway Concordance

A priori, the elements of the oh(cco) pathway maps most critical to optimizing treatment and survival in stages ii and iii colorectal cancer were established in consultation with clinical experts. Those elements included pretreatment endoscopy, pretreatment abdominal ct, surgery, medical oncology consultation, and receipt of chemotherapy. For the study, only patients undergoing a bowel resection were included in the analysis. Patients undergoing a nonresection surgery (such as diverting ileostomy, colostomy, or exploratory laparotomy) were excluded.

For each of the key elements, specific time intervals and a maximum permitted number of encounters were assigned based on expert consensus. Pretreatment endoscopy and abdominal ct were defined as concordant events if those elements occurred within 180 days before the diagnosis date and the date of resection, or within 60 days after the date of diagnosis, whichever came first. Resection was defined as an event if that element occurred within 180 days from the date of diagnosis and before the date of the first chemotherapy visit (when applicable). To allow for repeat testing that might be considered discordant, the maximum number of encounters permitted was 3 for pretreatment endoscopy, 2 for abdominal ct, and 1 for resection. Consultation with a medical oncologist was defined as a concordant event if it occurred within 270 days from the date of diagnosis and before the date of the first chemotherapy visit, when applicable. Chemotherapy treatment was defined as a concordant event if it occurred after the date of surgery and within 180 days of the date of diagnosis.

Using those 5 key elements, 4 groups of possible sequences were established. Sequence A included all 5 elements: endoscopy, abdominal ct, surgery, medical oncologist consultation, and chemotherapy. Sequence B included 4 elements: endoscopy, abdominal ct, surgery, and medical oncologist consultation. Sequence C included 3 elements: endoscopy, abdominal ct, and surgery. Sequence D included any 2 elements or fewer.

Patients with stage ii colon cancer who followed sequences A, B, or C were considered concordant with the pathway map because consultation with a medical oncologist and chemotherapy treatment were considered optional. Patients with stage iii colon cancer who followed sequence A were considered concordant because chemotherapy is considered mandatory.

Statistical Analysis

Descriptive statistics are used to report demographics and health care use. For stage ii colon cancer, concordance was reported as the proportion of patients assigned to any of sequences A, B, and C. For stage iii colon cancer, concordance was reported as the proportion of patients assigned to sequence A.

Logistic regression was used to identify predictors of concordance for both stage ii and stage iii colon cancer. The independent variables were substage at diagnosis (that is, stages iia, iib, or iic, and iiia, iiib, or iiic), age group, sex, score on the Charlson comorbidity index, screening group, type of surgery (elective vs. urgent), emergency room visits in the 30 days before the date of diagnosis, length of stay after surgery, urban–rural status, immigration tertile, and neighborhood median income quintile. The Wald test was used to evaluate the significance of the odds ratios of the covariates11. The C statistic was used to measure the goodness of fit of the logistic regression models for patients with stages ii and iii disease. For the study, a C statistic of 0.5 was considered to indicate a poor model; 0.7–0.8, to indicate a good model; and more than 0.8, to indicate a strong model11.

Given that no “gold standard” for measuring concordance along a pathway has been established, our group hypothesized, a priori, that the level of concordance between stages ii and iii colon cancer would differ, because those pathways have different key elements and because concordant care and survival would be associated. For the present analysis, Kaplan–Meier survival curves were used to predict unadjusted survival outcomes, and the log-rank test was used to compare survival distributions for the concordant and non-concordant groups of patients with stages ii and iii disease (combined and separately)12.

Statistical analyses were conducted using the SAS software application (version 9.4: SAS Institute, Cary, NC, U.S.A.).

RESULTS

Study Cohort

Initially, 5671 patients with colon cancer in the Ontario Cancer Registry were identified. Of those 5671 patients, 1616 with stage ii or iii colon cancer who underwent resection were identified and included in the study (Figure 1). The cohort was fairly evenly divided between those diagnosed with stage ii (n = 816) and stage iii (n = 800) disease.

 


 

FIGURE 1 Flowchart for the final study cohort. OCR = Ontario Cancer Registry.

Table I reports patient characteristics. Overall, mean age at diagnosis in this cohort was 70 years, and the distribution of male and female patients was relatively even. Most patients lived in urban areas and, based on the Charlson comorbidity index, had few comorbidities. Most also lived in areas with low concentrations of immigrants (foreign-born individuals). Median income quintiles were relatively evenly distributed. The overall 4-year survival for the cohort since diagnosis was 72.5% (stage ii: 79.2%; stage iii: 65.6%).

TABLE I Cohort characteristics

 

Health Services Use

Table II reports the results for services use by patients. During the diagnostic phase, most patients underwent pretreatment colonoscopy or flexible sigmoidoscopy (79.3%) and abdominal ct (95.1%). During the treatment phase, almost all patients (99.4%) underwent resection within 180 days of the date of diagnosis. Approximately two thirds underwent elective resection; the remaining third required urgent surgery. Median time from date of diagnosis to date of surgery was 17 days, and the median length of stay for surgery was 7 days. Of the patients with stage iii colon cancer, 92.4% had a consultation with a medical oncologist, and 53.5% received chemotherapy at a median of 56.5 days from the date of surgery.

TABLE II Health service utilization

 

Overall, 41.1% of the patients had at least 1 emergency room visit in the 30 days preceding the date of diagnosis, and 55.0% had at least 1 emergency room visit within 180 days after the date of diagnosis.

Pathway Concordance

Of the patients with stage ii colon cancer, 70% (n = 571) followed sequence A, B, or C and received care concordant with the oh(cco) pathway map (Table III). Multivariable regression analysis showed that patients staged iia, under-going elective surgery, and having a higher immigration tertile and income quintile were significantly more likely to receive care concordant with the oh(cco) pathway map (C statistic: 0.733; Table IV).

TABLE III Concordance by sequence and stage

 

TABLE IV Multivariate regression results for receipt of pathway-concordant carea

 

Of the patients with stage iii colon cancer, 39% (n = 312) followed sequence A and received care concordant with the oh(cco) pathway map (Table III). Multivariable regression analysis showed that patients of younger age, with a lower score on the Charlson comorbidity index, receiving elective surgery, and having a shorter length of stay were significantly more likely to receive care concordant with the oh(cco) pathway map (C statistic: 0.763; Table IV).

Overall Survival

The 4-year overall survival was 79.2% (n = 646) for patients with stage ii disease and 65.6% (n = 525) for those with stage iii disease. For patients with stage iii colon cancer, 4-year overall survival was significantly better when they received concordant care than when they did not receive concordant care (82% for sequence A vs. 60%, 51%, and 52% for sequences B, C, and D respectively; p < 0.0001). For patients with stage ii colon cancer, a trend toward improved 4-year overall survival was observed when they received concordant care; however, that trend was not statistically significant (88%, 81%, and 77% for sequences A, B, and C respectively vs. 76% for sequence D; p = 0.228).

DISCUSSION

We developed a method for measuring concordance between actual care received and the care prescribed in the oh(cco) pathway map. The overall results showed that, based on incident cases from 2010, concordant care was provided in 70% of stage ii and 39% of stage iii colon cancer cases in the province of Ontario. Those results also supported our a priori hypotheses, given that we found different levels of concordance for stages ii and iii colon cancer and that concordant care was associated with improved survival. Those results have therefore established “proof of concept” that our method of measuring concordance along a population-based pathway is feasible. Although more work will be necessary to validate the method, it might, in future, be a useful strategy for assessing overall health system performance and assisting with quality improvement by identifying unwarranted variations in care and implementing targeted strategies to reduce those variations.

For stage iii colon cancer, our results showed high concordance with all of the key elements, except for receipt of chemotherapy—an omission that led to an overall concordance rate of 39% for that patient cohort. That finding is not unique; a systematic review by Etzioni et al.13 reported rates of chemotherapy use ranging from 39% to 71% in twenty-two studies of patients with stage iii colon cancer. Our findings that more than 90% of patients had a medical oncology consultation is important, because one of the major limitations of our study is that we were unable to capture patients receiving oral chemotherapy alone. Considering that oral chemotherapy alone is most often reserved for older patients and that, of our stage iii cohort members, approximately 40% (305 of 800 patients) were 75 years or older, it is quite likely that our inability to capture oral chemotherapy alone led to an underestimate of the rate of chemotherapy use and thus the overall concordance rate in the cohort. Difficulty in capturing those patients might also in part explain why increasing age and comorbidity were more likely to be associated with non-concordant care. Access to oral chemotherapy data will therefore be important for monitoring the effectiveness of any future quality improvement initiatives aimed at increasing chemotherapy uptake. However, given that other studies have also shown variation in chemotherapy use in stage iii colon cancer, further studies of physician and patient factors driving the decision for chemotherapy might also be warranted to better understand those practice variations.

Another limitation of the present study is that our measure of concordance was relatively crude. It included only 5 key elements of care, none of which were weighted with respect to their relative association with survival. Currently, our group is developing and evaluating more sophisticated measures of concordance that will incorporate more elements of the pathway map and consider their relative association with survival, and that will also consider other important clinical and patient-reported outcomes. Lastly, our results are based on a limited set of incident patients with colon cancer in 2010 and will therefore have to be further validated with a larger cohort of patients and with other disease sites before our proposed method can be adopted into full-scale use for quality improvement across the province.

CONCLUSIONS

The overall results of our study showed that, in 70% of stage ii and 39% of stage iii colon cancer cases, patients received concordant care in the province of Ontario. The rate of concordance in the stage iii cohort was likely underestimated because of an inability to capture receipt of oral chemotherapy alone. Our method of measuring concordance seems feasible on a population level, but future studies to validate the method and to develop more sophisticated ways of measuring concordance in larger cohorts and other disease sites will be necessary before it can be adopted into full-scale use. From a health system perspective, measurement of clinical pathway concordance on a population level has the potential to be a useful tool for assessing system performance by identifying unwarranted clinical variation.

ACKNOWLEDGMENTS

This work was supported by oh(cco). The opinions, results, views, and conclusions reported in this paper are those of the authors and of oh(cco).

CONFLICT OF INTEREST DISCLOSURES

We have read and understood Current Oncology’s policy on disclosing conflicts of interest, and we declare that we have none.

AUTHOR AFFILIATIONS

*Ontario Health (Cancer Care Ontario),
Department of Surgery, Sunnybrook Health Sciences Centre,
Department of Surgery, Mount Sinai Hospital,
§Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, ON.

REFERENCES

1 Kinsman L, Rotter T, James E, Snow P, Willis J. What is a clinical pathway? Development of a definition to inform the debate. BMC Med 2010;8:31.
cross-ref  pubmed  pmc  

2 Rotter T, Kinsman L, James E, et al. Clinical pathways: effects on professional practice, patient outcomes, length of stay and hospital costs. Cochrane Database Syst Rev 2010;:CD006632.
pubmed  

3 Martens L, Goode G, Wold JF, et al. Structured syncope care pathways based on Lean Six Sigma methodology optimises resource use with shorter time to diagnosis and increased diagnostic yield. PLoS One 2014;9:e100208.
cross-ref  pubmed  pmc  

4 van Dam PA, Verheyden G, Sugihara A, et al. A dynamic clinical pathway for the treatment of patients with early breast cancer is a tool for better cancer care: implementation and prospective analysis between 2002–2010. World J Surg Oncol 2013;11:70.
cross-ref  

5 Rotter T, Kinsman L, James E, Machotta A, Steyerberg EW. The quality of the evidence base for clinical pathway effectiveness: room for improvement in the design of evaluation trials. BMC Med Res Methodol 2012;12:80.
cross-ref  pubmed  pmc  

6 Ontario Health (Cancer Care Ontario) [oh(cco)]. Pathway Map Development Methodology. Toronto, ON: oh(cco); 2017.

7 Hall S, Schulze K, Groome P, Mackillop W, Holowaty E. Using cancer registry data for survival studies: the example of the Ontario Cancer Registry. J Clin Epidemiol 2006;59:67–76.
cross-ref  

8 James PD, Rabeneck L, Yun L, et al. Repeated faecal occult blood testing is associated with decreased advanced colorectal cancer risk: a population-based study. J Med Screen 2018;25:141–8.
cross-ref  

9 Quan H, Li B, Couris CM, et al. Updating and validating the Charlson comorbidity index and score for risk adjustment in hospital discharge abstracts using data from 6 countries. Am J Epidemiol 2011;173:676–82.
cross-ref  pubmed  

10 Statistics Canada. Postal Code OM Conversion File (PCCF), Reference Guide. Ottawa, ON: Government of Canada: 2017.

11 Hosmer DW, Lemeshow S. Applied Logistic Regression. 2nd ed. New York, NY: John Wiley and Sons; 2000.
cross-ref  

12 Rosner B. Fundamentals of Biostatistics. 4th ed. Belmont, CA: Cengage Learning; 1994.

13 Etzioni DA, El-Khoueiry AB, Beart RW Jr. Rates and predictors of chemotherapy use for stage iii colon cancer: a systematic review. Cancer 2008;113:3279–89.
cross-ref  pubmed  


Correspondence to: Erin Kennedy, Mount Sinai Hospital, 600 University Avenue, Room 449, Toronto, Ontario M5G 1X5. E-mail: erin.kennedy@sinaihealthsystem.ca

(Return to Top)



Current Oncology, VOLUME 27, NUMBER 1, February 2020








Copyright © 2019 Multimed Inc.
ISSN: 1198-0052 (Print) ISSN: 1718-7729 (Online)