What is a clinically meaningful survival benefit in refractory metastatic colorectal cancer?

Ractice Guideline

What is a clinically meaningful survival benefit in refractory metastatic colorectal cancer?

Y.J. Ko, MD*, M. Abdelsalam, MD, P. Kavan, MD, H. Lim, MD§, P.A. Tang, MD||, M. Vincent, MD#, R. Wong**, M. Kish††, S. Gill, MD‡‡

doi: http://dx.doi.org/10.3747/co.26.4753


Assessment of the clinical benefit of cancer treatments can be highly subjective, influenced by both perspective and context. Such assessments are required in regulatory and policy decision-making, but consistency between jurisdictions is often lacking. Clear and consistent standards for determining when a treatment offers a meaningful benefit, relative to the current standard of care, can help to address issues of equity and transparency in health technology assessment.

For metastatic colorectal cancer (mcrc), no standardized Canadian definition of clinically meaningful benefit has yet been proposed. Colorectal Cancer Canada therefore convened a group of medical oncologists expert in colorectal cancer to review the literature about clinical significance. The resulting consensus is intended to apply to any therapeutic agent being considered in the setting of chemotherapy-refractory mcrc.

It was agreed that overall survival is the appropriate measure of clinical efficacy in chemorefractory mcrc. As quantitative targets for efficacy, an improvement of 2 months or more in median overall survival or a hazard ratio for survival of 0.75 or lower (or both) are proposed as the threshold for clinically meaningful benefit. That threshold could be influenced by a treatment’s effect on quality of life. Treatment toxicity is also relevant to the assessment of clinical benefit in this setting, specifically when significant differences in treatment tolerability are evident.

KEYWORDS: Colorectal cancer, metastatic, treatment-refractory disease, clinical significance, quality of life, patient functioning, treatment benefit, toxicity, tolerability


Clinical significance refers to the practical or applied value ... of an intervention—whether [it] makes a real (e.g., genuine, palpable, practical, noticeable) difference in everyday life.

—A.E. Kazdin, 19991

Clinical significance differs from statistical significance in that no standard operational definition exists for determining when clinical significance has been achieved. Whether the clinical effect of a treatment differs trivially or substantially from that of the current standard of care is often a matter of judgment, influenced both by the context of the decision and by the perspective of the viewer.

Figure 1 identifies three key stakeholder groups who might have differing perspectives when evaluating the therapeutic benefit of a cancer treatment—namely, patients with cancer, together with their caregivers and the patient organizations that represent them; regulators, including payers and health technology assessment (hta) groups; and treating clinicians, particularly medical oncologists. Of those three groups, patients and regulators almost never interact, but physicians interact frequently and directly with patients, and occasionally and often indirectly with regulators. Thus, the treating clinician serves as a bridge between the organizations and individuals who determine policy about treatment access and the patients whose lives are affected by such decisions.



FIGURE 1 Stakeholders in the discussion of the therapeutic benefit of a cancer treatment. The position of the clinician, typically a medical oncologist, is unique in that clinicians interact both with patients and with health technology assessment bodies (such as payers), two groups that rarely have direct access to one another.

In Canada, access to and funding for cancer treatments are determined on a province-by-province basis, with separate systems in place for Quebec and for the rest of Canada. Discordant judgments in various jurisdictions raise issues of equity and access and might undermine public confidence in the decisions being made2. The development of quantitative, or at least objective, criteria for assessing meaningful benefit in various cancer indications would be a valuable step toward greater consistency and transparency in hta policy.

For physicians, judgments about therapeutic benefit arise daily in cancer care, such as when they confer with their patients about the value or futility of continuing treatment or when they advocate for access to licensed or experimental treatments. Such judgments can arise in other circumstances as well. Physician organizations consider treatment benefit when they advise on treatment guidelines or the choice of meaningful and appropriate study endpoints—a crucial methodology issue for clinical trials comparing experimental cancer drugs against the current standard of care. For instance, a working group from the U.S. National Cancer Institute’s Breast Cancer Steering Committee recently offered recommendations for choosing overall survival (os) or progression-free survival (pfs) for clinical trials3. That evaluation was based on current life expectancy and other clinical factors facing patients with various subtypes of breast cancer. Similar guidance for colorectal cancer was developed previously by Canadian specialists4.

Chemorefractory metastatic colorectal cancer (mcrc) represents a challenging clinical setting in which patients have exhausted currently available therapies. Within the past 5 years, Health Canada has approved two agents for chemorefractory mcrc—regorafenib and trif luridine/tipiracil—based on phase iii evidence demonstrating a statistically significant improvement in os5,6. However, neither drug has been recommended for funding by the pan-Canadian Oncology Drug Review, possibly reflecting uncertainty about the criteria for evaluating clinically meaningful benefit.

Recognizing the perspective of clinicians in such evaluations, Colorectal Cancer Canada (ccc) recently convened a meeting of gastrointestinal medical oncologists. The impetus for the meeting was the recognition of an unmet need for new therapies for chemorefractory mcrc. The group was tasked with developing clear and consistent criteria for assessing the clinical benefit of any drug under consideration for use in chemorefractory mcrc.


Seven medical oncologists, representing 5 Canadian provinces, attended a consensus meeting in Toronto on 7 September 2018. The attendees determined that an 85% majority (6 of 7) would suffice for acceptance of a consensus statement. Two ccc members (Filomena Servidio-Italiano and Barry D. Stein) and one trainee (MK) were present as nonvoting participants. Voting was conducted anonymously, using electronic keypads. It was agreed that any dissenting opinions could be noted in this report. However, all decisions were made with unanimous support.

Chemorefractory mcrc was defined as the clinical setting in which patients with advanced disease have either progressed on or demonstrated intolerance to fluoropyrimidine, oxaliplatin, irinotecan, bevacizumab, and if wild-type for KRAS, an anti–epidermal growth factor receptor therapy. Based on findings from a pre-meeting electronic survey of members of the Canadian Association of Medical Oncology and of the invited participants, preliminary consensus statements had been drafted by the co-chairs. At the meeting, those draft statements were presented to the group for discussion, input, and revisions. A final vote was then undertaken for each consensus statement.


Current Literature About Clinically Meaningful Benefit

The consensus group reviewed the published literature concerning quantitative standards for assessing clinically meaningful benefit in oncology. Two landmark resources were considered, one published by the American Society of Clinical Oncology (asco)7 and one by the European Society for Medical Oncology (esmo)8. The latter organization developed the Magnitude of Clinical Benefit Scale (mcbs), a validated tool for rating the benefit of a cancer treatment based on data from randomized controlled trials8. The asco and esmo systems are both expert consensus–based and both emphasize the use of os as the primary measure for assessing benefit in various oncology indications.

The asco consensus, published in 2014, proposed that, as a quantitative standard to be applied in various cancer indications, a relative improvement in median os of at least 20% is needed to establish meaningful benefit. However, they established some flexibility in that assessment, noting that if a therapy is less toxic than prevailing treatments, a smaller degree of improvement in efficacy might suffice, whereas a therapy with substantial toxicity might require a greater degree of survival advantage to be considered meaningfully beneficial7.

The asco expert group for colorectal cancer recommended that, relative to a 4- to 6-month expected survival time in advanced colon cancer, a 3- to 5-month absolute improvement in median survival, associated with a hazard ratio (hr) for mortality of approximately 0.67 over standard-of-care treatment, would represent a meaningful improvement. Other quantitative cut-offs for meaningful benefit included a 25%–35% increase in the 1-year survival rate, or a 3- to 5-month improvement in pfs, designated as a secondary endpoint7. In discussing that work, the Canadian experts noted that the asco opinion was published in 2014 and might not entirely reflect contemporary opinions.

The esmo mcbs approach8, published in 2017, is more complex than that taken by the asco group7, in that it considers a variety of factors beyond measures of survival in determining therapeutic benefit. Those considerations include improved quality of life (qol) and reduced treatment toxicity.

The mcbs tool assigns grades from 1 point (trivial or no benefit) to 5 points (substantial treatment benefit) for various criteria relating to treatments with curative or noncurative intent, to indications with a shorter or longer life expectancy, and to studies with os or pfs as the primary endpoint8. Some, but not all, of the esmo mcbs scoring metrics include questions about qol and treatment toxicity. Thus, for potentially curative treatments, toxicity or qol are not considered; but for treatments that are not likely to be curative, an initial score based on os or pfs can be adjusted by considering toxicity and qol. The esmo tool also recognizes a class of noninferiority randomized controlled trials with endpoints other than os or pfs, in which no survival advantage is expected. In evaluating such studies, neither os nor pfs is scored. Rather, response rate, symptomatic improvement, reduced treatment toxicity, and improved qol can be used to assess clinical benefit8.

The esmo method of grading for survival benefit relies on both hr for mortality and an absolute improvement in os or pfs. For example, a study of mcrc reporting os as a primary endpoint would be judged to have the highest level of clinical significance if it showed an os gain of 3 months or more, with a hr of 0.65 or better, or if it showed a 10% or greater increase in 2-year survival. Evaluation of toxicity is restricted to grades 3 and 4 events with an impact on daily well-being. Thus, the mcbs specifies that chronic nausea, diarrhea, or fatigue should be considered, but that myelosuppression (a laboratory finding) and alopecia (an event with less daily impact) are excluded from the assessment8.

In esmo’s test application of the mcbs, four randomized controlled trials examining cetuximab, panitumumab, trifluridine/tipiracil, and regorafenib in later-stage mcrc were included. For three of those agents, the mcbs scores were 1 or 2 relative to placebo or best supportive care. However, the cetuximab study scored 4 for clinical benefit, suggesting a meaningful benefit for that agent in the target population—namely, individuals with refractory mcrc and wild-type KRAS8.

The Patient Perspective on mCRC

The consensus group examined the results of an October 2017 mcrc patient and caregiver survey related to the patient experience of mcrc and its various treatments. As part of a patient group’s hta submission for trifluridine/tipiracil, ccc surveyed 80 Americans, Canadians, and Europeans, including 64 patients with mcrc and 16 caregivers. Additionally, 9 patients and 2 caregivers participated in a structured follow-up interview.

Approximately half the respondents (49%) agreed that their needs were not met by available mcrc medications (ccc. Data on file). Consistent with the perspective of the asco and esmo authors, survey respondents judged extension of os and pfs to be a potentially valuable feature of a medication, and they also agreed that qol improvement while on therapy could, in itself, represent a significant benefit. Thus, 84% indicated that they would want access to a drug that offered them minimal os benefit but that improved qol for the duration of treatment.

Conversely, potential side effects were acceptable to the surveyed respondents, provided that the treatment brought the prospect of longer survival. A substantial proportion of the respondents (72%) agreed that they would endure “significant toxicities” to extend their life by 1 year. Many patients also valued os improvement of a smaller magnitude: 51% and 47% of respondents agreed that they would choose to use a drug that caused significant side effects to achieve an os benefit of 6 and 2 months respectively. That response, from individuals with lived experience of mcrc, is in reasonable agreement with the esmo expert group8, who deemed an os benefit of 3 or more months to be clinically significant.

In their response to open-ended questions, interviewees expressed enthusiasm for treatments that might be “kinder” than traditional regimens. One described a treatment as a “win” if it offered stable disease control over a period of months, combined with good qol and minimal fatigue. The patients and caregivers uniformly supported greater access to and financial support for a treatment if it was well tolerated and offered any extended survival time and good tolerability. One noted that “any chance to prolong a patient’s life with ... mild side effects should be allowed.”

Development of Consensus Statements

The consensus group reviewed the principles applied by the asco and esmo groups and then worked to articulate a Canadian approach to the assessment of clinically meaningful benefit, specifically for patients with chemorefractory mcrc.

The consensus group considered and rejected defining clinical benefit based on a threshold value for improved median survival. Instead, they favoured a composite criterion that considers both os increase and hr. Unlike the single time point described by median survival, the hr reflects the difference in survival spanning the full duration of a comparative study. The proposed composite measure is similar to the principle set out by the esmo group, which used a combined threshold of hr and minimal absolute improvement in median os (or pfs)8.

With respect to treatment toxicity, the group agreed that adverse events are of concern primarily if they affect treatment tolerability or if they result in morbidity. Laboratory abnormalities that are unlikely to lead to symptoms are not meaningful when weighing a treatment’s clinical benefit. Thus, although neutropenia or other measures of myelosuppression might be excluded from the assessment, febrile neutropenia would be a consideration. That policy echoes the esmo approach, as described earlier8.

The consensus group also considered the role of patient-reported qol in the evaluation of meaningful benefit, agreeing with the esmo group’s inclusion of qol data when assessing the clinical significance of clinical trial data in mcrc. In addition, the group argued for an inclusive standard in which other classes of findings can be considered if they speak to the patient’s experience while on therapy. Thus, the group suggested that measures of patient performance status would also be relevant alongside qol tools such as the widely used and validated 36-item Short Form Health Survey (RAND Corporation, Santa Monica, CA, U.S.A.) metric9. The group noted that performance status is observed to correlate with qol measures in cancer patients. For instance, one study of elderly patients receiving treatment for solid tumours of various types found that a symptom cluster including pain, fatigue, insomnia, and mood disturbance accounted for much of the variance in both function and qol10.

One metric of patient performance status, the Eastern Cooperative Oncology Group (ecog) scale11, is a clinically relevant measure of patient function that physicians use to determine appropriateness of treatment or to monitor a patient’s response to therapy. Maintenance of ecog status might preserve options for subsequent treatment, and so that attribute could represent an important benefit of a noncurative treatment for patients who require time to access additional lines of treatment.

The quality-adjusted time without symptoms of disease or toxicity (q-twist) method is another metric related to the patient experience while on treatment. As with maintenance of ecog status, q-twist measures the time during which a patient remains in a relatively desirable state—in this case, free of symptoms and disease progression, and simultaneously, free of grade 3 adverse events. Thus, q-twist assesses both quantity of life and qol in a single measurement12,13.

The consensus group therefore proposed that measures such as ecog, maintenance of ecog status, and q-twist be used alongside or as alternatives to patient-reported qol metrics when assessing the clinical benefit of treatment.


Table I sets out the 5 agreed-upon consensus statements. All were approved unanimously by the consensus group. All statements pertain to chemorefractory advanced mcrc.

TABLE I Consensus statements


An important aim in the future is to incorporate values considerations (clinical and economic evaluations, and social value judgments, including considerations of efficiency and effectiveness)14 into the assessment of meaningful clinical benefit.

Recognizing the need for the patient perspective, ccc is leading a Patient Values Project. The project is designed to better define, measure, and incorporate the preferences of patients with colorectal cancer into the hta framework for cancer drug approval. Outcomes of a national qol survey and discrete-choice experiment will be used to determine what patients and caregivers consider to be of value in the drug treatment of colorectal cancer. Those survey data will be used in the development of key metrics to allow for a more objective and research-based evaluation of patient-group submissions, with the goal of consistent assessment of patient input into the drug evaluation process.

By better defining and measuring patient preferences and by incorporating those preferences into the evaluative framework for the hta drug approval process, cancer patient groups will be able to provide clear, objective, and research-based input that will assist expert committees in the evaluation of their input. Thus, the Patient Values Project will allow for a more reasoned and balanced rationale in the assessment of new cancer drugs by the expert committees overseeing those funding recommendations. The resulting framework can be applied to other cancer disease sites within Canada and elsewhere.

In summary, chemorefractory mcrc remains an area of unmet therapeutic need. It is hoped that the statements developed by the consensus group will provide a benchmark for the adoption of new therapies for Canadian patients, based on a defined threshold of clinically meaningful benefit observed in clinical trials.


The authors thank Filomena Servidio-Italiano and Barry D. Stein (Colorectal Cancer Canada) for organizing the consensus meeting, and they gratefully acknowledge the support of ccc in providing a medical writer, John Ashkenas phd (imc North America, Toronto, ON), to help develop this article. The authors also thank Dr. Ronald Burkes (Mount Sinai Hospital, Toronto, ON) for his review and endorsement of the consensus guideline.


We have read and understood Current Oncology’s policy on disclosing conflicts of interest, and we declare the following interests: YJK has received honoraria for advisory or speaking roles from Taiho, Amgen, Servier, and Eli Lilly; MA has received honoraria for advisory or speaking roles from AstraZeneca, Boehringer Ingelheim, Bristol–Myers Squibb, Celgene, Ipsen, Janssen, Johnson and Johnson, Merck, NB Urology, Novartis, Pfizer, Roche, Sanofi, Shire, Taiho, and Yewtree Consulting; he is a co-investigator or principal investigator on industry trials sponsored by AbbVie, Amgen, Aragon Pharmaceuticals, Astex Pharmaceuticals, Astra-Zeneca, Bayer, Boehringer Ingelheim, Bristol–Myers Squibb, the Canadian Cancer Trials Group, Chiltern/Pfizer, Eisai, Exactis Innovation, GlaxoSmithKline, Merck and Co., Merck KGaA, Novartis, Odonate Therapeutics, Pharmacyclics, Picomole, Puma Biotechnology, Purdue Pharma, the Quebec Clinical Research Organization in Cancer, Roche, Sanofi Aventis, Takeda, Turnstone Biologics, and United Therapeutics; PK has received honoraria for advisory roles from Amgen, Taiho, and Merck; he has also received educational grants from Amgen, Bayer, Merck, Roche, and Taiho; HL has received honoraria for advisory roles from Roche, Amgen, Eisai, Taiho, Lilly, and Bristol–Myers Squibb; he has also received travel support from Eisai and is a co-investigator or principal investigator on industry trials sponsored by Bristol–Myers Squibb, AstraZeneca, Amgen, Roche, and Astellas; PAT has received honoraria for advisory or speaking roles from Novartis, Genomic Health International, Amgen, Merck, Taiho, and AstraZeneca; MV has received honoraria for advisory or speaking roles from Taiho, Merck, Bristol–Myers Squibb, Boehringer Ingelheim, Eli Lilly, Sanofi Aventis, Amgen, Novartis, Pfizer, Roche, Amgen, and AstraZeneca; SG has received honoraria for advisory or speaking roles from Amgen, Roche, Taiho, and Eli Lilly. The remaining authors have no conflicts of interest to disclose.


*Sunnybrook Health Sciences Centre, University of Toronto, Toronto, ON,
Division of Medical Oncology, Department of Medicine, Dalhousie University, Halifax, NS,
Department of Oncology, McGill University, Montreal, QC,
§Division of Medical Oncology, Faculty of Medicine, University of British Columbia, Vancouver, BC,
||Department of Oncology, Tom Baker Cancer Centre, Cumming School of Medicine, University of Calgary, Calgary, AB,
#Division of Medical Oncology, London Regional Cancer Program, London, ON,
**CancerCare Manitoba, Department of Hemato-Oncology, University of Manitoba, Winnipeg, MB,
††Department of Pharmacology and Toxicology, University of Toronto, Toronto, ON,
‡‡BC Cancer, University of British Columbia, Vancouver, BC.


1 Kazdin AE. The meanings and measurement of clinical significance. J Consult Clin Psychol 1999;67:332–9.
cross-ref  pubmed  

2 Srikanthan A, Penner N, Chan KKW, Sabharwal M, Grill A. Understanding the reasons for provincial discordance in cancer drug funding-a survey of policymakers. Curr Oncol 2018;25:257–61.
cross-ref  pubmed  pmc  

3 Seidman AD, Bordeleau L, Fehrenbacher L, et al. National Cancer Institute Breast Cancer Steering Committee Working Group report on meaningful and appropriate end points for clinical trials in metastatic breast cancer. J Clin Oncol 2018;:JCO1800242.
cross-ref  pubmed  

4 Gill S, Berry S, Biagi J, et al. Progression-free survival as a primary endpoint in clinical trials of metastatic colorectal cancer. Curr Oncol 2011;18(suppl 2):S5–10.
cross-ref  pubmed  pmc  

5 Grothey A, Van Cutsem E, Sobrero A, et al. on behalf of the correct study group. Regorafenib monotherapy for previously treated metastatic colorectal cancer (correct): an international, multicentre, randomised, placebo-controlled, phase 3 trial. Lancet 2013;381:303–12.

6 Mayer RJ, Van Cutsem E, Falcone A, et al. on behalf of the recourse study group. Randomized trial of TAS-102 for refractory metastatic colorectal cancer. N Engl J Med 2015;372:1909–19.
cross-ref  pubmed  

7 Ellis LM, Bernstein DS, Voest EE, et al. American Society of Clinical Oncology perspective: raising the bar for clinical trials by defining clinically meaningful outcomes. J Clin Oncol 2014;32:1277–80.
cross-ref  pubmed  

8 Cherny NI, Sullivan R, Dafni U, et al. A standardised, generic, validated approach to stratify the magnitude of clinical benefit that can be anticipated from anti-cancer therapies: the European Society for Medical Oncology Magnitude of Clinical Benefit Scale (esmo-mcbs). Ann Oncol 2015;26:1547–73.
cross-ref  pubmed  

9 Ware JE Jr, Sherbourne CD. The mos 36-item Short-Form Health Survey (sf-36). I. Conceptual framework and item selection. Med Care 1992;30:473–83.
cross-ref  pubmed  

10 Cheng KK, Lee DT. Effects of pain, fatigue, insomnia, and mood disturbance on functional status and quality of life of elderly patients with cancer. Crit Rev Oncol Hematol 2011;78:127–37.

11 Oken MM, Creech RH, Tormey DC, et al. Toxicity and response criteria of the Eastern Cooperative Oncology Group. Am J Clin Oncol 1982;5:649–55.
cross-ref  pubmed  

12 Husson O, Jones RL. q-twist: what really matters to the cancer patient? Cancer 2017;123:2200–2.
cross-ref  pubmed  

13 Goldhirsch A, Gelber RD, Simes RJ, Glasziou P, Coates AS. Costs and benefits of adjuvant therapy in breast cancer: a quality-adjusted survival analysis. J Clin Oncol 1989;7:36–44.
cross-ref  pubmed  

14 United States, Department of Health and Human Services, Institutes of Health, Institute of Medicine. Assessing and Improving Value in Cancer Care: Workshop Summary. Washington, DC: National Academies Press; 2009. [Available online at: https://www.ncbi.nlm.nih.gov/books/NBK219551/pdf/Bookshelf_NBK219551.pdf; cited 4 September 2018]

Correspondence to: Sharlene Gill, BC Cancer, 600 West 10th Avenue, Vancouver, British Columbia V5Z 4E6. E-mail: sgill@bccancer.bc.ca

(Return to Top)

Current Oncology, VOLUME 26, NUMBER 2, April 2019

Copyright © 2019 Multimed Inc.
ISSN: 1198-0052 (Print) ISSN: 1718-7729 (Online)