C.L. Gauvreau, PhD*, N.R. Fitzgerald, MA*, S. Memon, MPH*, W.M. Flanagan, BM†, C. Nadeau, PhD†, K. Asakawa, PhD†, R. Garner, PhD†, A.B. Miller, MD‡, W.K. Evans, MD§, C.M. Popadiuk, MD‖, M. Wolfson, PhD#, A.J. Coldman, PhD**
The Canadian Partnership Against Cancer was created in 2007 by the federal government to accelerate cancer control across Canada. Its OncoSim microsimulation model platform, which consists of a suite of specific cancer models, was conceived as a tool to augment conventional resources for population-level policy- and decision-making. The Canadian Partnership Against Cancer manages the OncoSim program, with funding from Health Canada and model development by Statistics Canada.
Microsimulation modelling allows for the detailed capture of population heterogeneity and health and demographic history over time. Extensive data from multiple Canadian sources were used as inputs or to validate the model. OncoSim has been validated through expert consultation; assessments of face validity, internal validity, and external validity; and model fit against observed data. The platform comprises three in-depth cancer models (lung, colorectal, cervical), with another in-depth model (breast) and a generalized model (25 cancers) being in development. Unique among models of its class, OncoSim is available online for public sector use free of charge. Users can customize input values and output display, and extensive user support is provided.
OncoSim has been used to support decision-making at the national and jurisdictional levels. Although simulation studies are generally not included in hierarchies of evidence, they are integral to informing cancer control policy when clinical studies are not feasible. OncoSim can evaluate complex intervention scenarios for multiple cancers.
Canadian decision-makers thus have a powerful tool to assess the costs, benefits, cost-effectiveness, and budgetary effects of cancer control interventions when faced with difficult choices for improvements in population health and resource allocation.
KEYWORDS: OncoSim, cancer modelling, cancer outcome projections, cancer control, cancer system resource allocation
Against the backdrop of an aging population, new technologies and drugs, and constrained health care budgets, the Canadian Partnership Against Cancer (hereinafter, the Partnership) was created in 2007 with a mandate from the federal government to accelerate cancer control across Canada. From the outset, a key strategy for the Partnership was to mobilize the best evidence and deliver it into the hands of policy- and decision-makers. The OncoSim microsimulation model platforma, with its suite of specific cancer models, was conceived as a tool to augment conventional information resources for population-level Canadian decision-making, within the reality of thirteen jurisdictional health care systems. With funding from Health Canada, OncoSim is led and supported by the Partnership, with model development by Statistics Canada, to help answer complex questions facing Canada’s cancer control system.
Here, we describe the rationale, development, and application of the OncoSim platform, providing the cancer control community with foundational knowledge about a Canada-specific analytical tool for population-level decision-making.
The objectives of the OncoSim program are fourfold:
The Canadian Partnership Against Cancer initiated the OncoSim program through a request-for-proposals process. Applicants were not limited to Canada, but the process required that the models be based on Canadian data for disease burden, treatment patterns, outcomes, and costs. Furthermore, the models were intended for a multiuser environment in which users could explore scenarios and not be limited to pre-specified options. Models were to be able to plausibly project outcomes, resource use, and costs for simulated scenarios. Applicants were also asked to facilitate access for general users. Statistics Canada was the successful applicant, and their statisticians, mathematicians, economists, and programmers continue to serve on the team of model developers.
The Partnership’s health economics team manages and promotes OncoSim and provides access to the model for internal and external users. Technical advisory panels composed of external specialists and Statistics Canada and Partnership staff provide strategic advice and clinical, epidemiologic, computing, and economics expertise for model development, update, and progress review. An oversight committee provides direction to the program, with its membership consisting of a pan-Canadian group of cancer control specialists, executive Partnership staff, and program staff. These management and oversight structures are accountable to Health Canada through its funding of the Partnership.
The microsimulation modelling approach allows for the simulation of cases at the level of individuals, and captures the heterogeneity of the population’s health and demographic history over time. Some facets of the model allow for interactions between “individuals” that can affect cancer outcomes, such as infectious disease transmission and hereditary genetic profiles. A high degree of flexibility is available in the construction and use of the components.
The OncoSim suite of models is built on a common framework (Figure 1), with users being given the capability to control input parameters. Examples include human papillomavirus (hpv) vaccination; risk factors such as smoking and use of hormone replacement therapy; screening characteristics such as modalities and associated costs, test sensitivities and specificities, participant eligibility, and follow-up protocols; and treatment characteristics such as procedure types, probabilities, and costs. Users also have a limited ability to modify the natural history of a cancer, duration of health states, and underlying demographic and economic characteristics. A specific alternative population cohort can be defined by the user. Similarly, outputs can be customized to stratify results, present data in tabular or graphic form, and export results for further customization or analysis. Outputs include incidence, mortality, resource utilization (for example, follow-up procedures, treatments), direct health care costs, life expectancy, quality-adjusted life-years, incremental cost-effectiveness ratios, and macroeconomic factors (for example, employment, taxes, labour productivity), most of which can be stratified by age, sex, year, and province or territory. Details of the OncoSim approach and its user functions have been described in previous publications1–3.
FIGURE 1 Conceptual diagram of the OncoSim platform.
Inputs to the model, as well as the targets that the model must replicate, were obtained from multiple Canadian sources: data for cancer-specific rates of incidence (Canadian Cancer Registry); cancer survival (multiple sources, including provincial sources); cancer and other-cause mortality (Statistics Canada); past and projected demography (Statistics Canada); costs of procedures (multiple provincial sources); costs of cancer management (multiple provincial sources); patterns of practice [Institute for Clinical Evaluative Sciences, provincial administrative databases, data from randomized controlled clinical trials (rcts)]; health behaviours (Canadian Community Health Survey, National Population Health Survey); screening performance (pan-Canadian screening networks); measures of health-related quality of life (Statistics Canada, publications using Canadian sources); and environmental radon data (Health Canada). Certain parameter estimates derived from the published literature are adjusted to provide projections consistent with observed Canadian data.
Rigorous assessments are performed to validate OncoSim.
Experts not involved in model development are engaged in assessing components or the model in whole—for example, pan-Canadian expert reviews of treatment pathways and case-study evaluations. Those assessments ensure conformity with clinical and modelling best practice and knowledge, with the case-study evaluations generating parameter sensitivity analyses from independently specified scenarios.
Simulated individual life trajectories are inspected for plausibility, and extreme scenario analyses are conducted.
Model inputs (for example, incidence rates) are determined to be consistent with outputs (for example, incident cases), and model construction logic is verified by both internal and external experts.
Outputs are compared against data not used to build the model—for example, observed data, rcts, and results from other comparable models such as the U.S. Cancer Intervention and Surveillance Modeling Network (cisnet) models.
Each model is required to demonstrate acceptable close fit to targets derived from Canadian sources (for example, incidence and staging in the Canadian Cancer Registry) and published sources. Typically, several thousand estimations are made to meet hundreds of targeted outcomes in individual subcomponents such as screening, natural history, and cancer incidence and mortality, and then again to meet key prioritized targets in whole-model calibration.
Additionally, peer-reviewed publication of model results, including comparisons with other modelling results, is encouraged from both internal and external users2–10. As newer data emerge, the model is used to compare past projections to current real-world data, thus continually assessing its ability to accurately project future events. Recalibration is undertaken as new inputs or new functionalities are added.
The OncoSim platform currently comprises in-depth models for lung, colorectal, and cervicalb cancer. An in-depth breast cancer model and a generalized cancers model (the 25 most common cancers) are under development.
Unique among models of its class, OncoSim is available for public sector use without charge. It is accessible online, freeing users from purchase and maintenance of computing resources. Extensive user support, tailored to the technical expertise of the user, is provided by the Partnership and by Statistics Canada. Users can run their own simulations or request that the program team do the work on their behalf. Default settings reflecting the Canadian status quo are available, and users can customize those settings for their own jurisdiction, subpopulations, or particular intervention application. Users have access to full documentation about the internal model structures, assumptions, and parameter descriptions.
OncoSim has been used to support decisions at the national and jurisdictional level. For example, after an independent evaluation, the Canadian Task Force on Preventive Health Care used OncoSim modelling to support its 2016 guidelines for colorectal and lung cancer screening11,12. Various provincial health ministries and cancer agencies are using OncoSim to study the effects of hpv testing in cervical cancer screening, to develop a business case for introducing lung cancer screening, and to explore potential levers to reduce wait times in colorectal cancer screening. OncoSim has also supported the Canadian Cancer Society in its annual statistics reports by making projections for special-topics chapters13,14. Similarly, the Partnership’s signature System Performance reports have relied on OncoSim analyses to examine future-state scenarios for key indicators of cancer control15. OncoSim is additionally playing a significant role in a Genome Canada–funded research project that is exploring the potential use of genetic information to tailor breast cancer screening in women primarily to breast cancer risk rather than to age. Table i presents other examples of OncoSim applications.
TABLE I Use of OncoSim to support policy- and decision-making
OncoSim was developed to aid evidence-based policy- and decision-making in cancer control in Canada. Published hierarchies of medical evidence usually recognize systematic reviews, principally based on rcts, as the highest level of evidence, and continue down through decreasing levels of evidence18,19. Simulation studies have not traditionally been included in such hierarchies. Nevertheless, simulation studies in which intervention effects are estimated for populations are integral in informing cancer control policy when clinical studies are not feasible or desirable20.
In clinical studies, only a small, finite number of interventions can typically be compared at one time, even though, in practice, multiple options might have to be considered and compared. For example, cervical cancer screening can be implemented using the Pap test or hpv test, both with varying follow-up protocols, and with or without prior hpv vaccination. Furthermore, the eligible age ranges, frequencies of screening, vaccine choice, and choice to vaccinate or not vaccinate boys as well as girls are variables that lead to multiple scenarios. Simulation studies are a feasible approach to evaluate the full range of potential choices. Likewise, outcomes of interest for real-world decision-making such as cost-effectiveness are not provided by most clinical cancer studies, given that extended time horizons are not possible. Simulation studies allow for the projection of costs and effects into the future and thus can take into account important trends such as population aging and declining smoking rates.
OncoSim has several limitations.
First, OncoSim is a complex set of models that might appear to be a “black box.” To shed light, all inputs are viewable with the published model and are accompanied by a data dictionary and supporting documentation (visit http://www.oncosim.ca).
Second, OncoSim has the capacity to incorporate and project the effects of parameter uncertainty, but because of the computational demands involved in handling the model’s more than 100,000 inputs, it does not yet easily or fully support probabilistic analysis, as recommended by guidelines21–23. However, assessment and implementation steps are being taken to enable probabilistic analysis.
Third, the models require continual updates to refresh their data inputs. Some components of OncoSim were initially developed using 2008 data and, in some cases, relied on expert opinion. Those components are currently being updated as access to more recent administrative data has become available. An update cycle is ongoing for all components of the model.
Fourth, external validation is a challenging but critical contributor to the plausibility of model projections. The colorectal cancer screening module has recently undergone an extensive validation exercise that replicates four rcts (submitted for publication), and work has begun to evaluate cervical cancer screening against rcts. To ensure the external validity of lung cancer low-dose computed tomography screening, OncoSim results were compared with those from the U.S. National Lung Screening Trial (nlst) and the relevant cisnet models24. Using simulated trial characteristics and a cohort matched to the nlst, outcomes similar to those reported from the trial were achieved3. Compared with the averaged results of the five cisnet models for scenarios created in an identical manner, OncoSim produced higher estimates of overdiagnosis and lower estimates of mortality reduction (Table 5 in Flanagan et al.3). Those discrepancies might be attributable to the demographic and smoking prevalence differences between Canada and the United States as well as to the underlying model assumptions, but further investigation is needed. In a comparison with the nlst primary outcome of mortality reduction, OncoSim results fell well within the range of those from the individual cisnet models, with all model results converging to the nlst results by the trial median follow-up of 6 years (Figure 2).
FIGURE 2 Ratio of cumulative number of lung cancer deaths in control arm to screen arm: comparison of OncoSim to CISNET (U.S. Cancer Intervention and Surveillance Modeling Network) and NLST (U.S. National Lunch Screening Trial) results. Source: OncoSim version 2.1 and Appendix Figure 2 in de Koning et al.24
Finally, OncoSim requires extensive and comprehensive data that are sometimes not available—perhaps because of non-collection or protection under privacy legislation. Consequently, although data are gathered from the best available sources, they might not be fully representative across all jurisdictions. Nevertheless, OncoSim can be tailored to inform decision-making within relevant contexts using local costs or patterns of practice. Model inputs, projections, and validation—and consequently decision-making—could be enhanced by innovative methods to leverage existing data holdings through data linkage and reduced barriers to accessing data.
OncoSim, available online and free of charge for public sector use, is the only model of its kind in Canada and is devoted to policy-pertinent pan-Canadian decision-making. Built on an extensive set of epidemiologic and economic data and rigorously validated, OncoSim can evaluate complex cancer control scenarios for multiple cancers. Policy- and decision-makers thus have a powerful tool to assess the costs, benefits, and cost-effectiveness of cancer control interventions when faced with difficult choices for improvements in population health and resource allocation.
OncoSim is led and supported by the Canadian Partnership Against Cancer, with model development by Statistics Canada, and is made possible through funding provided by Health Canada. We thank the many individuals who have contributed to the conceptualization, development, review, and application of OncoSim (see http://www.cancerview.ca/qualityandplanning/oncosim/acknowledgements/).
We have read and understood Current Oncology’s policy on disclosing conflicts of interest, and we declare that we have none.
*Health Economics, Canadian Partnership Against Cancer, Toronto, ON;,
†Health Analysis, Statistics Canada, Ottawa, ON;,
‡ Dalla Lana School of Public Health, University of Toronto, Toronto, ON;,
§Department of Oncology, McMaster University, Hamilton, ON;,
‖Faculty of Medicine, Memorial University, St. John’s, NL;,
#School of Epidemiology, Public Health and Preventive Medicine, University of Ottawa, Ottawa, ON;,
**Cancer Control Research, BC Cancer Research Centre, Vancouver, BC..
1. Evans WK, Wolfson MC, Flanagan WM, et al. Canadian Cancer Risk Management Model: evaluation of cancer control. Int J Technol Assess Health Care 2013;29:131–9.
2. Evans WK, Wolfson M, Flanagan WM, et al. The evaluation of cancer control interventions in lung cancer using the Canadian Cancer Risk Management Model. Lung Cancer Manag 2012;1:25–33.
3. Flanagan WM, Evans WK, Fitzgerald NR, Goffin JR, Miller AB, Wolfson MC. Performance of the cancer risk management model lung cancer screening module. Health Rep 2015;26:11–18.
4. Goffin JR, Flanagan WM, Miller AB, et al. Biennial lung cancer screening in Canada with smoking cessation—outcomes and cost-effectiveness. Lung Cancer 2016;101:98–103.
5. Miller AB, Gribble S, Nadeau C, et al. Evaluation of the natural history of cancer of the cervix, implications for prevention. The Cancer Risk Management Model (crmm)—human papilloma-virus and cervical components. J Cancer Policy 2015;4:1–6.
6. Louie AV, Rodrigues GB, Palma DA, Senan S. Measuring the population impact of introducing stereotactic ablative radiotherapy for stage i non–small cell lung cancer in Canada. Oncologist 2014;19:880–5.
7. Coldman AJ, Phillips N, Brisson J, et al. Using the Cancer Risk Management Model to evaluate colorectal cancer screening options for Canada. Curr Oncol 2015;22:e41–50.
8. Popadiuk C, Gauvreau CL, Bhavsar M, et al. Using the Cancer Risk Management Model to evaluate the health and economic impacts of cytology compared with human papillomavirus dna testing for primary cervical cancer screening in Canada. Curr Oncol 2016;23(suppl 1):S56–63.
9. Sadowski DC, Fitzgerald N, Flanagan WM, Hilsden RJ. Post colonoscopy colorectal cancer: how should we calculate a rate? Implications for quality programs [abstract Mo1930]. Gastroenterology 2015;148:S-742.
10. Coldman A, Flanagan W, Nadeau C, et al. Projected effect of fecal immunochemical test threshold for colorectal cancer screening on outcomes and costs for Canada using the OncoSim microsimulation model. J Cancer Policy 2017;13:38–46.
11. Bacchus CM, Dunfield L, Gorber SC, et al. on behalf of the Canadian Task Force on Preventive Health Care. Recommendations on screening for colorectal cancer in primary care. CMAJ 2016;188:340–8.
12. Lewin G, Morissette K, Dickinson J, et al. on behalf of the Canadian Task Force on Preventive Health Care. Recommendations on screening for lung cancer. CMAJ 2016; 188:425–32.
13. Bryan S, Demers A, Ellison L, et al. Chapter 7: Special topic: hpv-associated cancers. In: Canadian Cancer Society’s Advisory Committee on Cancer Statistics. Canadian Cancer Statistics 2016. Toronto, ON: Canadian Cancer Society; 2016.
14. Xie L, De P, Semenciw R, et al. Chapter 7: Special topic: predictions of the future burden of cancer in Canada. In: Canadian Cancer Society’s Advisory Committee on Cancer Statistics. Canadian Cancer Statistics 2015. Toronto, ON: Canadian Cancer Society; 2015.
15. Canadian Partnership Against Cancer (cpac). Cancer System Performance: 2017 Report. Toronto, ON: cpac; 2017.
16. Institute of Health Economics (ihe). Low Dose Computed Tomography for the Screening of Lung Cancer in Adults. Edmonton, AB: ihe; 2014. [Downloadable from: https://www.ihe.ca/publications/low-dose-computed-tomography-for-the-screening-of-lung-cancer-in-adults; cited 21 September 2017]
17. Canadian Partnership Against Cancer (cpac). Quality and Sustainability in Cancer Control: A System Performance Spotlight Report. Toronto, ON: cpac; 2016. [Downloadable from: http://www.systemperformance.ca/reports/; cited 21 September 2017]
18. Burns PB, Rohrich RJ, Chung KC. The levels of evidence and their role in evidence-based medicine. Plast Reconstr Surg 2011;128:305–10.
19. Rutter CM, Knudsen AB, Marsh TL, et al. Validation of models used to inform colorectal cancer screening guidelines: accuracy and implications. Med Decis Making 2016;36:604–14.
20. Weinstein MC, Toy EL, Sandberg EA, et al. Modeling for health care and other policy decisions: uses, roles, and validity. Value Health 2001;4:348–61.
21. Canadian Agency for Drugs and Technologies in Health (cadth). Guidelines for the Economic Evaluation of Health Technologies: Canada. 4th ed. Ottawa, ON: cadth; 2017.
22. Caro JJ, Briggs AH, Siebert U, Kuntz KM on behalf of the ispor–smdm Modeling Good Research Practices Task Force. Modeling good research practices—overview: a report of the ispor–smdm Modeling Good Research Practices Task Force—1. Value Health 2012;15:796–803.
23. International Society for Pharmacoeconomics and Outcomes Research (ispor). Outcomes Research Guideline Index > Synthesizing Health Care Evidence > Modeling/Decision Analysis Methods [Web page]. South Lawrenceville, NJ: ispor; n.d. [Available at: https://www.ispor.org/GuidelinesIndex/Default.aspx#HEEM; cited 4 August 2017]
24. de Koning HJ, Meza R, Plevritis SK, et al. Benefits and harms of computed tomography lung cancer screening strategies: a comparative modeling study for the U.S. Preventive Services Task Force. Ann Intern Med 2014;160:311–20.
(Return to Top)
aUntil 2016, the OncoSim model platform was called the Cancer Risk Management Model. ( Return to Text )
bThe cervical cancer model has a companion HPV transmission pre-model that provides input into the cancer natural history component of the main cervical cancer model. ( Return to Text )
Current Oncology, VOLUME 24, NUMBER 6, December 2017