Evidence in medicine: math versus biology!

Editorial


Evidence in medicine: math versus biology!


T. Younis, MBBCh*, M. Thana, MD*, C. Skedgel, MDE PhD§


doi: https://doi.org/10.3747/co.24.3970

The drive for optimal clinical decisions based on “best” evidence has gained significant momentum in the last few decades. In parallel with the evidence-based medicine approach, various “hierarchy of evidence” stratifications have also emerged13. Overall, those stratifications attempt to characterize underlying bias (that is, validity) and emphasize the relative quality of different forms of evidence. In a traditional pyramidal representation (Figure 1)2,3, systematic reviews and meta-analyses of randomized clinical trials (rcts) represent the pinnacle of the pyramid, followed by individual rcts and observational studies (for example, case–control or cohort studies). Conversely, case reports and series, often called “anecdotal evidence,” are placed at the bottom of the pyramid. Notwithstanding a number of limitations and criticisms4, including an overreliance on systematic reviews and meta-analyses, clinical decision-making continues to be guided by such “hierarchies of evidence.” Increasingly, however, the evidence-based medicine approach—and thus clinical decision-making—is coming to rely on mathematical or statistical characterizations of biologic processes (for example, disease progression or relapse, patient survival, and quality of life).

 


 

FIGURE 1 Hierarchy of evidence in medicine.

The applications of math and statistics in clinical medicine are numerous and include characterizations of biologic observations and mathematical modelling of health outcomes5. As examples of the former, the biologic observations encountered in rcts (such as disease progression or relapse, patient survival, and quality of life) are often represented by bio-statistical measures that include estimates of effect size and the uncertainties involved (for example, type i and ii errors, p values, and 95% confidence intervals)6,7. As well, meta-analyses often use mathematical measures that extrapolate efficacy from individual rcts to compute compound measures of effect size and uncertainty. Not surprisingly, for clinicians who chose a science path driven by human biology, the underlying math and statistics are not infrequently perceived as complex. Nonetheless, a working knowledge of the underlying statistical methods is necessary for clinicians to properly interpret the current evidence. Furthermore, knowledge that is more in-depth is also often required to properly interpret the clinical evidence and avoid the pitfalls of mere binary interpretation of study outcomes as simply positive or negative6,7.

Clinical evidence that stems from biologic observations cannot address all pertinent gaps in knowledge. As an example, observational studies and rcts cannot in themselves typically examine all the potential lifetime clinical and economic effects of cancer screening strategies that involve a variety of modalities or time intervals in various at-risk population-based cohorts. Mathematical models such as decision-analysis frameworks are particularly helpful when traditional clinical study designs are neither feasible nor practical5. A working knowledge of the building blocks underlying such models is therefore warranted.

All mathematical models incorporate a variety of input parameters within a framework structure to compute relevant outcomes (Figure 2). The underlying frameworks can range from relatively simple decision trees to more complex state-transition or Markov models and to even more complex microsimulation models and discrete event simulations, depending on the nature of the process to be modelled. The input parameters (clinical or economic, or both) are often derived from various sources of observational data (for example, rcts and records of medical resource use), but can also include estimates based on assumptions and expert opinion. Model outputs typically include clinical and economic outcomes such as incidence or prevalence, survival, quality of life, life-years, quality-adjusted life-years, costs, and cost-effectiveness. The effects of the uncertainties in input parameters on model outcomes are usually examined through one-way sensitivity analyses and probabilistic sensitivity analyses. The former examine the effect of each parameter independently; the latter simultaneously incorporate all parameter uncertainties so as to understand how interactions in parameter uncertainty could affect model outcomes. Models are also often subjected to a number of validation processes (for example, face, internal, and external validity) in an attempt to reduce bias.

 


 

FIGURE 2 Simplified schema of mathematical models.

A critical appraisal of the evidence stemming from models in medicine is essential5. Indeed, a “hierarchy of evidence” approach (Figure 1) for economic and decision analyses does exist3. As an example, within the levels of evidence set out by the Oxford Centre for Evidence-Based Medicine3, an analysis incorporating clinically sensible costs or alternatives, one or more systematic reviews of the evidence, and multiway sensitivity analyses would be placed higher than another analysis incorporating limited or poor-quality inputs, no systematic review or reviews of the evidence, and only limited or one-way sensitivity analyses. An analysis that does not incorporate sensitivity analyses would rank even lower, and evidence based on “expert opinion” alone would constitute the lowest level of evidence. Conversely, a properly conducted systematic review of multiple economic analyses would often represent the highest level of economic evidence. Other approaches for the critical appraisal of model-based evidence are also available, including scoring systems for economic analyses8 and guidelines for good modelling practices5 and for the conduct of systematic reviews for economic evaluations9. Evidence in medicine that is based on mathematical modelling therefore varies with respect to its strengths and should affect clinical decisions only accordingly.

In this issue of Current Oncology, Gauvreau and colleagues10 highlight Canada’s OncoSim platform—a Web-based suite of cancer-specific microsimulation models designed to augment conventional resources for population-level decision-making in Canada. OncoSim currently comprises three in-depth cancer models (lung, colorectal, cervical), which are built on a common framework, with another in-depth model (breast cancer) and a generalized model (25 cancers) in development. The microsimulation platform is led and supported by the Canadian Partnership Against Cancer, with development by Statistics Canada and funding by Health Canada. The OncoSim platform is freely available online, with user capabilities that allow for customization of a number of input parameters and model outputs. In OncoSim, the input parameters are derived from various Canadian sources, with model validation (that is, face, internal, and external validity) and fit against observed data (calibration). Its outputs can be customized to include cancer burden and outcomes (for example, incidence, mortality, life expectancy, and quality-adjusted life-years) as well as health care resources and economics (for example, procedures or treatments, health care costs, and cost-effectiveness). One-way sensitivity analyses can currently be conducted, and the future plan is to incorporate multiway probabilistic sensitivity analyses.

Notwithstanding the limitations inherent in all microsimulation models, OncoSim is a powerful Web-based tool that projects the future burden of cancer and its economic effects in Canada and that allows for a critical assessment of the benefits and costs associated with various potential population-based cancer control interventions. Indeed, OncoSim has already been used to support a number of clinical decisions (for example, colorectal and lung cancer screening guidelines prepared by the Canadian Task Force on Preventive Health Care) and reports (for example, the system performance reports produced by the Canadian Partnership Against Cancer) at the national and jurisdictional levels.

In an era dominated by evidence-based medicine, applications of math to characterize or model biologic observations and processes are inevitable and cannot be avoided or dismissed as “black boxes.” As health care providers, we have come a long way in better understanding the biostatistics of observational and experimental studies. We now must also better understand—and scrutinize—the mathematical modelling platforms that have become an integral component of today’s evidence in medicine.

CONFLICT OF INTEREST DISCLOSURES

We have read and understood Current Oncology’s policy on disclosing conflicts of interest, and we declare the following interests: TY has served on a number of advisory and working groups for the Canadian Partnership Against Cancer and is the “breast cancer expert” member of the Working Group for the OncoSim breast cancer model that is currently in development. The views presented here are those of the authors and not those of the Partnership.

AUTHOR AFFILIATIONS

*Department of Medicine, qeii Health Sciences Centre, Halifax, NS;,
Faculty of Medicine, Dalhousie University, Halifax, NS;,
Health Economics Group, Norwich Medical School, University of East Anglia, Norwich, U.K.;,
§School of Pharmacy, Dalhousie University, Halifax, NS..

REFERENCES

1. Berlin JA, Golub RM. Meta-analysis as evidence: building a better pyramid. JAMA 2014;312:603–5.
cross-ref  pubmed  

2. Murad MH, Asi N, Alsawas M, Alahdab F. New evidence pyramid. Evid Based Med 2016;21:125–7.
cross-ref  pubmed  pmc  

3. Centre for Evidence-Based Medicine (cebm). Oxford Centre for Evidence-Based Medicine – Levels of Evidence (March 2009) [Web page]. Oxford, U.K.: cebm; 2017. [Available at: http://www.cebm.net/oxford-centre-evidence-based-medicine-levels-evidence-march-2009; cited 20 November 2017]

4. Miller CG, Miller DW. The real world failure of evidence-based medicine. Int J Pers Cent Med 2011;1:. [Available online at: http://www.ijpcm.org/index.php/IJPCM/article/view/70; cited 20 November 2017]

5. Caro JJ, Briggs AH, Siebert U, Kuntz KM on behalf of the isporsmdm Modeling Good Research Practices Task Force. Modeling good research practices—overview: a report of the isporsmdm Modeling Good Research Practices Task Force–1. Value Health 2012;15:796–803.
cross-ref  pubmed  

6. Pocock SJ, Stone GW. The primary outcome fails—what next? N Engl J Med 2016;375:861–70.
cross-ref  pubmed  

7. Pocock SJ, Stone GW. The primary outcome is positive—is that good enough? N Engl J Med 2016;375:971–9.
cross-ref  pubmed  

8. Husereau D, Drummond M, Petrou S, et al. on behalf of the cheers Task Force. Consolidated Health Economic Evaluation Reporting Standards (cheers) statement. BMJ 2013;346:f1049.
cross-ref  

9. van Mastrigt GA, Hiligsmann M, Arts JJ, et al. How to prepare a systematic review of economic evaluations for informing evidence-based healthcare decisions: a five-step approach (part 1/3). Expert Rev Pharmacoecon Outcomes Res 2016;16:689–704.
cross-ref  pubmed  

10. Gauvreau CL, Fitzgerald NR, Memon S, et al. The OncoSim model: development and use for better decision-making in Canadian cancer control. Curr Oncol 2017;24:401–406.


Correspondence to: Tallal Younis, QEII Health Sciences Centre, Dalhousie University, Halifax, Nova Scotia B3H 2Y9. E-mail: tallal.younis@nshealth.ca

(Return to Top)



Current Oncology, VOLUME 24, NUMBER 6, December 2017








Copyright © 2018 Multimed Inc.
ISSN: 1198-0052 (Print) ISSN: 1718-7729 (Online)