Patient-reported outcomes in Alberta: rationale, scope, and design of a database initiative

Original Article

Patient-reported outcomes in Alberta: rationale, scope, and design of a database initiative

C.A. Cuthbert, RN NP PhD*, L. Watson, RN PhD, Y. Xu, MD PhD, D.J. Boyne, MSc, B.R. Hemmelgarn, MD PhD, W.Y. Cheung, MD MPH*




The collection of patient reported outcomes (pros) is a standard of care in many cancer organizations. In Alberta, pros have been integrated into routine clinical practice since 2012. This longitudinal collection of pros provides a wealth of data and a unique research opportunity to improve cancer care. The goal of this pro data initiative is to establish a robust repository of information for ongoing clinical care and research focused on pros. In this paper, we describe the rationale, scope, and design of this initiative.


The initiative consists of pros and other administrative health data from the province of Alberta. Retrieval of health data from a variety of provincially governed sources will create a platform of information on pros, health outcomes, cancer data, other health conditions, and demographics. The aims of the initiative are to use the data to inform best practices at the point of care; to conduct health services research, particularly clinical epidemiology studies; and to evaluate a variety of pro-related outcomes.


Because this effort represents our first to integrate routinely collected pros with other administrative health data, a unique and robust data repository will be created. The ability to integrate various types of data will provide a comprehensive mechanism to evaluate a variety of outcomes. Because cancer care in Alberta is governed by a single health care system, the data linkages will include population health and psychosocial cancer data. We anticipate that research related to this initiative will ultimately help to inform more patient-centred care.

KEYWORDS: Patient-reported outcomes, pros, patient-centred care, health services research, administrative health data


A cancer diagnosis and subsequent treatments can have multiple physical, psychological, and practical effects on patients and their families1. In recognition of those effects, increasing emphasis is being placed on a more global assessment of a cancer patient’s symptoms and supportive care needs in addition to conventional biologic outcome measures such as disease-free and overall survival2,3. One way to achieve such a holistic assessment is to ask patients directly about how their disease, their treatments, and their illness affect various aspects of their lives. These patient-reported outcomes (pros) can include measurements of health-related quality of life, symptoms, supportive care needs, and the broader patient experience4.

Since about 2010, pros have been increasingly integrated into routine clinical care. The growing interest in pros stems from the aim of achieving a more comprehensive evaluation of patients to inform care that is timely, evidence-based, personalized, and patient-centred2,4. As outlined by the U.S. Institute of Medicine, patient-centred care should be directed toward meeting the patient’s physical, emotional, and practical needs, while maintaining or improving quality of life and respecting the patient’s values, preferences, and needs1. Thus, collecting, reviewing, and using pros as part of a standard of care can serve two main purposes: to help achieve high-quality care and to evaluate the patient-centredness of care.

In oncology, the collection of pros has been endorsed as a new standard by national organizations such as the Canadian Partnership Against Cancer and the U.S. National Comprehensive Cancer Network5,6. Provincial implementation of pro frameworks (such as by Cancer Care Ontario) provide evidence of uptake in Canada. In addition to improving patient–clinician communication and the quality of care, pros have the potential to provide real-world evidence for clinicians and researchers who are working to assess the quality of health care systems and to help patients with decision-making7.

Alberta has a province-wide ambulatory cancer care program, and the decision to implement pros as part of routine clinical care was based on the inclusion of screening for distress as an accreditation standard for ambulatory cancer agencies in Canada. The specific pro measures used in Alberta include the Edmonton Symptom Assessment System, revised version (esasr)8,9, and the Canadian Problem Checklist (cpc)10. Those instruments were selected in alignment with the national patient experience reporting criteria established by the Canadian Partnership Against Cancer11. Province-wide implementation started in 2012 with the development of a standardized pro questionnaire to be used at all 17 ambulatory cancer facilities within Alberta. Implementation strategies included targeted practice change support to ensure that each pro questionnaire was reviewed within the clinical encounter and that a shared decision-making process was enacted with the patient to identify the most meaningful clinical response to the patient’s priority concerns12.

The deliberate and longitudinal collection of pros in routine clinical practice within Alberta provides a wealth of data to inform best practice at the point of care and a unique research opportunity. From the research perspective, our health services research group is a collaborative network of clinicians and researchers across Alberta whose shared goal is to collect and link this integrated health and pro data to establish a robust repository of information for ongoing quality improvement and clinical research focused on pros. Linkage of pros to other provincial administrative health data such as health outcomes, clinical characteristics, and demographics will further enhance the province’s capacity to optimize cancer care and research. In the remainder of this paper, the objectives, initial and future research projects, and design of that data initiative are described.


Objectives for PRO Data Collection and Initial Research Projects

One of the objectives of collecting pro data is to use the data to inform care—an objective that can be accomplished by health services research, particularly clinical epidemiology studies, to evaluate a variety of outcomes related to pros. Initial research questions identified by our group include, but are not limited to, the following:

  • ■ Can pros, in addition to cancer type, treatment type, demographics, and health profile (comorbidities), help to identify patients with the highest physical symptom burden and psychosocial needs at the outset of treatment such that optimal clinical care pathways or better coordinated delivery of cancer care (or both) can be determined?

  • ■ Can pros, in addition to cancer type, treatment type, demographics, and health profile (comorbidities), help to identify patients with the highest physical symptom burden and psychosocial needs at the completion of their primary treatment so as to develop tailored survivorship models of care?

  • ■ Can pros, in addition to cancer type, treatment type, demographics, and health profile (comorbidities), stratify patients into groups based on risk of disease progression, risk of hospitalization, and overall survival?

  • ■ Are there gaps between patient-reported physical symptoms or psychosocial needs and the care that those patients receive from cancer and non-cancer health care providers?

  • ■ What is the patient perspective about the future research or implementation needed to ensure that pros are meeting the patient’s needs?


The core component of the data initiative is the actual pro data collected from individual patients during their visits to 1 of 17 ambulatory oncology clinics in Alberta. Approximately 3000 patients are seen each month. A paperand-pencil questionnaire called the Putting Patients First (ppf) survey is collected from each ambulatory cancer patient. The completed information is then entered into the oncology electronic medical record system. A number of additional data sources can be leveraged and linked to the pro questionnaire data, including data about the cancer diagnosis, non-cancer comorbidities, demographics, vital statistics, laboratory measures, and cancer and non-cancer treatments received (Figure 1).



FIGURE 1 Integration of data sources for patient-reported outcomes research.

PPF Survey

The ppf survey (Figure 2) was modelled after recommendations in the clinical practice guideline from the Canadian Partnership Against Cancer and the Canadian Association of Psychosocial Oncology concerning assessment of the psychosocial health care needs of adult cancer patients13. The minimum dataset outlined in the guideline includes the esasr and the 21-item cpc.



FIGURE 2 Putting Patients First survey.

The esasr is a well validated questionnaire8,9,14 that has been used in a variety of cancer populations to screen for symptoms commonly experienced by cancer patients (pain, tiredness, nausea, fatigue, depression, anxiety, drowsiness, appetite, lack of well-being, and shortness of breath)10. Patients rate the severity of their symptoms using a numeric scale, with 0 representing the absence of symptoms and 10 representing the most severe symptoms. The esasr has been used to describe symptoms in cancer patients at various phases of the cancer trajectory15, including to identify symptom clusters and symptom severity16. It has also been used to explore associations between cancers, patient variables, and symptoms experienced17.

The cpc was initially developed by the Screening for Distress Toolkit Working Group10 to complement the esasr by capturing other psychosocial, practical, and informational health concerns that cancer patients might be experiencing or need help managing. The cpc list of concerns is based on prior research into the supportive care needs of cancer patients18 and the U.S. National Comprehensive Cancer Network’s list of common problems19. The cancer program in Alberta added to the national cpc by including an additional 33 items that capture other issues or concerns related to cancer treatments that were not included in the esasr or the original cpc12. The expanded cpc items were grouped into 7 domains: emotional; social, family, or spiritual; practical; physical; mobility; informational; and nutrition. Patients are typically instructed to check all concerns that they have had since their last visit.

In addition to those two questionnaires, the ppf also collects information from patients related to recent health care encounters, medication changes, and preferences for information about goals of care or advance care planning. Those items were added to the ppf in response to standards set out by Accreditation Canada and to the province’s health services policies. Health care providers review the completed ppf with patients so as to understand which symptoms or concerns are most troublesome for the patient and to identify symptoms that require further clinical assessment and intervention. A process of shared decision-making with the patient is meant to follow, with the goal of identifying and putting into action the most meaningful clinical responses20. The health care provider then documents the patient’s priority concern, clinical responses, and any referrals to other providers that have been initiated.

Health Data within Alberta

Because health care in Canada is provincially managed, Alberta Health is the provincial ministry responsible for organizing and funding delivery of health care services in Alberta (primary care, acute care, and home or community care). Alberta Health is also the custodian of all health information in Alberta. Alberta Health Services, an arm of the ministry, is the provincial health authority responsible for the planning and delivery of health services (outside of primary care) within Alberta. CancerControl Alberta is a provincial program within Alberta Health Services primarily responsible for the delivery of ambulatory oncology services.

Health information is collected by Alberta Health and can be accessed for analysis and storage by various groups within Alberta Health Services (Figure 1). Each Albertan has a personal health information number that is associated with each of their health care interactions in Alberta. All health care data are available electronically and can be linked based on the personal health information number. With appropriate ethics approval, those data can be accessed and shared depending on the purpose.

Alberta Health Data

The main data elements available from Alberta Health come from the population registry, physician claims, vital statistics, and hospital inpatient and outpatient encounters. The population registry includes the individual’s unique personal health information number, date of birth, sex, First Nations status, and 6-character postal code (providing information about geographic location within the province). Vital statistics provides date of death, place of death, hospital identifier, and cause of death according to the International Classification of Diseases, 10th revision (icd-10). Practitioner claims include fee-for-service and shadow-billed claims that have been submitted by medical doctors and include provider identification (speciality, role, and location), patient identifier, and service information [date of service, location, facility, and diagnostic codes using the International Classification of Diseases, 9th revision, clinical modification (icd-9-cm)]. Other data available through Alberta Health that could be used in future analyses include pharmaceutical dispensing information (all non-cancer medications, which are stored in the pharmaceutical information network), laboratory measures, diagnostic imaging information, and data about ambulatory care (emergency department) visits.

Alberta Health Services Data Repositories

Alberta Health Services data elements originate with Alberta Health and are then stored in provincial data repositories. Traditionally, data relating to cancer measures and outcomes have been stored separately from all other patient data. The personal health information number allows for linkages between the data repositories. Non-cancer data elements held by Alberta Health Services include inpatient encounters [admission and discharge dates; diagnostic and procedure codes using the icd-9-cm (until 31 March 2002) or the icd-10]; ambulatory care and emergency visits, including type of specialist provider seen, dates of services and diagnostic and procedure codes using the icd-9-cm or icd-10; and ambulatory diagnostic imaging visits. Diagnostic and procedure codes from physician claims are used to determine comorbid medical conditions based on a validated coding algorithm for Charlson comorbidities21.

Alberta cancer data that include patient encounters with oncology are obtained from CancerControl Alberta. Data from CancerControl Alberta include information from electronic medical records such as the ppf data; date and type of systemic cancer treatment received; date, type, and dose of radiation therapy received; and cancer stage.

Alberta Cancer Registry Data

The Alberta Cancer Registry collects and records population-based information for all new cancer or precancerous pathologies within Alberta. A unique cancer patient identifier linked to the individual’s Provincial Health Information Number is assigned to every cancer. The cancer registry is responsible for reporting cancer data to Alberta Health and the Canadian Cancer Society. Cancer stage is coded using the Collaborative Staging System to derive American Joint Commission on Cancer TNM stage22. Other data collected by the cancer registry includes date of cancer diagnosis, date of birth, sex, and postal code. All data are available electronically and are shared with other health data management groups within Alberta Health Services and Alberta Health.

PRO Entry, Retrieval, and Storage

The pros are collected on a paper ppf questionnaire, which is then manually entered into the patient’s electronic oncology medical record by Alberta Health Services staff. That approach allows for real-time collection of pro data, with subsequent retrospective correlation with all other personal health data in other Alberta Health Services and Alberta Health databases. Currently, ppf data can be retrieved for research projects and be merged with other administrative health data, as dictated by the research question, to form specific datasets. For example, we will establish one dataset to answer our initial set of research questions. In the near future, retrieval of patient- and aggregate-level pro data will occur in real time and take place at the point of care (for example, in clinics) so that the data can be used to directly and immediately inform care and treatment in addition to research. One co-author (LW) is currently implementing the presentation of pro data in the clinical practice environment as real-time viewing of patient- and aggregate-level data (“pro dashboard”). The hope is that access to such data by patients and their health care team will be a useful tool to facilitate personalized management and accurate prioritization of concerns.


The ppf questionnaire is part of routine clinical care, and as such, collection of those data does not require informed consent. Ethics approval through the Health Research Ethics Board of Alberta has been received for data retrieval related to the first 3 research questions identified in this paper ( Further studies could require new ethics approval if they fall outside the scope of the original ethics approval. Before conducting any further research using amalgamated pro and other administrative health data, we will seek the advice of the Alberta provincial research ethics board and obtain ethics approval as needed.

Strengths and Limitations of the PRO Data

Establishing the infrastructure and platform for pro data collection is timely and resource-intensive, but it represents an investment for improving clinical care, enhancing quality initiatives, and expanding patient-centred research. Because this effort represents the province’s first attempt to integrate routinely collected pro data with electronic medical records and other administrative health data, a unique and robust data repository will be created. The ability to integrate various types of data will provide a comprehensive mechanism to evaluate a variety of important cancer outcomes in Alberta. In addition, because Alberta has a single provider of ambulatory cancer care in the province, the merged data will be population-based. The ppf includes the minimum cpc dataset recommended by established national guidelines and the validated esasr questionnaire.

Despite the strengths of the pro data, there are some limitations. Administrative health data are limited by the variables collected, and so important variables might be missing from the database (for example, detailed demographics, lifestyle factors, treatments with nontraditional medicine, interventions given by allied health care providers). In addition, data about comorbidities and encounters with the health care system are limited by physician accuracy in submitting appropriate claims for billing. Although our cancer care system is provincial, not all patients are seen at an ambulatory oncology centre (for example, patients with early-stage colorectal cancer might be seen by a non-cancer-centre surgeon and might not always be referred to the cancer centre). We might therefore not have pro data for some patients for whom we have other health data. The pro data are currently entered manually, which increases the risk for data entry errors, but there are plans to transition to direct electronic data entry by the patient or caregiver. In addition, patients can choose not to complete the pro questionnaire or to only partially complete it. Although pro completion, review by a clinician, and entry of data into the electronic medical record is considered the standard of care, the process might not achieve a 100% compliance rate. Because the cpc is a checklist, there is no validation for the tool, nor are there established norms or clinical cut-offs. The published research to help guide cpc data analysis or interpretation is limited to some descriptive studies19, and so more studies are needed to help establish cut-off scores (for example, “4 of 21 problems indicates a high level of unmet need”). Likewise, additional insights into the clinical relevance of the types of psychosocial problems reported and the types of clinical actions considered appropriate to address those problems are warranted.

To address some of the limitations, we are establishing a group consisting of epidemiologists, statisticians, and clinician–scientists who will investigate, compare, and implement methods for addressing concerns related to pro data (such as residual confounding, measurement error, missing data, and selection bias). In addition, members within our health services research group are currently developing algorithms that will allow for the identification of variables not routinely or reliably captured in clinical practice—for example, symptom onset. We believe that this methodology work will be a substantial contribution to pro research in general, because many of the foregoing data concerns are inherent to all pro data analysis.


The evidence for the effect of incorporating pros into clinical care is growing with recent clinical trials showing positive results for a number of outcomes. A recent systematic review by Kortronoulas et al.23 demonstrated that pros in cancer care can increase the frequency of conversations about a patient’s symptoms and concerns during clinical visits, can improve symptom control, and can enhance supportive care measures and patient satisfaction. Similarly, Velikova et al.24 found increased patient satisfaction and improved communication in their randomized controlled trial using a pro-based intervention. Most recently, Basch et al.25 demonstrated superior overall survival in patients randomized to a pro intervention compared with control subjects not receiving the intervention. Those studies highlight the value of pros in routine cancer care.

However, some limitations in the knowledge of how to most effectively use pros in clinical practice remain. Specifically, there is a lack of information about the appropriate timing for pros, how to tailor measures for specific patient populations, how clinicians can use pros to improve care delivery, the most effective processes or guidelines for addressing identified symptoms and needs, and how to ascertain the effect of routine use of pros on cost-effectiveness and health services use4,7,23. Establishing this provincial initiative for collecting pro data has the potential to support ongoing research into the value of pro data in enhancing ambulatory cancer care.

Future Directions

We anticipate that our data and related research activities will ultimately help to inform more patient-centred care experiences in the province. In addition, given a current emphasis on pro research in other provinces (such as Ontario), there is the potential to compare and contrast the implementation and ongoing evaluation of pros to best inform patient outcomes. We believe that outcomes research using these data will help to inform and substantiate national efforts to use pros in clinical practice.

We foresee future research, including qualitative inquiry, to engage with patients as partners to continue to refine pro instruments and processes, and to ensure that the patient voice is continually included. In addition, intervention research could lead to more tailored approaches to evaluating and addressing psychosocial outcomes in cancer patients. We are particularly interested in tailoring pros to specific patient populations and developing symptom management pathways, including self-management programs, based on patient-identified priorities. In conjunction with patient partners at a 2018 pro symposium through CancerControl Alberta (2018 Patient Reported Outcomes Symposium: Going “PRO” in Alberta), head-and-neck cancer, lung cancer, and colorectal cancer were identified as priority tumour groups for which pro measures could begin to be tailored and symptom management pathways could be developed and tested. We believe that engaging with patient partners to identify priority areas will help to ensure more successful planning and implementation of pro-related programs, ultimately directing future research.

In the end, our goal with this pro initiative is to inform more patient-centred care. Knowledge translation is imperative to the initiative and will help to achieve our goals. Our research findings will be disseminated through peer-reviewed publications and presentations at scientific conferences. In keeping with the recent move toward integrated knowledge translation, we believe that involving patients as partners in presentations and at conferences will help to ensure that the patient voice in included in cancer care.


We have read and understood Current Oncology’s policy on disclosing conflicts of interest, and we declare that we have none.


*Department of Oncology, Cumming School of Medicine, University of Calgary, Calgary, AB,
Alberta Health Services CancerControl and Faculty of Nursing, University of Calgary, Calgary, AB,
Department of Community Health Sciences, University of Calgary, Calgary, AB.


1 Hewitt ME, Greenfield S, Stovall E, eds. From Cancer Patient to Cancer Survivor: Lost in Transition. Washington, DC: National Academies Press; 2006.

2 Snyder CF, Aaronson NK, Choucair AK, et al. Implementing patient reported outcomes assessment in clinical practice: a review of the options and considerations. Qual Life Res 2012;21:1305–14.

3 Van de Poll-Franse LV, Horevoorts N, van Eenbergen M, et al. on behalf of the Profiles Registry Group. The patient reported outcomes following initial treatment and long term evaluation of survivorship registry: scope, rationale and design of an infrastructure of the study of physical and psychosocial outcomes in cancer survivorship cohorts. Eur J Cancer 2011;47:2188–94.

4 Chen J, Ou L, Hollis SJ. A systematic review of the impact of routine collection of patient reported outcome measures on patients, providers and health organizations in an oncologic setting. BMC Health Serv Res 2013;13:211.

5 Canadian Partnership Against Cancer (cpac). Effective, Efficient, Evolving: Enhancing Cancer Control in Canada. Annual report 2015/16. Toronto, ON: cpac; 2017. [Available online at:; cited 15 August 2018]

6 Nardi EA, McCanney J, Winckworth-Prejsnar K, et al. Redefining quality measures in cancer care. J Natl Compr Canc Netw 2018;16:473–8.

7 Basch E. New frontiers in patient-reported outcomes: adverse event reporting, comparative effectiveness and quality assessment. Ann Rev Med 2014;65:307–17.

8 Watanabe S, Nekoliachuk C, Beaumont C, Mawani A. The Edmonton Symptom Assessment System: what do patients think? Support Care Cancer 2009;17:675–83.

9 Watanabe SM, Nekoliachuk C, Beaumont C, Johnson L, Myers J, Strasser F. A multicenter study comparing two numerical versions of the Edmonton Symptom Assessment System in palliative care patients. J Pain and Symptom Manage 2011;41:456–68.

10 Bultz BD, Groff SL, Fitch M, et al. Implementing screening for distress, the 6th vital sign: a Canadian strategy for changing practice. Pyschooncology 2011;20:463–9.

11 Canadian Partnership Against Cancer (cpac). The 2012 Cancer System Performance Report. Toronto, ON: cpac; 2012. [Available online at:; cited 15 August 2018]

12 Watson L, Groff S, Tamagawa R, et al. Evaluating the impact of provincial implementation of screening for distress on quality of life, symptom reports, and psychosocial well-being in patients with cancer. J Natl Comp Cancer Netw 2016;14:164–72.

13 Howell D, Currie S, Mayo S, et al. A Pan-Canadian Clinical Practice Guideline: Assessment of Psychosocial Health Care Needs of the Adult Cancer Patient. Toronto, ON: Canadian Partnership Against Cancer (Cancer Journey Action Group) and the Canadian Association of Psychosocial Oncology; 2009.

14 Nekolaichuk C, Watanabe S, Beaumont C. The Edmonton Symptom Assessment System: a 15 year retrospective review of validation studies (1991–2006). Palliat Med 2008;22:111–22.

15 Hui D, Bruera E. The Edmonton Symptom Assessment System 25 years later: past, present and future developments. J Pain and Symptom Manage 2017;53:630–43.

16 Cheung WY, Le LW, Zimmermann C. Symptom clusters in patients with advanced cancers. Support Care Cancer 2009;17:1223–30.

17 Cheung WY, Le LL, Gagliese G, Zimmermann C. Age and gender differences in symptom intensity and symptom clusters among patients with metastatic cancer. Support Care Cancer 2011;19:417–23.

18 Ashbury RD, Findlay H, Reynolds B, McKerracher K. A Canadian survey of cancer patient’s experiences: are their needs being met? J Pain Symptom Manage 1998;16:298–306.

19 Blais MC, St-Hilaire A, Fillion L, De Serres M, Tremblay A. What to do with screening for distress scores? Integrating descriptive data into clinical practice. Palliat Support Care 2014;12:25–38.

20 Tamagawa R, Groff S, Anderson J, et al. Effects of a provincial wide implementation of screening for distress on health professionals’ confidence and understanding of person-centered care in oncology. J Natl Compr Can Netw 2016;14:1259–66.

21 Deyo R, Cherkin DC, Ciol MA. Adapting a clinical comorbidity index for use with icd-9-cm administrative databases. J Clin Epidemiol 1992;45:613–19.

22 Amin MB, Edge SB, Greene FL, et al., eds. AJCC Cancer Staging Manual. 8th ed. Chicago, IL: American College of Surgeons; 2018.

23 Kotronoulas G, Kearney N, Maguire R, et al. What is the value of the routine use of patient reported outcomes measures toward improvement of patient outcomes, processes of care, and health services outcomes in cancer care? A systematic review of controlled trials. J Clin Oncol 2014;32:1480–501.

24 Velikova G, Booth L, Smith AB, et al. Measuring quality of life in routine oncology practice improves communication and patient well-being: a randomized controlled trial. J Clin Oncol 2004;22:714–24.

25 Basch E, Deal AM, Dueck AC, et al. Overall survival results of a trial assessing patient-reported outcomes for symptom monitoring during routing cancer treatment. JAMA 2017;318:197–8.

Correspondence to: Colleen Cuthbert, Department of Oncology, Cumming School of Medicine, University of Calgary, Room AE173N, 1331–29th Street NW, Calgary, Alberta T2N 4N2. E-mail:

(Return to Top)

Current Oncology, VOLUME 26, NUMBER 4, August 2019

Copyright © 2019 Multimed Inc.
ISSN: 1198-0052 (Print) ISSN: 1718-7729 (Online)