The Brain Metastases Symptom Checklist as a novel tool for symptom measurement in patients with brain metastases undergoing whole-brain radiotherapy

Purpose We evaluated the feasibility, reliability, and validity of the Brain Metastases Symptom Checklist (bmsc), a novel self-report measure of common symptoms experienced by patients with brain metastases. Methods Patients with first-presentation symptomatic brain metastases (n = 137) referred for whole-brain radiotherapy (wbrt) completed the bmsc at time points before and after treatment. Their caregivers (n = 48) provided proxy ratings twice on the day of consultation to assess reliability, and at week 4 after wbrt to assess responsiveness to change. Correlations with 4 other validated assessment tools were evaluated. Results The symptoms reported on the bmsc were largely mild to moderate, with tiredness (71%) and difficulties with balance (61%) reported most commonly at baseline. Test–retest reliability for individual symptoms had a median intraclass correlation of 0.59 (range: 0.23–0.85). Caregiver proxy and patient responses had a median intraclass correlation of 0.52. Correlation of absolute scores on the bmsc and other symptom assessment tools was low, but consistency in the direction of symptom change was observed. At week 4, change in symptoms was variable, with improvements in weight gain and sleep of 42% and 41% respectively, and worsening of tiredness and drowsiness of 62% and 59% respectively. Conclusions The bmsc captures a wide range of symptoms experienced by patients with brain metastases, and it is sensitive to change. It demonstrated adequate test–retest reliability and face validity in terms of its responsiveness to change. Future research is needed to determine whether modifications to the bmsc itself or correlation with more symptom-specific measures will enhance validity.


INTRODUCTION
Brain metastases are a common complication in many malignancies, causing significant morbidity and directly contributing to cancer mortality 1 .They cause localized symptoms related to the location of tumour involvement and generalized symptoms from raised intracranial pressure.Many individuals with brain metastases have advanced disease, and survival remains extremely limited 2,3 .
Focal weakness is the presenting complaint in approximately 20% -40% of patients, and seizures are the first sign of brain metastasis in approximately 10%-20% 4 .Progressive neurologic deficits can occur, the nature of which depends on size and location of the lesions.A 2003 review of the literature on the natural history of brain metastases found that headache (39%), motor weakness (33%), ataxia (31%), altered mental status (31%), and dysphasia (15%) were the main presenting symptoms 5  number of treatment options are available, whole-brain radiation therapy (wbrt), together with steroids, has been the mainstay of treatment 6,7 .In up to 75% of patients, wbrt has been found to improve symptoms 8 and to be associated with an increase in median overall survival from 1 month with no treatment to 2-7 months depending on performance status 1 .
Given a median survival of 3-6 months after wbrt 6 , the primary goals of the intervention are to relieve symptoms, to improve quality of life, and to maintain those benefits in the absence of steroid therapy.Important secondary objectives include prevention of disease progression and its associated symptoms, together with avoidance of radiotherapy-related toxicity.The most common outcomes assessed in clinical trials of patients with brain metastases are survival, radiologic response, and overall neurologic status.Although symptom relief is the primary goal, that outcome is often not adequately assessed.Further, many of the scales commonly used in such trials are neither specific enough nor brief enough to be used in routine clinical practice.
Building on earlier work by Bezjak et al. 4 , the Princess Margaret Cancer Centre's Brain Metastases Symptom Checklist (bmsc) was designed to address the gap in symptom measurement in patients with brain metastases.The availability of a tool to document the symptom profile of these patients over time and to establish treatment response is invaluable to facilitate an understanding of their condition and to care for these patients.At the time that the bmsc was developed, no comparable tool was available for this purpose.The initial checklist had 17 items, scored on a 4-point ordinal scale, to assess symptom response 1 month after radiation in patients with symptomatic brain metastases 4 .The bmsc was subsequently modified to include 18 items scored on a 10-point scale similar to the well-validated Edmonton Symptom Assessment Scale (esas) 9 .Table i presents a full version of the bmsc.
The primary objectives of the present prospective single-arm study were to test the reliability, validity, and feasibility of the bmsc for assessing symptoms in a clinical setting in patients with brain metastases undergoing wbrt.

Design of the BMSC
The bmsc is an 18-item checklist that asks patients to rate their symptoms on an intensity scale from 0 (asymptomatic) to 10 (worst possible symptoms), based on the severity of their symptoms during the preceding 24 hours.Symptom scores of 1-3 are classified as "mild"; 4-6, as "moderate"; and 7-10, as "severe."An additional section on the questionnaire asks patients to select up to 3 symptoms from the intensity scale that have been the most troublesome and to indicate whether they have improved, remained stable, or worsened.The current dose of dexamethasone is also recorded on the questionnaire.
The items in the bmsc were selected based on a retrospective review of the literature and on institutional experience.To enhance its utility in the clinical setting, the bmsc was designed to fit onto a single page and to be completed by patients or caregivers within 5-10 minutes.

Study Design
This single-arm prospective observational study set out to establish the following characteristics of the questionnaire: n Test-retest reliability n Reliability of caregiver scoring n Feasibility of documenting common symptoms in a clinical setting and sensitivity to change n Feasibility of distinguishing symptoms secondary to metastases from other causes n Correlation with other performance status measures The study was approved by the institutional research ethics review board.

Eligibility
Patients with brain metastases who were candidates for wbrt at Princess Margaret Cancer Centre and who were able to complete questionnaires were invited to participate.Patients who had received prior radiotherapy or surgery were not excluded from the study.The primary caregivers of the patients, as identified by the patients, were also invited to participate.The questionnaire was available only in English, although translation services were available.

Methods
After providing informed consent, patients were invited to complete the bmsc at 6 time points: twice at baseline based on current symptoms (once before and once after the consultation appointment); a retrospective questionnaire, completed before the consultation appointment, based on symptoms before starting dexamethasone; at the end of wbrt; and at weeks 1, 2, 4, and 6 after wbrt.Patients were accrued over a period spanning 2006-2012.
Here, we report the results from the 4-week follow-up because that time point is the most common for clinical assessment of response to treatment at Princess Margaret Cancer Centre.Significant attrition at week 6 also limited analysis.Individuals identified as caregivers by the patient participants were invited to independently complete the questionnaire twice: at the initial consultation before the start of radiotherapy, and at the conclusion of radiotherapy.Those time points were the same ones at which the participants completed their questionnaires.All participating caregivers provided informed consent.
Patients received a dose of 20 Gy in 5 fractions, which is the institutional practice for wbrt for brain metastases.Tapering or adjustment of the steroid dose was conducted as per clinical practice and was not prescribed by the study.Attribution of symptoms to metastases, steroids, or adverse effects of wbrt by the research coordinator or the attending physician was planned at each time point, but was inconsistently completed.Follow-up questionnaires were completed either in person or over the telephone.
Patients also completed the Short Orientation-Memory-Concentration Test (somc) 10 and the EuroQol EQ-5D-3L (EuroQol Group, Rotterdam, Netherlands) quality of life questionnaire 11 at the same time as the bmsc.Those tests were administered only once (after the consultation visit).The somc is a validated measure of cognitive impairment that is easily administered by non-physicians and that can discriminate between mild, moderate, and severe cognitive defects.It is scored out of 28, with a score less than 20 indicating cognitive impairment.The EQ-5D-3L is a standardized scale that consists of a descriptive component and a visual analog scale.The descriptive component consists of 5 dimensions: mobility, self-care, usual activities, pain or discomfort, and anxiety or depression.Each dimension is scored as "no problems," "some problems," or "extreme problems."The visual analog scale records the patient's self-rated health on a vertical scale of 0-100, where the endpoints are labelled "best imaginable health state" and "worst imaginable health state." Other disease and treatment details recorded were the extent of the primary cancer; the number of metastases; the location of the metastases (supratentorial, infratentorial, or both); dexamethasone dose, including date of first dose and current dose; duration of neurologic symptoms; Eastern Cooperative Oncology Group (ecog) performance status; and U.K. Medical Research Council (mrc) neurologic functioning 12 .The ecog and mrc scores were collected prospectively.

Statistical Analysis
A sample size of 55 patients was calculated to produce a 95% confidence interval equal to the sample intraclass correlation (icc) plus or minus 0.1 when the estimated icc is 0.8.Reliability was measured by correlating the pre-and post-consultation symptom scores at baseline.Similarly, the reliability of caregiver scoring was measured using an icc score comparing patient and caregiver symptom intensity scores.
The validity of the bmsc was assessed by its sensitivity to change over time and by its correlation to changes as observed in the somc, EQ-5D-3L, ecog performance status, and mrc neurologic status (Spearman correlation coefficients).To assess the proportion of patients with a change in symptoms at week 4, a required sample size of 57 was calculated.

RESULTS
A total of 137 patients and 48 caregivers participated in the study.

Reliability
Pre-and post-consultation scores were available for 82 patients (60%, Figure 1).Overall, the median icc score was 0.59 (range: 0.14-0.85).Speech, confusion, and vomiting were the only scores below 0.5.Paired patient and proxy caregiver scores were available for 38 patient-caregiver pairs at baseline [Figure 2(A)] and for 28 pairs at the end of radiation [Figure 2(B)].The median icc at baseline was 0.52, and it was maintained at the end of radiation with an icc of 0.58.The items that showed the least equivalency included neurocognitive symptoms such as confusion (icc: 0.02) and speech changes (icc: 0.14).

Symptom Profile
Figure 3 shows the symptom profile of the patient sample at baseline.With some exceptions, the symptoms at baseline were mostly mild to moderate.More than 70% of participants (97 of 137) identified tiredness at baseline, and almost 70% of those 97 patients (n = 66) identified their tiredness as moderate or severe.The least frequently cited symptom was vomiting, which was experienced by only 10% of participants (14 of 137).

Responsiveness to Change
Three different methods were used to measure the sensitivity of the bmsc to clinical change: n Direction of symptom change (improve, stable, worsen) n Change in symptom severity (mild, moderate, severe) n Change in mean symptom intensity Figure 4 shows the direction of symptom change.Overall, weight gain and sleep showed the greatest degree of improvement, with 42% and 41% of participants respectively noting a decrease in their symptoms.The greatest increase in symptom intensity was seen in tiredness and drowsiness, which worsened in 62% and 59% of participants respectively.Most other symptoms remained relatively stable, and less than one third of the study cohort experienced worsening of other symptoms.
From baseline to week 4, the most significant increases in the proportion of participants reporting their symptoms as severe were found for drowsiness and tiredness.At baseline, drowsiness was reported as severe by 9% of participants; that proportion increased to 24% at week 4. Similarly, the proportion of participants reporting tiredness as severe increased by 10%, to 33% at week 4 from 23% at baseline.
Table iii shows symptom intensity scores from the retrospective pre-dexamethasone results, from the time of consultation, and from week 4. Overall, a modest absolute change in mean symptom scores was observed.The symptoms often attributed to steroids-weight gain and sleep-were more problematic at the time of consultation; they declined at week 4.However, fatigue and ankle swelling appear to have increased slightly despite our general   practice of tapering steroids on completion of radiotherapy.We observed improvements in sleep and vomiting, but worsening of leg weakness, which might reflect steroid myopathy.No change in dizziness was observed.The section of the questionnaire in which patients were asked to select their 3 most bothersome symptoms was either incomplete or completed incorrectly by many participants.Many patients selected the 3 symptoms that were the most bothersome at each time point, rather than monitoring the same symptoms from week to week.

Correlation with Other Measures
Table iv presents correlations of the total bmsc score with change in other functional measures.Overall, correlations between changes in mrc neurologic status, ecog performance status, EQ-5D, somc, and bmsc were low.However, the face validity of the bmsc was demonstrated by its consistency with the other tools in terms of direction of symptom change, as indicated by the positive correlation coefficients.

DISCUSSION
In this validation study of the bmsc, 137 patients with brain metastases were assessed at the time of consultation for wbrt until 4 weeks after treatment.The findings confirm the feasibility of the tool to establish the symptom profile of patients at the time of presentation and to capture a wide range of symptoms.The bmsc is sensitive to clinical change and shows generally good test-retest reliability.Validation of the bmsc using correlations with mrc neurologic status, ecog performance status, somc, and EQ-5D in terms of symptom severity was suboptimal, although consistency in the direction of change was observed.
Patient-reported outcomes are critical for the evaluation of interventions intended to provide symptom relief, and they are increasingly being used in radiation oncology and in other clinical cancer treatment settings 13 , showing improved symptom control, increased supportive care measures, and patient satisfaction 14 .Understanding the symptoms experienced by patients with brain metastases can benefit from this approach.Contemporary clinical trials in brain metastases tend to focus on neurocognitive and quality-of-life change.Although the importance of those domains cannot be disputed, we often have difficulty explaining to patients how our treatments will directly affect their presenting or index symptoms.For example, a 0.2 change in a quality-of-life scale is not as meaningful to a patient as being told that they can expect a 20% improvement in their headache.Patient-reported symptom checklists (or scales) can fill that gap.
There are a number of design considerations in developing a patient-reported outcomes checklist.The advantage of a disease-specific checklist over a generic questionnaire is the high precision achieved through the narrow scope and the responsiveness to small, clinically relevant changes 15 .Compared with the singular construct of pain, symptoms related to brain metastases can be complex 16 .As a result, the use of multi-item scales becomes important.The use of several complementary items to describe particular symptom domains or different aspects of a particular concept can contribute to the precision and comprehensiveness of the scale 17 .The bmsc attempts to incorporate 3 different forms of measurement to provide response assessment in several dimensions.It includes intensity scores (as used in pain scales), direction of change (improved, stable, worsened), and how much the symptom is bothersome (as used in quality-of-life scales).
The esas, a widely used patient-reported general symptom checklist was used by Chow et al. 18 after wbrt for brain metastases.The tool was sensitive in detecting significant deteriorations in fatigue, drowsiness, and appetite.However, it is nonspecific to brain metastases and was unable to address many specific neurologic symptoms.Our group previously used a 17-item checklist (and pre-specified response criteria) in a prospective study, highlighting the modest benefit with wbrt, with only 14 of 75 patients showing improvement in their presenting neurologic symptoms at 1 month 19 .The tool was translated and adopted by investigators in Poland, who reported on mean scores by symptom at 1 month after wbrt 20 .
The MD Anderson Symptom Inventory-Brain Tumor Module (mdasi-bt) was undergoing development at about the same time as the bmsc.Table v compares the mdasi-bt with the bmsc.The mdasi-bt was first developed as an instrument to measure both neurologic and cancer-related symptoms in brain tumour patients; it was validated in a cohort of 124 patients with brain metastases 21 .It consisted of 22 items, scored on a 0-10 numeric scale, that rate a patient's health status within the preceding 24 hours.The items included 13 general cancer symptoms, 6 interference items, and 9 symptoms specific to brain tumours.The items were finalized using factor analysis and cluster analysis to arrive at the subscales.The bmsc consists of 18 symptoms that were selected based on the most frequently used descriptors offered by patients and collated in our previous study.We deleted the single item "seizure" in the final version, because seizures are either present or absent (rather than measured by intensity), are seldom ongoing with effective anticonvulsants, and can be easily added as a free-text item if needed.The structure of the questionnaire is designed to be a natural extension of the esas, a tool that has been widely implemented in clinical practice in many populations.
Validation of tools can be achieved through anchorbased correlation with other measures and by distributionbased methods, in which validity is based on the magnitude and range of the treatment effect 13 .Both methods were used in the present validation study of the bmsc.Although our study demonstrated reliability, correlation with related scales could benefit from further work.Several factors might have affected some of the lower correlation scores.The wording and layout of the questions could have affected how well respondents understood the meaning of each item on the scale.For example, it might not have been clear to the patients whether the symptom "appetite/ weight gain" implied an increased or decreased appetite, or how to distinguish between tiredness, drowsiness, and difficulty sleeping.That phenomenon was observed for some esas items 14 , and a revised version, the esas-r, which includes short and simple definitions of each symptom, was associated with a significantly improved ability for patients to complete the questionnaire accurately 22 .In the present study, the inclusion of patients with cognitive impairment (24% of the sample) could have affected the accurate completion of the bmsc.Ongoing development and refinement will no doubt improve the performance of the bmsc.
An interesting observation from our data is the adequacy of proxy ratings by caregivers, with good correlation observed for symptoms such as weakness and visual problems, but poor correlation observed for neurocognitive items such as confusion (Figure 2).Although patient-reported scores are clearly the "gold standard" in understanding subjective experiences, it is anticipated that, at certain times, patients with brain metastases might be unable to accurately report their own symptoms.Recognizing the items for which proxy ratings by caregivers can be used will be valuable in understanding treatment effects.In the case of patients with a poor performance status, caregiver ratings on the mdasi-bt were least equivalent for 7 specific brain tumour module items (pain, disturbed sleep, distress, problem remembering things, lack of appetite, drowsiness, and sadness) 23 .Those items included (among others) difficulty understanding and difficulty concentrating, with higher scores consistently being assigned to those items by the caregivers than by the patients themselves.
With respect to time trends in the symptom profile for patients with brain metastases, the bmsc was found to effectively document symptom changes.The symptoms reported on the bmsc were mostly mild to moderate.From the retrospective pre-dexamethasone to the baseline time point, headaches and vomiting showed the greatest improvement (decrease) in mean symptom scores, and weight gain showed the greatest worsening (increase).During the 1-month period after wbrt, tiredness and drowsiness showed the highest mean symptom scores and were reported by the greatest number of patients.Overall, headache appears to be the symptom most effectively palliated; speech and confusion were the least effectively palliated (Figure 4).In understanding the benefit of wbrt, an obvious confounder has been the routine co-administration of corticosteroids 24,25 .When the baseline data were first examined in relation to the symptom scores at week 4, small absolute changes were observed.However, when the retrospective pre-dexamethasone questionnaires were compared with the week 4 data, a more significant overall improvement in certain symptom scores was seen, indicating that some of the improvement was attributable to the steroid effect.A systematic review of the effect of corticosteroids in the management of brain metastases with wbrt found that the descriptions of steroid use in randomized controlled trials were non-uniform and provided few details 25 .Our group previously examined the feasibility of using a standardized tapering schedule, but observed a high rate of deviation, especially in patients with infratentorial disease, suggesting that an individualized tapering strategy is likely required 26 .The use of composite endpoints and the stratification of change scores by dexamethasone dose level are strategies worth exploring.
Our study has a number of limitations.The attrition rate at 1 month was high, although it is consistent with other studies in the literature in this population.Attempts were made to contact patients by telephone at home, but because of the limited prognosis and often rapid deterioration of these patients, many were lost to follow-up.The caregiver participation rate was also modest at 35%.To better understand and validate proxy measures, methods to improve caregiver recruitment are worth considering.The poor completion of index symptom monitoring each week necessitated the use of intensity scoring as the primary method of symptom evaluation.Intensity scores are limited by recall bias and response shift; the use of retrospective symptom evaluation could have amplified those biases.Furthermore, patients with cognitive impairment are expected to have difficulty providing reliable measures, and bothersome symptoms can be affected by the patient's overall performance status.For example, a patient who is nonambulatory might not report the symptom of dizziness as bothersome.Another limitation is the generalized design of the other measures against which the bmsc was validated.Future work will define whether the bmsc can be modified and validated for specific symptoms against more sensitive cognitive assessment tools.

CONCLUSIONS
This first analysis of the bmsc as a symptom assessment tool for patients with brain metastases undergoing wbrt has demonstrated preliminary feasibility, reliability, and sensitivity to change.Future work is planned to better define its validity and clinical utility and to improve the clarity of its instructions and items.However, the bmsc shows promise as a tool to document and follow the substantial symptom burden of patients with brain metastases receiving wbrt.

FIGURE 2 (
FIGURE 2 (A) Median intraclass correlation (ICC) scores at baseline for patients and proxy caregivers (n = 38).(B) Median ICC scores at end of radiotherapy for patients and proxy caregivers (n = 28).

FIGURE 4
FIGURE 4 Symptom change from baseline to week 4.

TABLE II
Descriptive characteristics of the study patients

TABLE III
Symptom intensity scores a a Rated on a 0-10 scale and based on symptoms experienced in the preceding 24 hours.e244 Current Oncology, Vol. 23, No. 3, June 2016 © 2016 Multimed Inc.

TABLE V
The Brain Metastases Symptom Checklist (BMSC) compared with the MD Anderson Symptom Inventory-Brain Tumor (MDASI-BT) instrument a Seizures were included in the original design of the BMSC.However, this item was removed in a subsequent iteration because patients found seizures difficult to describe on an intensity scale.Patients were still able to report seizures in the free-text section of the questionnaire.e246 Current Oncology, Vol. 23, No. 3, June 2016 © 2016 Multimed Inc.