Implementation and Evaluation of Patient-Reported Outcome Measures in Paediatric Solid Organ Transplantation
Principal Investigator: S. Anthony
Co-Investigators: A. Goldberg, D. Hartell, J. Mitchell, J. Stinson, L. Hamiwka, L. West, M. Barwick, M. Brudno, M. Dieude, M. Solomon, N. Nalli, R. Parekh, S. Urschel, T. Blydt-Hansen, U. Allen.
Site Leads: A. Goldberg, T. Blydt-Hansen, L. Hamiwka, S. Urschel.
Patient Partners: J. Mitchell, K. Sutherland, S. Baig, S. Boucher, S. Logan.
The success of paediatric solid organ transplantation can no longer be defined on the basis of objective clinical outcomes alone. Subjective evaluation of outcomes of disease, medical care and treatment from the patient’s perspective has become increasingly important. Patient-reported outcome measures (PROMs) play a vital role in addressing the burden of disease or/and treatment from the patient perspective. PROMs are defined as: “any report of the patient’s health condition that comes directly from the patient, without interpretation of the patient’s response by a clinician or anyone else”. They have the ability to engage patients meaningfully, to give them a voice in their healthcare, and to capture their varied experience and attitudes.
PHASE 1: Systematic Review
Aim: Subjective evaluation of disease outcomes, medical care and treatment from the patient’s perspective has become increasingly important. Patient-reported outcome measures (PROMs) play a significant role in engaging patients in their healthcare and capturing their experiences. PROMs are defined as: “any report of the patient’s health condition that comes directly from the patient, without interpretation of the patient’s response by a clinician or anyone else”. The aim of this review was to identify PROMs used in the field of paediatric solid organ transplantation and to analyse their health concepts, domains and psychometric properties.
Methods: A systematic review of English language articles was performed on key databases (i.e., MEDLINE, EMBASE, CINAHL, PsychINFO) to identify publications utilizing PROMs in a paediatric transplant population. Screening and study selection were undertaken by two independent researchers. Data abstraction was completed by one reviewer and reviewed by a second. Methodological quality and evidence of psychometric properties of instruments validated within paediatric transplantation were assessed using the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) checklist. The systematic review was prospectively registered with PROSPERO.
Results: The search yielded 3,670 articles with a final data set of 62 articles discussing 47 different PROMs. The PedsQL – Generic Core Scales (n=25) was the most frequently used PROM. PROMs were classified as either generic (n=42) or disease specific (n=5) and were categorized according to their health concept: quality of life (n=16), psychological (n=23), physical (n=4), or social functioning (n=3), adherence behaviour (n=2). Seven of the 47 PROMs were validated in a paediatric transplant population, with limited evidence for their psychometric properties.
Conclusions: This systematic review identified PROMs used in the field of paediatric solid organ transplantation. With limited evidence of the psychometric properties of PROMs within the specific patient population, more extensive validation and evaluation is warranted. This review helps guide methodological decision-making processes necessary for integration of PROMs into clinical care.
Knowledge Translation: Patient-Reported Outcome Measures Within Pediatric Solid Organ Transplantation: A Systematic Review. Anthony SJ, Stinson H, Lazor T, Young K, Hundert A, Santana MJ, Stinson JN, West LJ. Pediatric Transplantation, 2019;23(6):e13518.
Funded by: The International Society for Heart and Lung Transplantation
PHASE 2: Interviews with Key Stakeholders
Aim: This study explores the perspectives of pediatric solid organ transplant patients, caregivers and healthcare providers on implementing PROMs into clinical practice.
Methods: Semi-structured interviews were conducted with participants from five Canadian pediatric transplant centres. Maximum variation sampling was used to provide concurring and confirming data, and ensure saturation. An iterative coding process was applied with constant comparative data analysis to identify themes.
Results: A total of 20 patients, 22 caregivers and 21 healthcare providers participated. Nearly all participants (n = 59, 95%) were supportive of implementing PROMs with the primary goals to (1) enhance patient-provider communication – “Opens conversations”; “[Patients] state what they’re feeling”, and (2) increase patient engagement – “Like you’re part of the team”. Participants discussed the potential impact of PROMs as the provision of preventative – “You’ll be proactive”, and holistic care – “You cannot separate medical from other aspects. [Patients] are struggling from a mental health perspective”. Participants identified selected PROMs, the PedsQLTM Generic Core Scales – “Covers aspects I wouldn’t usually include in my assessment” and the PedsQLTM Transplant Module – “More related to transplant patients”. Recommendations included 1) PROM data collection for patients 8 to 10 years of age and older, 2) remote administration and completion of electronic PROMs, prior to clinical appointments, and 3) visual representation of PROM data.
Conclusions: Findings highlight support for the implementation of PROMs into pediatric solid organ transplantation. Future research is needed to develop implementation strategies to effectively integrate PROMs into clinical workflow and assess the impact on patient health outcomes.
Knowledge Translation: Patient-Reported Outcome Measures in Pediatric Solid Organ Transplantation: Exploring Stakeholder Perspectives on Clinical Implementation through Qualitative Description. Anthony SJ, Young K, Pol SJ, Selkirk E, Blydt-Hansen T, Boucher S, Goldberg A, Hamiwka L, Haverman L, Mitchell J, Urschel S, Santana M, Stinson J, Sutherland K, West LJ. Quality of Life Research. 2021. doi: 10.1007/s11136-020-02743-8.
Funded by: The Canadian Institutes of Health Research and the Canadian Donation and Transplantation Research Program
PHASE 3: Consensus Workshop
Aim: While the promise of integrating PROMs into clinical practice is large, key operational questions must be addressed before their use can be widely disseminated. The aim of this consensus workshop was to employ patient-oriented research to inform methodological and practical decisions necessary to guide the systematic and effective integration of PROMs into ‘real-world’ patient care settings.
Methods: A two-day consensus workshop was hosted in December 2018 in Toronto to advance the program of research exploring the integration of PROMs into pediatric transplant clinical practice. Workshop proceedings were informed by results of previous study phases (Phases 1 and 2).
Results: Twenty-five leading experts in pediatric solid organ transplantation, PROMs, implementation science and computational medicine, including patients, caregivers, healthcare providers, researchers and administrators from across Canada attended the workshop. Workshop objectives were accomplished including: (1) establishment of consensus on key methodological and operational decisions around the implementation of PROMs into practice (e.g. which standardized PROMS to utilize, the setting and timing of assessment, as well as the mode for administering PROMs); (2) creation of a research plan to design and develop an electronic PROM platform (Phase 4), evaluate the usability and implementation of a PROM platform and its impact on practice protocol and patient outcomes (Phases 5 & 6); and (3) generation of a knowledge translation strategy to disseminate research findings (e.g. newsletter, peer-reviewed publications, website posting, national and international presentations).
Conclusions: The consensus workshop captured attendees’ perspectives around practice and systems-based facilitators and barriers to implementing PROMs and was instrumental to ensure our research continues to be relevant and meaningful to stakeholders.
A few comments from patient partner participants: “I found the workshop to be engaging, informative, and productive. As a patient, I was a bit nervous prior to the workshop about whether or not I would feel comfortable to share my input, but the environment and space that your team created eased my worries”; “Thank you for including the patient’s voice!!! I just wanted to say how impressed and encouraged I am by the work you are doing. It is such a vital and necessary part of the continuum of care”.
Knowledge Translation: Anthony Lab Newsletter – Winter 2020
Funded by: The Canadian Institutes of Health Research: Strategy for Patient-Oriented Research Collaboration Grant
PHASE 4 & 5: Platform Creation and Usability Testing
Funded by: Health Canada, SickKids Transplant & Regenerative Medicine Centre, Canadian Donation and Transplantation Research Program & Astellas Pharma Canada.
Aim: The current study aims to create Voxe, a paediatric user-centred electronic PROM platform, by engaging patients and healthcare providers throughout the design and development process.
Methods: The creation of Voxe will occur over two phases that build on previous research. The design phase (Phase 4) employs a ‘user-centric’ approach to identify end-users’ needs and iteratively refine the look and layout of Voxe to meet these needs. Transplant recipients, aged 10–17, and healthcare providers will participate in three rounds of testing (24 participants total). Participants will: (1) complete task-based activities (outcomes—effectiveness and efficiency), (2) complete questionnaires (outcome—satisfaction) and (3) participate in a semi structured interview. The following phase (Phase 5) involves software development and Voxe usability testing. Transplant recipients, aged 8–17, and healthcare providers will participate in four rounds of iterative testing (24–40 participants total). The think-aloud technique will be employed, and participants will describe their thoughts and feelings while interacting with a Voxe prototype. Participants will: (1) log into Voxe and complete tasks (outcomes—time on task, successful task completion, frequency of critical and non-critical errors and error-free rate), (2) complete questionnaires (outcome—satisfaction) and (3) participate in a semi structured interview.
Results: Pending. Findings will lead to the creation and launch of a user-centred electronic PROM platform. This research is critical to answering methodological and operational questions to inform Voxe implementation in paediatric clinical settings and facilitate PROM data collection.
Knowledge Translation: The Creation of an Electronic Patient-Reported Outcome Measure Platform Voxe: A Mixed Methods Study Protocol in Pediatric Solid Organ Transplantation. Anthony SJ, Pol SJ, Lin J, Barwick M, Brudno M, Manase D, Parekh R, Silva A & Stinson JN. BMJ Open. 2021;11:e053119. doi: 10.1136/bmjopen-2021-053119.
User-Centered Design and Usability of Voxe as a Pediatric Electronic Patient-Reported Outcome Measure Platform: Mixed Methods Evaluation Study. Anthony SJ, Pol SJ, Selkirk E, Matthiesen A, Klaassen RK, Manase D, Silva A, Barwick M, Stinson J, Damer A, Ayibiowu M, Dong SX, Oreskovich S & Brudno M. Journal of Medical Internet Research Human Factors. 2024;09:11:e57984.
Design and Usability of Voxe Infographic
Funded by: Health Canada, SickKids Transplant & Regenerative Medicine Centre, Canadian Donation and Transplantation Research Program & Astellas Pharma Canada.
PHASE 6: Implementation-Effectiveness Trial
Funded by: Health Canada, SickKids Transplant & Regenerative Medicine Centre, Canadian Donation and Transplantation Research Program & Astellas Pharma Canada.