Abstract
Purpose: The Canadian Network for Observational Drug Effect Studies (CNODES), a network of pharmacoepidemiologists and other researchers from seven provincial sites, provides evidence on the benefits and risks of drugs used by Canadians. The Knowledge Translation Team, one of CNODES' four main teams, evaluates the impact of its efforts using an iterative and emergent approach. This article shares key lessons from early evaluation phases, including identifying stakeholders and their evaluation needs, choosing evaluation theories and approaches, and developing evaluation questions, designs, and methods appropriate for the CNODES context. Methods: Stakeholder analysis was conducted using documentary analysis to determine key contextual factors and research evidence needs of decision maker partners and other stakeholders. Selected theories and frameworks from the evaluation and knowledge translation literature informed decisions about evaluation design and implementation. A developmental approach to evaluation was deemed appropriate due to the innovative, complex, and ever-changing context. Results: A theory of change, logic model, and potential evaluation questions were developed, informed by the stakeholder analysis. Early indicators of program impact (citation metrics, alternative metrics) have been documented; efforts to collect data on additional indicators are ongoing. Conclusion: A flexible, iterative, and emergent evaluation approach allows the Knowledge Translation Team to apply lessons learned from completed projects to ongoing research projects, adapt its approaches based on stakeholder needs, document successes, and be accountable to funders/stakeholders. This evaluation approach may be useful for other international pharmacoepidemiology research networks planning and implementing evaluations of similarly complex, multistakeholder initiatives that are subject to constant change.
| Original language | English |
|---|---|
| Pages (from-to) | 8-25 |
| Number of pages | 18 |
| Journal | Pharmacoepidemiology and Drug Safety |
| Volume | 29 |
| Issue number | S1 |
| DOIs | |
| Publication status | Published - Jan 1 2020 |
Bibliographical note
Funding Information:The Canadian Network for Observational Drug Effect Studies (CNODES), a collaborating center of the Drug Safety and Effectiveness Network (DSEN), is funded by the Canadian Institutes of Health Research (Grant Number DSE‐111845 and DSE‐146021). I.S., R.T., and K.K. have received salary support, in part, from CIHR for the CNODES project; Nova Scotia Health Research Foundation (NSHRF) REAL Evaluation Services (NC, DW) have been contracted to consult on evaluation planning for CNODES. The authors wish to acknowledge Kaireen Chaytor, Heidi Staples, and Elaine Burland for their careful review and helpful comments on the manuscript, as well as Leah Canning for providing clarification on the role of DSEN.
Funding Information:
The Canadian Network for Observational Drug Effect Studies (CNODES), a collaborating center of the Drug Safety and Effectiveness Network (DSEN), is funded by the Canadian Institutes of Health Research (Grant Number DSE-111845 and DSE-146021). I.S., R.T., and K.K. have received salary support, in part, from CIHR for the CNODES project; Nova Scotia Health Research Foundation (NSHRF) REAL Evaluation Services (NC, DW) have been contracted to consult on evaluation planning for CNODES. The authors wish to acknowledge Kaireen Chaytor, Heidi Staples, and Elaine Burland for their careful review and helpful comments on the manuscript, as well as Leah Canning for providing clarification on the role of DSEN. Parts of this research have been discussed at the following conferences/meetings: Sketris IS. Canadian Network for Observational Drug Effect Studies: A research program to rapidly respond to targeted drug safety and effectiveness queries. 2016 International School on Research Impact Assessment (ISRIA), Melbourne, Australia. September 20, 2016. Sketris IS, Carter N, Traynor R, Watts D, Kelly K. Exploring a developmental evaluation approach for measuring impact of a complex drug safety and effectiveness research network. 21st Annual Performance and Planning Exchange Symposium (PPX), Ottawa, Canada. May 23?25, 2017. Proof of Concept Evaluation Is the approach congruent with the goals and objectives of the broader CNODES initiative? Is the approach congruent with its intended purposes and rationale? Process Evaluation To what extent is knowledge being produced, translated, and mobilized as intended? To what extent is CNODES' knowledge translation operating in an effective and efficient manner? Outcome Evaluation Is CNODES' knowledge translation contributing to increased access to relevant knowledge for stakeholders? Is CNODES' knowledge translation contributing to increased use of knowledge to inform decision-making?
Funding Information:
Working with the Nova Scotia Health Research Foundation (NSHRF) and Credentialed Evaluators¶¶¶ by the Canadian Evaluation Society### provided researchers the support necessary for managing this highly complex evaluation.
Funding Information:
Sentinel is funded by the US Food and Drug Administration (FDA) to study the safety of medical products. Sentinel data partners include private insurers, the Center for Medicare and Medicaid Services, the Veterans Health Administration, and the Department of Defense. Sentinel uses a common data model.61,62
Publisher Copyright:
© 2019 The Authors. Pharmacoepidemiology and Drug Safety Published by John Wiley & Sons Ltd.
ASJC Scopus Subject Areas
- Epidemiology
- Pharmacology (medical)