AcademyHealth’s mission is to improve health and the performance of the health system by supporting the production and use of evidence to inform policy and practice. When our President and CEO, Lisa Simpson, talks about our mission, she often underscores that second part: the use of evidence in policy and practice. It’s not enough to produce evidence and let that evidence sit on the shelf. For it to be most valuable, it needs to be used.
Understanding the uptake and use of health services research is a challenging endeavor for funders and researchers alike, with challenges including the time lag between research funding and research impact, difficulty attributing impact directly to a particular study, and variation in how the field defines impact, among other challenges. AcademyHealth has engaged in a number of activities to better document the impact of health services research, particularly on health policy, and to help both funders and researchers consider ways to more effectively measure and communicate impact.
Public Funder Perspectives on Research Impact Assessment
Public funders are not immune to the challenges in documenting and communicating the impact of research investments. Yet, perhaps more so than private funders, they face the added pressure of demonstrating accountability for their investments, particularly in challenging fiscal environments with limited government funding available for research.
To delve into the unique perspective of public funders, AcademyHealth convened representatives from 10 of the major public funders of health services research to discuss effective ways to systematically capture research impact, communicate relevant insights and expectations to grantees and other stakeholders, and to direct limited funds to the investments and research activities that yield the highest return.
At the meeting, participants discussed their approaches to evaluating the impact of their research investments and activities, including advancing knowledge but also impact beyond the traditional academic outputs and outcomes, and two things became clear. First, there is considerable diversity across the public funders in terms of their specific missions, funding structures, priority populations, research activities, audiences, etc. And second, their current approaches to research impact assessment are truly a combination of art and science. Some agencies are statutorily required to assess their effectiveness. Others apply a combination of required, regular grant reporting, ad hoc program evaluations, and formal and informal qualitative information gathering to assess their investments. Some have developed and adopted impact frameworks, whereas others are only beginning to formalize their processes.
Public Funder Priorities for Research Impact Assessment
A few key priorities emerged from the discussion in terms of what the public funders view as most important for carrying out impact assessment activities. The first is to clearly define the agency’s key stakeholders. Participants noted that specifying their audiences and who they serve makes it much easier to identify changes that have led to an outcome that the stakeholder audiences care about.
Second, participants agreed that funders undertaking research assessment activities need to be clear on why they are assessing their research investments. One commonly accepted framework refers to the 4 “A’s”: advocacy, accountability, analysis, and allocation. The purpose of the assessment should drive the methodological or evaluation approach.
Participants agreed that given the nonlinear ways in which research may impact health policy and practice, qualitative methods are needed to tell the full story. Some public funders regularly create case studies, others solicit narrative information via grantee reporting, and most also incorporate informal anecdotes that inform their understanding of how their research is being used.
But participants acknowledged that beyond impact measurement challenges, current incentives do not often support the type of research activities that are most likely to lead to impact. Participants questioned whether the field of health services research needs a broader culture change or paradigm shift in terms of the type of research activities that are supported and how the field thinks about the impact of research, beyond publications.
Considerations for Researchers
While wide-sweeping culture change may take years to materialize, there are things that researchers can do now to better communicate the impact of their research and to increase their impact. Below I offer three areas of opportunity for researchers to consider: measurement, translation and third party knowledge brokers to help move evidence into action.
Use measurement as a tool to promote impact
If you’re not already tracking the outputs, outcomes, and impact resulting from your work, it’s not too late to start. Colleagues and I published this paper on considerations for measuring the impact of policy-relevant research, which includes examples of things you might want to track if you aim to influence health policy. Have you briefed a policymaker, whether formally or through informal email or phone communication? Have you spoken with a reporter about your work? Have you made an important connection at a conference? Keep a record! And don’t let perfect be the enemy of the good. You don’t have to get everything, but something is always better than nothing. And don’t forget to share these insights with your funder. Even if the ultimate impact you hope to achieve may take years or decades to develop, what can you share with your funder now to indicate you are on the right track?
Make sure your research is ready to fly off the shelf
We heard from the public funder participants that given the complexity of much of the research and evaluations they support, the research needs to be translated into digestible formats before it is shared with their key stakeholders, which could include members of Congress, providers, or the general public. If you plan to release findings in a peer-reviewed journal article, think about repackaging those findings into different formats for a range of audiences, so that it’s end-user ready. For example, could you draft a one-page summary or short issue brief on your study, or create a visual abstract of your research to share on Twitter? Or perhaps an infographic to share with busy policymakers to drive home your key finding?
Consider adding knowledge brokers or translators to your research team
Not sure how to repackage your findings or write them for lay audiences? Could you include a knowledge broker or communications expert on your research team? It might be a good idea to reach out to your institution’s communications team or ask for support from your funder. You might also consider getting some training for yourself. For example, AcademyHealth staff is offering a pre-conference seminar on building your own communications plan at our Annual Research Meeting on June 1. Our Translation and Dissemination Institute has also published work on how evidence can be made more responsive to policymakers. Knowledge translation is an acquired skillset, but it’s a prerequisite for getting your research understood and used outside of academia.
Funding for this conference was made possible by 1 R13 HS025877-01 from the Agency for Healthcare Research and Quality (AHRQ). The views expressed do not necessarily reflect the official policies of the Department of Health and Human Services; nor does mention of trade names, commercial practices, or organizations imply endorsement by the U.S. Government.