Many of you have probably submitted an abstract to AcademyHealth’s Annual Research Meeting’s (ARM) Call for Abstracts (CFA), and then patiently waited for, what probably seemed like forever, to receive your acceptance notification. What you may not know, is that the abstract review process involves many steps, each with its own mini steps, which kicks off about ten months prior to the actual ARM taking place. So let’s start at the beginning….
Each August AcademyHealth holds a call for volunteer reviewers. Abstract reviewers play a critical role in developing the program for the ARM around our 21 ARM themes. As part of our diversity, equity, and inclusion goals, AcademyHealth collects self-reported data on gender, ethnicity, and racial identities when soliciting volunteer reviewers. We also strive to ensure that each review committee includes students, Interest Group (IG) members, and patient collaborators, as well as diversity across discipline, type of organization, and career level. For ARM 2024, we used over 750 of the almost 900 volunteers for the original CFA abstract review.
Around the same time, we also begin soliciting recommendations for theme co-leaders. Each of the 21 themes is led by two experts in the field who work together to make final decisions for the sessions in that topic area. Co-leader recommendations come from the ARM Planning Committee, AcademyHealth leadership, and of course past theme leaders who are able to offer up new names from their field for consideration. In addition to the diversity requirements, additional criteria for theme co-leaders include having one senior level and one junior person paired up, and for seven of the themes, having one of the co-leads be either a Community Based Organization (CBO) or patient partner so that these important perspectives are included in the program planning.
Once the theme co-leaders have been identified, they are asked to update the Areas of Focus for their theme. The Areas of Focus are used to help submitters determine which theme is the most appropriate for their submission. Co-leaders are also given the opportunity to invite additional reviewers from their network, where they feel that the committee would benefit from more expertise and/or diversity.
The CFA typically opens in mid-November and stays open for about two months, closing in early/mid-January. The ARM offers five submission types during the original CFA, each with their own criteria:
- Call for Abstracts Sessions
- Panels on Critical and Emerging Issues in HSR
- Policy Roundtables
- Research Panels
- Student Posters
In recent years, ARM has received approximately 2500 abstracts across these five submission types. After some initial cleanup removing duplicate and incomplete submissions, the abstracts are randomly assigned to reviewers by theme, and the peer-review process begins. Each abstract is assigned to six reviewers to ensure reliability in the overall average score to help theme leaders make final decisions. All individual abstracts and research panel submissions undergo blind peer review. Policy Roundtables and Panels on Critical and Emerging Issues in Health Services Research (HSR) submissions are not blinded. Reviewers are asked to abstain from any abstracts that they feel they may have a conflict of interest in scoring and are also given the opportunity to indicate interest in chairing a session on certain topics.
At the end of the grading period, which takes us to early-February, AcademyHealth shares reports with average scores and standard deviation, along with reviewer comments and a variety of other resources, to assist the co-leaders in making final decisions. Each theme is allotted a specific number of session slots on the program, proportionate to the number of submissions it received that year. Theme leaders are then charged with the all-important task of deciding:
- CFA abstracts to accept for podium
- Panels to accept
- CFA abstracts to accept for poster
- Abstracts and panels to be rejected
- Topical groupings for the CFA podium abstracts (including a session title and description)
- Chairs/Discussants for each CFA session
- Best of ARM nomination
- Best Poster nomination
As you can imagine, this is a challenging and complex undertaking, which requires the co-leaders to work together to make tough decisions, while considering many factors and still honoring the peer review process.
Once all decisions are made, AcademyHealth collects this information and tackles the data entry, making sure that every abstract, session, and speaker are added to the recipe at just the right time. With approximately 2/3 of the overall ARM program coming from the CFA, it is extremely important to review the proposed agenda once everything is in place, to look for duplication as well as any holes in the program. Abstract notifications are sent out in mid-March, and we then work to help presenters accept intent, transfer roles, and adjust as needed.
The ARM agenda is finished off with a pinch of Career Development sessions, a dash of Methods sessions, and a smidgeon of plenaries and other invited special sessions. And then, VOILA! The final program is made public in mid-April for everyone to peruse, and plan their attendance at the upcoming ARM.
We can’t wait to see you in Baltimore June 29-July 2, to enjoy the remarkable 2024 programming that so many of you have helped put together. Register here by April 29, 2024 for a discounted rate.