Skip to main contentLink to External Link Policy
Información en Español
National Center for Complementary and Integrative Health

NCCIH Policy: Study Accrual and Retention for Human Subject Research

Background

Timely and robust accrual is an essential aspect for ensuring the ethical conduct of clinical research. When clinical studies fall short of meeting recruitment targets and are unable to address their study aims as a result, participants are exposed to undue risk without generating a benefit to science. Such underaccrued studies pose an ethical dilemma even when the study interventions and procedures are evaluated as minimal risk to participants. NCCIH exercises stewardship of public funds by monitoring the accrual and retention benchmarks set by the study team throughout the grant lifecycle. NCCIH works closely with study teams to address and intervene early in clinical studies in which accrual and retention challenges arise.

Guidance

Establishing Accrual Benchmarks in the Study Accrual Plan

For each clinical study covered by this policy, the study investigator in consultation with the NCCIH PO shall agree to benchmarks for monitoring accrual and retention as outlined in the approved SARP. The approved SARP shall provide the anticipated date of the first participant enrollment and stipulate the number of participants to be enrolled and randomized (if applicable) by calendar month for the active award period. While some studies expect to accrue in a linear fashion, accrual benchmarks shall make allowance for studies with irregular enrollment patterns based on the specific study design or ease of recruiting select study populations. From the outset, some studies may anticipate a higher-than-average expected lost to followup rate due to the nature of the intervention or the targeted study population. Under these select circumstances, the SARP may reflect retention benchmarks that exceed the threshold standards set forth by this policy. The NCCIH PO will formally review the SARP and approve it before initiation of the clinical research study. NCCIH may negotiate with the study team to ensure that the approved SARP maximally optimizes the success of the planned study.

Applicability

This policy applies to NCCIH funded grants that involve clinical research.

Changes to the Approved Study Accrual and Retention Plan

For cause with strong justification (for example, suspension of enrollment for a particular unforeseen circumstance), investigators may petition NCCIH for changes in projected accrual benchmarks and accrual timetable after clinical study initiation. On rare occasions, a clinical study or an individual site of a multi-site clinical study may accrue more rapidly than outlined in the approved SARP. If actual accrual exceeds planned benchmarks as outlined in the approved SARP by more than 150 percent for a given reporting period, the NCCIH program official shall be notified. The NCCIH program official will decide whether an alteration to the approved SARP is warranted. NCCIH’s decision to accept the revised accrual and retention benchmarks will depend on the maintenance of the study’s statistical power and study completion within the time and resources available.

Monitoring Study Accrual and Retention Plan Performance

NCCIH will formally review human subject accrual during the study according to the terms and conditions outlined in the Notice of Award. NCCIH requires updates on participant accrual and retention at least every 4 months while active enrollment and data collection is ongoing. Based on the updated accrual and retention data provided by the study team, NCCIH will compare actual accrual and retention performance to established benchmarks by corresponding timeframe as outlined in the SARP.

Review Criteria Prompting NCCIH Action

Accrual and retention performance will be classified by performance levels. NCCIH actions are described according to the corresponding performance level. In addition to the NCCIH actions described in each of the performance levels, NCCIH may take action according to all preceding performance levels. For example, all Level 1 actions are applicable to Level 2.

On-Track Performance (studies meeting any of the following criteria):

  • If study accrual is < 150 percent and ≥80 percent of the benchmark for a given time point
  • If study initiation is delayed by no more than one accrual reporting period (approximately 4 months)
  • If the actual lost to followup rate is less than or equal to the expected lost to followup rate.

NCCIH Action: If study initiation has been delayed, the NCCIH PO may query the reasons for the delayed start.

Level 1 Performance (studies meeting any of the following criteria):

  • If study accrual is < 80 percent and ≥60 percent of the benchmark for a given time point
  • If study initiation is delayed by more than one accrual reporting periods (approximately 5-8 months)
  • If the actual lost to followup rate exceeds the expected lost to followup rate by up to 5 percent. For example, the expected lost to followup rate is 10 percent while the actual lost to followup rate is 15 percent.

NCCIH Action Toward Level 1 Performance: NCCIH may increase monitoring frequency and request a formal report evaluating barriers to meeting accrual benchmarks as experienced to date. Depending on the specific accrual challenges, NCCIH may also request either a screening evaluation to determine the primary reasons why potential participants do not qualify for the study/refuse participation or a retention evaluation to determine the primary reasons why consented participants are lost to followup or revoke consent. NCCIH may also ask the study team to discuss and document the relative success of current recruitment strategies. NCCIH will encourage the study team to engage their NCCIH-approved Independent Monitoring Committee/Data and Safety Monitoring Board and institutional resources in developing strategies for augmenting accrual. A biostatistician may be consulted to analyze the effect of accrual trends on the study’s power and ability to answer primary study aims.

Level 2 Performance (studies meeting any of the following criteria):

  • If study accrual is < 60 percent of the benchmark for a given time point
  • If study initiation is delayed by more than three accrual reporting periods (approximately 12−18 months)
  • If the actual lost to followup rate is between > 5 percent and < 9 percent more than the expected lost to followup rate. For example, the expected lost to followup rate is 10 percent while the actual lost to followup rate is between 16 percent and 19 percent.

NCCIH Action Toward Level 2 Performance: NCCIH may increase monitoring frequency and engage the study team in bimonthly or weekly phone conversations to assess ongoing progress and compliance with the SARP. NCCIH may request that the principal investigator (PI) and study coordinator meet on a routine basis and document discussions of accrual trends and efforts taken to address ongoing barriers. A site visit may be performed by NCCIH or contracted staff. NCCIH may also request that the study expand eligibility criteria, add additional recruitment or clinical sites, and/or propose revisions to the study design, key personnel, or primary aims. Additionally, NCCIH may request that the original SARP be revised and documentation supplied regarding enhanced communication among the study team in meeting revised accrual benchmarks. Finally, NCCIH will likely consider implementing budgetary controls (e.g., restricting funds, withholding funds not yet awarded, etc.) or request Interim Progress Reports until actual accrual reaches approved benchmarks. NCCIH may ask the study team for a formal corrective action plan.

Level 3 Performance (studies meeting any of the following criteria):

  • If study accrual remains < 60 percent of the benchmark for two or more consecutive reporting periods
  • If study initiation is delayed by more than 18 months
  • If the actual lost to followup rate exceeds the expected lost to followup rate by 10 percent or more. For example, the expected lost to followup rate is 10 percent while the actual lost to followup rate is 20 percent or more.

NCCIH Action Toward Level 3 Performance: NCCIH will request a formal Corrective Action Plan with revised accrual and retention targets and timelines or implement budgetary controls including restricting funds already awarded, withholding funds not yet awarded, extending the period of active award, and/or requesting a revised budget. If a study has not yet commenced, NCCIH may request that the proposed study not be pursued and may recommend an orderly shutdown of site activities. When ongoing efforts to stimulate accrual lack viability, NCCIH may request an orderly shutdown of the clinical study and no further data collection will be permitted.

Implementation

Before the Study Is Initiated

  1. For clinical studies that fall under this policy, the PI and authorizing business official (ABO) will receive a letter informing them of the NCCIH Policy on Study Accrual and Retention Plans. The awardee will have 30 days to provide a SARP for each clinical study proposed in the award.
  2. If necessary, the PI may engage in discussions with the NCCIH PO to craft a comprehensive and attainable SARP.
  3. The NCCIH PO will review the proposed benchmarks for appropriateness and for compatibility with the budget and specific aims, and, if appropriate, request the awardee institution to submit revised benchmarks and/or budget (same dollars, just reallocation between years) that more accurately reflect the SARP.
  4. When the SARP is finalized, the NCCIH program official will request that the PI submit the finalized SARP for countersignature from the ABO.
  5. The countersigned SARP is submitted to the NCCIH grants management officer (GMO) and copied to the NCCIH PO and the NCCIH Office of Clinical and Regulatory Affairs.
  6. The NCCIH PO sends an email to the PI indicating that the SARP has been approved.
  7. Funding may be withheld until NCCIH and the awardee institution agree on accrual and retention benchmarks. If an award is made prior to finalizing the SARP, the award will most likely be restricted to not allow for any clinical activities until the SARP has been approved.

After the Study Is Initiated

Official actions taken by NCCIH as a result of suboptimal accrual and retention performance as outlined in either the original or revised SARP will be documented and included in the permanent grant file. The PI and the ABO will receive official notification of NCCIH’s action(s) within 14 days of rendering any decision with budgetary impact.

NCCIH Decisions With Budgetary Impact

Depending on accrual and retention performance in comparison with the approved SARP, NCCIH may include special terms and conditions in the Notice of Award and/or revise the Notice of Award to address deficiencies complying with this policy. Special terms and conditions may include restricting funds already awarded, withholding funds not yet awarded, extending the period of active award, and/or requesting a revised budget, a revised SARP, or a study closeout plan. When special terms and conditions are imposed, NCCIH will notify the PI and ABO in writing of the nature of the conditions, the reason why they are being imposed, the type of corrective action needed, the time allowed for completing corrective actions, and the method for requesting reconsideration of the conditions. The NCCIH PO will review and, if necessary, discuss with the PI and ABO supportive documentation addressing the special terms and conditions. Revised awards may be issued as warranted. For more information, please consult the NIH Grants Policy Statement at grants.nih.gov/grants/policy/nihgps/HTML5/toc_test_417.htm.

Additional Resources

Please refer to the NCCIH Electronic SARP Template [60KB Word file] for more information.

Definition

Lost to followup: NCCIH considers lost to followup as a research subject who was participating in the study at a certain point in time and subsequently missed two consecutive study visits and is unresponsive to study contact, or is no longer participating in study activities.