Frequently Asked Questions (FAQs)
General Information
What is NREPP?
Who sponsors and operates NREPP?
Will SAMHSA only fund programs that appear on this site?
What is the recommended format for referencing an NREPP intervention summary?
How do I become a reviewer for NREPP?
How do I contact NREPP with a question or comment?
What are “legacy” programs and how do they differ from the programs included on NREPP?
Why do “legacy” program profiles look different from other program profiles?
Using the NREPP Site
How do I decide which intervention to use?
How do I find an intervention?
How do I get more information on an intervention I found on the site?
The NREPP Review Process
How does NREPP identify programs and practices to be rated?
Who rates the interventions reviewed and how is the review completed?
How does NREPP determine whether a program works?
What do the evidence ratings used on NREPP mean?
Does NREPP Rate Readiness for Dissemination?
Does NREPP share the intervention summary with the program developer before publishing it on the NREPP website?
Can the program developer decline publication of the intervention summary on the NREPP website?
I currently have a program on NREPP. When will it be re-reviewed?
Submitting an Intervention for Review
How do I find out if my intervention qualifies for submission?
Is there any cost to submit an intervention to NREPP?
Where can I find information on preparing a submission to NREPP?
Will every intervention that is submitted be reviewed?
Will there be another open submission period?
Can I submit multiple interventions for review?
Can interventions developed outside the United States be reviewed by NREPP?
How can I share information with SAMHSA about how I was able to successfully use program or other content from the NREPP site in my community?
General Information
What is NREPP?
The National Registry of Evidence-based Programs and Practices (NREPP) is an evidence-based repository and review system designed to provide the public with reliable information on mental health and substance abuse interventions. All interventions in the registry have met NREPP's
minimum requirements for review. The programs' effects on individual outcomes have been independently rated by certified NREPP reviewers.
The purpose of NREPP is to help people learn more about available evidence-based programs and practices and determine which of these may best meet their needs. NREPP is one way SAMHSA is working to improve access to information on evaluated interventions and reduce the lag time between creation of scientific knowledge and its practical application in the field.
back to top
Who sponsors and operates NREPP?
The National Registry of Evidence-based Programs and Practices is an initiative of the Substance Abuse and Mental Health Services Administration (SAMHSA)), the agency within the U.S. Department of Health and Human Services that leads public health efforts to advance the behavioral health of the nation. It is managed by the Center for Behavioral Health Statistics and Quality (CBHSQ). SAMHSA contracts with Development Services Group, Inc. (DSG) to coordinate the evidence review process and provide content, as well as manage the website.
back to top
Will SAMHSA only fund programs that appear on this site?
No! NREPP is a resource to assist researchers, practitioners, and policy makers in using evaluation evidence for practical decision making and program implementation. NREPP is not intended to be an exhaustive list of programs that merit investment. SAMHSA also recognizes the importance of supporting innovative approaches and practices that may not yet have extensive evidence of effectiveness. For information about these other types of programs, see the
Practice-based Evidence section of the NREPP Learning Center.
back to top
What is the recommended format for referencing an NREPP intervention summary?
NREPP recommends the following format for references:
[Program name]. Intervention summary retrieved on [date] from the Substance Abuse and Mental Health Services Administration's National Registry of Evidence-based Programs and Practices, [URL of summary]
For example:
Motivational Interviewing. Intervention summary retrieved on April 15, 2013, from the Substance Abuse and Mental Health Services Administration's National Registry of Evidence-based Programs and Practices, http://www.nrepp.samhsa.gov/ViewIntervention.aspx?id=130
back to top
How do I become a reviewer for NREPP?
NREPP reviewers are recruited and selected based on their experience and expertise in mental health promotion, mental health treatment, substance abuse prevention, substance abuse treatment, or co-occurring disorders. In addition to possessing content area expertise and a doctoral-level degree, reviewers should be familiar with current methods for evaluating prevention and treatment interventions. Potential reviewers also must complete training on the procedures and criteria they will use to rate interventions. For more information, see the
Reviewer Selection & Training page.
If you are interested in applying to become a reviewer for NREPP, please send an email to
nrepp@samhsa.hhs.gov including a description of your professional background/areas of expertise and your resume or CV.
back to top
How do I contact NREPP with a question or comment?
You can email the SAMHSA NREPP staff at
nrepp@samhsa.hhs.gov, call 1-866-436-7377, or visit the
Contact Us page to use our online contact form.
back to top
What are “legacy” programs and how do they differ from the programs included on NREPP?
NREPP has gone through several major changes since its inception in 1997. Programs were originally designated as Model, Effective, or Promising but these designations were discontinued in 2004 and a new 4-point rating system was introduced. A 4 represented the highest quality of evidence, and a 0 represented very poor quality of evidence. The ratings were based on six criteria: 1) reliability of measures, 2) validity of measures, 3) intervention fidelity, 4) missing data and attrition. 5) potential confounding variables, and 6) appropriateness of analysis. Readiness for Dissemination was also rated on a four-point scale, with 4 representing the highest rating.
Beginning in 2014 through 2015, NREPP went through a major redesign and relaunch. Beginning on November 23, 2015, all reviewed programs on NREPP refer to those that have been reviewed under the new guidelines published in the July 7, 2015
Federal Register . The 356 programs reviewed under the previous criteria are now referred to as “legacy” programs and will be re-reviewed under the new guidelines over the next 4 years (2015 through 2018).
back to top
Why do “legacy” program profiles look different from other program profiles?
When searching for a program, users will notice that search results are provided in two groups: results for programs and results for legacy programs. Under the new guidelines, NREPP classifies programs into three levels: “Effective,” “Promising” and “Ineffective” based on ratings of individual outcomes. Legacy programs were not classified but were scored from 1-4. Because the programs have been reviewed with two different sets of criteria, the program profiles look different and provide different information. These Legacy profiles will be replaced with new ones as the programs are re-reviewed. (See “What are “legacy” programs and how do they differ from the other NREPP programs?” for more on legacy programs and rating systems.)
back to top
Using the NREPP Site
How do I decide which intervention to use?
First, identify your community's needs so you can determine which type of intervention will be the most appropriate. This can be done through a needs assessment, which can take many forms (see the Planning section of the Learning Center for frameworks for
needs assessments. Next, assess your organization's capacity (e.g., financial resources, organizational commitment, community buy-in) to implement an intervention with fidelity. Once you have this information, search NREPP or another registry to find an intervention that best matches your community's needs and that can be implemented given your organization's capacity (see
for a comprehensive list of evidence-based repositories and implementation information). More information on selecting and implementing an intervention can be found in NREPP's Learning Center —
A Road Map for Implementing Evidence-Based Programs.
back to top
How do I find an intervention?
If you know the name of the intervention, you can either type it in the keyword
Search box or select it from the
View All Interventions list. From these pages on the website, you can access summaries describing each of the interventions in NREPP.
If you sign up for
e-updates, you will receive periodic announcements with links to interventions recently added to the site as well as other NREPP announcements.
If you do not know the name of a specific intervention, try entering keywords in the
Search box or use the
Advanced Search to select search criteria such as type of program, outcome categories, geographic location, age, race/ethnicity, setting, gender, special population, or rating.
back to top
How do I get more information on an intervention I found on the site?
For more information about an intervention included on this site, contact the developer or technical assistance provider directly, using the contact information provided in the “snapshot” section of the intervention summary. To assist you in gathering additional information, NREPP has compiled a list of
Questions To Ask as You Explore the Possible Use of an Intervention (PDF, 54KB).
back to top
The NREPP Review Process
How does NREPP identify programs and practices to be rated?
NREPP identifies programs for review in three ways:
- Nominations from the field: SAMHSA announces an open submission period. The open submission period generally lasts several months and allows developers, researchers, practitioners, and other interested parties to submit programs for review.
- Environmental scans: SAMHSA and NREPP contractor staff conduct environmental scans (including literature searches, focus groups, public input, and interviews) to identify potential interventions for review.
- Agency nomination: SAMHSA identifies programs and practices addressing specific agency priorities.
back to top
Who rates the interventions reviewed and how is the review completed?
NREPP contractor staff identify two independent
certified reviewers to conduct the review. All Study Reviewers are subject matter and research methodology experts who complete training and certification prior to reviewing studies. Reviewers must complete a Conflict of Interest form to confirm no conflict exists that would require recusal. Each intervention is reviewed by two reviewers. Each reviewer receives a review packet of materials and information that has been assembled by NREPP contractor staff and includes copies of studies and supporting documents. Working independently, each reviewer assesses the information presented in the packet using NREPP’s rating criteria. Each reviewer then completes an NREPP Outcome Rating Instrument. NREPP contractor staff prepare a program profile. (The only exception to this is that re-reviews of programs posted on NREPP before September 2015 may be completed by one reviewer rather than two.) For more information, see the
Review Process.
back to top
How does NREPP determine whether a program works?
Programs are pre-screened to ensure that at least one evaluation study meets the
minimum criteria for review. Programs are submitted to SAMHSA for approval to move into the
review process. Applicants are notified whether their intervention has been accepted for review or rejected.
All evaluation studies that meet minimum criteria, including being published within the past 20 years (1995 or later), and falling within a 10-year time frame—as defined by the most recent eligible article of a study—are eligible for review. NREPP certified reviewers use the NREPP Outcome Rating Instrument to review each eligible outcome in the program’s evidence base of evaluation studies. Program outcomes are reviewed on the following four dimensions:
1. Rigor. Rigor assesses the strength of the study methodology and is composed of 10 elements: 1) design/assignment, 2) intent-to-treat – original group assignment (ITT-OGA), 3) statistical precision, 4) pretest equivalence, 5) pretest adjustment, 6) analysis method, 7) other threats to internal validity, 8) measurement reliability, 9) measurement validity, and 10) attrition.
2. Effect Size. An effect size is a way to measure whether a program had an impact, how big that impact was, and whether it helped or hurt the treatment group. Effect sizes are calculated when evaluation studies provide the data needed to do so.
3. Program Fidelity. Reviewers examine the evaluation studies to determine if the program was delivered as intended and to the target population. This dimension is composed of two elements: 1) service utilization and 2) service delivery.
4. Conceptual Framework. This dimension is concerned with how clearly the components of a program are articulated. It is composed of three elements: 1) program goals, 2) program components, and 3) theory of change.
By rating the elements within these dimensions, a determination is made as to the strength of the methodology (rigor), program fidelity, effect size, and conceptual framework. Numerical values are assigned to each element in the NREPP Outcome Rating Instrument (with the exception of effect size). See
Program Review Criteria for more about these criteria. See
Review Process for more details on how these dimensions are used to form the outcome rating.
back to top
What do the evidence ratings used on NREPP mean?
Each outcome associated with a program is given a rating. The table below shows the definitions of each rating.
| Outcome Evidence Rating |
Icon |
Definition |
| Effective |
 |
The evidence base produced strong evidence of a favorable effect. |
| Promising |
 |
The evidence base produced sufficient evidence of a favorable effect. |
| Ineffective |
 |
The evidence base produced sufficient evidence of a negligible effect.
ORThe evidence base produced sufficient evidence of a possibly harmful effect. |
| Inconclusive |
* |
Limitations in the study design or a lack of effect size information preclude from reporting further on the effect. |
* Outcomes rated as “inconclusive” are not depicted with an icon and program profiles are not prepared for programs with ‘inconclusive’ ratings for every outcome.
back to top
Does NREPP Rate Readiness for Dissemination?
No. NREPP no longer requires programs and practices to have developed implementation materials, training and support resources, and quality assurance procedures.
Prior to July 7, 2015, readiness for dissemination (RFD) was rated and required for a program to be eligible for an NREPP review. These resources were rated based on their quality and assigned a “Readiness for Dissemination” score. While these materials are no longer required, programs and practices with resources for dissemination and implementation will be prioritized for review if the programs fall within the priorities established by the SAMHSA
review process. These resources are catalogued and reported in the program profile.
back to top
Does NREPP share the intervention summary with the program developer before publishing it on the NREPP website?
Yes. At the conclusion of the review, NREPP sends the program developer or (person who nominated the program for inclusion during the open submission period) a draft intervention summary and asks him or her to provide comments. NREPP's response to these comments may include making factual corrections or other adjustments to clarify the results of the review.
back to top
Can the program developer decline publication of the intervention summary on the NREPP website?
Under previous iterations of NREPP, program developers had the option to decline publication of the summary. However, that is no longer the case. Since each NREPP review represents a considerable investment of time and public funds, SAMHSA reserves the right to publish all programs on the website that were reviewed and rated.
back to top
I currently have a program on NREPP. When will it be re-reviewed?
A
Federal Register notice issued July 7, 2015 announced the intent to re-review all programs posted on NREPP prior to September 2015. Approximately 100 of these legacy programs will be re-reviewed each year over the next 3 years. Program contacts/developers will be notified at least 45 days prior to the re-review date that their program has been selected for re-review. The re-review will follow the procedures presented on the
Review Process page and be reviewed with the criteria presented on the
Program Review Criteria page.
Once all of the programs on NREPP prior to September 2015 have been re-reviewed, interventions may undergo a new review if they have been included in the registry for 5 years and if new research and/or updated dissemination materials are available.
back to top
Submitting an Intervention for Review
How do I find out if my intervention qualifies for submission?
Submission requirements are determined and announced on the NREPP website on the
Reviews and Submissions page. Requirements may also be specified in a Federal Register notice several months prior to the start of the open submission period. These notices are posted on the
NREPP Newsroom page.
To receive announcements about future open submission periods, please sign up for our
Submissions mailing list.
back to top
Is there any cost to submit an intervention to NREPP?
There is no fee to submit an intervention to NREPP. However, submitters must provide all the materials to be used in the review, including one copy of all implementation and dissemination materials, at their own expense, if these materials are not available online.
back to top
Where can I find information on preparing a submission to NREPP?
This information can be found on NREPP's website in the
Open Submissions page.
This page contains information on the
minimum requirements, information for submission, and the dates of the next open submission period.
back to top
Will every intervention that is submitted be reviewed?
No. Each submission undergoes a preliminary assessment to determine whether it has met
minimum requirements. Submissions that meet the minimum requirements may be notified that they have been accepted for review, depending on the availability of current funding. Submissions that do not meet minimum criteria will also be notified.
back to top
Will there be another open submission period?
Yes. Open submission periods are generally held twice a year. The dates of the next open submission period will be posted on the NREPP
home page. To receive announcements about open submission periods, please sign up for our
Submissions mailing list.
back to top
Can I submit multiple interventions for review?
Yes. Each intervention must be submitted separately.
back to top
Can interventions developed outside the United States be reviewed by NREPP?
Yes, if they meet the minimum requirements and have at least one replication conducted in the United States.
back to top
How can I share information with SAMHSA about how I was able to successfully use program or other content from the NREPP site in my community?
We are very interested in hearing about success stories related to the use of NREPP program profiles or other content from the site. Use our
Submit Feedback form to provide us with the content you found useful (for example, a specific program profile), how it was used, what you found useful or challenging about the information, and what the outcome was related to your use of the content. SAMHSA may share this information with staff who may contact you directly to gather further information and feedback about your experience.
back to top