Methods and standards
Issue Brief: Response to Comments from the Public on Revised HomVEE Procedures and Standards
For the first time since its inception, HomVEE has substantially revised several of the evidence review's procedures and standards. These revisions were released in the Version 2 Handbook of the Procedures and Evidence Standards, which was published in December 2020 (see below). To inform the revisions, HomVEE solicited feedback through a public comment period announced in two simultaneous Federal Register notices (FRNs) published on August 5, 2020. This new brief summarizes HomVEE's approach to addressing key comments from the public and expert consultants in the final Version 2 Handbook.
Home Visiting Evidence of Effectiveness (HomVEE) Systematic Review Handbook of Procedures and Evidence Standards: Version 2
This handbook describes the methods used by the Home Visiting Evidence of Effectiveness (HomVEE) review to review existing research and report the findings. It is designed for use by researchers, policymakers, and other stakeholders. For the first time since its inception, HomVEE has substantially revised several procedures and standards for its review. HomVEE consulted with methodological experts and other federal evidence reviews to refine and update the procedures and standards in this document. These changes bring HomVEE generally into alignment with procedures and standards for other federally sponsored systematic evidence reviews, and they address critical topics in the evolution of the home visiting field. HomVEE will apply these procedures and standards beginning with the 2021 annual review.
Home Visiting Evidence of Effectiveness (HomVEE) Systematic Review Handbook of Procedures and Evidence Standards: Version 1
This document describes the original process and standards HomVEE used to review research from impact studies. (In 2020, HomVEE released its Handbook of Procedures and Standards: Version 2.0; those updated procedures and standards are available elsewhere on this page.)
HomVEE updated reporting guide for study authors
The author reporting guide, which has been updated from the 2016 version, provides evaluators guidance about how to describe randomized controlled trials and matched comparison group design studies, and report their findings clearly so that systematic reviews can use their results. Reporting the information described in this guide is considered a best practice in general, and the information can help HomVEE reviewers assess the appropriate rating to assign to the study. The latest update clearly identifies the information that HomVEE seeks from evaluators, which will help the project assign prioritization points that determine which early childhood home visiting models are reviewed.
Flowchart illustrating non-experimental comparison group design standards
This flowchart shows HomVEE’s process for rating matched comparison group studies, along with definitions of key concepts the HomVEE team considers when rating studies. Users of this flowchart may also read more about producing study ratings elsewhere on the HomVEE website
Flowchart illustrating randomized controlled trial standards
This flowchart shows HomVEE’s process for rating randomized controlled trials, along with definitions of key concepts the HomVEE team considers when rating studies. Users of this flowchart may also read more about producing study ratings elsewhere on the HomVEE website.
Federal Register notice about HomVEE version 2 procedures and standards
This Federal Register notice, published in 2020, details clarifications and updates that HomVEE proposed to its procedures and standards. The notice preceded publication of the HomVEE Version 2 handbook, where final procedures and standards are now documented.
Federal Register notice about HomVEE definition of home visiting model and model versions
This Federal Register notice, published in 2020, details definitions, rules, and procedures to guide how HomVEE handles home visiting model versions (commonly referred to in the home visiting research literature as adaptations and enhancements) in the review. The notice was released in advance of publishing the final HomVEE Version 2 handbook, where final definitions, rules, and procedures are now documented.
HomVEE fact sheet
HomVEE systematically reviews the research on early childhood home visiting models that serve pregnant women or families with children up to kindergarten entry. It determines which models have enough rigorous evidence to be considered evidence-based according to criteria defined by the U.S. Department of Health and Human Services (HHS). This 3-page fact sheet describes how HomVEE evaluates home visiting programs and provides stakeholders with an overview of how evidence-based home visiting models are identified through a four step evaluation process.
Addressing Attrition Bias in Randomized Controlled Trials: Considerations for Systematic Evidence Reviews
This paper is focused on attrition and the HomVEE attrition standard in particular. The paper begins by defining attrition and explaining why the bias that attrition introduces into randomized controlled trials can be problematic when interpreting research results. HomVEE uses an attrition standard adapted from the Department of Education’s What Works Clearinghouse (WWC), another systematic evidence review. HomVEE’s population of interest includes pregnant women, and families with children age birth to kindergarten entry; the population is different than the school-age children whose test scores were the basis of the attrition standard for the WWC. This paper describes findings testing the sensitivity of the assumptions underlying the HomVEE standard using data about parents and young children.