Models are eligible for review as long as they meet HomVEE’s criteria for inclusion. HomVEE will select models to review by calculating a prioritization score for each model using a combination of model and manuscript characteristics. (Note that when HomVEE prioritizes a model, the review includes both the model and related versions.) After a model is selected for review, HomVEE will review all previously unreviewed research on the model. See the Version 2 Handbook of Procedures and Evidence Standards located on the Methods and Standards section of the website for more information on these processes.
The HHS criteria do not consider the magnitude of impacts in determining whether a model has evidence of effectiveness.
Even though magnitude is not considered in the HHS criteria, HomVEE reports standard effect sizes for findings rated high and moderate when they were available in the manuscripts or the HomVEE team had enough information to calculate them. Effect sizes can be found in the Effects shown in research section of each model effectiveness research report. The reports can be accessed through the Model Effectiveness Menu.
A full list of models reviewed by HomVEE, and whether they meet the HHS criteria, is available in the Model Effects Summary Table. The information is also available in the summary brief, available on the HomVEE Summary Page. Notably, a HomVEE designation as an evidence-based model does not guarantee that a model is eligible to be implemented with MIECHV funding (see above, “Are models on the HomVEE list automatically eligible for MIECHV funding?”).
No. A model may be evidence-based, according to HHS criteria, on the strength of either RCT research or QED research (including non-experimental comparison group, single-case, and regression discontinuity designs). The research must be of moderate or high quality based on HomVEE’s published standards. Additional HHS criteria apply when the research uses an RCT design. For details, please see the HHS criteria for evidence-based early childhood home visiting service delivery models.
HomVEE assesses whether each model meets the HHS criteria for an “evidence-based early childhood service delivery model.” To operationalize the HHS criteria related to studies, and because study findings may be reported across several manuscripts, HomVEE rates manuscripts based on the highest rated finding reported in that manuscript. Any high- or moderate-rated finding from a study about a model is considered as part of the evidence base for that model. Notably, for models with research solely from either RCT or SCD studies, additional criteria apply (see the HHS criteria). The additional criteria for RCTs to be from peer-reviewed journal articles and to have sustained findings align with MIECHV Program statutory requirements. The HHS criteria for an evidence-based model have no additional requirements for RDD or NED studies. A model may be evidence based on the strength of subgroup findings alone only if the research about it satisfies all of the subgroup criteria.
HomVEE uses the term “evidence-based model” to refer specifically to a model that meets HHS criteria developed based on statutory requirements in the authorizing legislation for the MIECHV Program. HomVEE recognizes that other systematic reviews may use different criteria to evaluate evidence of effectiveness. Thus, an evidence-based model in the context of HomVEE might or might not meet requirements for evidence of effectiveness according to other systematic reviews. The HHS criteria for an evidence-based early childhood home visiting service delivery model are designed to ensure confidence in model impacts through replicated findings from well-designed research.
For the purposes of the HomVEE review, an early childhood home visiting model is an intervention in which trained home visitors meet with expectant parents or families with young children to deliver a specified set of services through a specified set of interactions. These programs are voluntary interventions that are either designed or adapted and tested for delivery in the home. During the visits, home visitors aim to build strong, positive relationships with families to improve child and family outcomes. Services may be delivered on a schedule that is defined or can be tailored to meet family needs. A model has a set of fidelity standards that describe how the model is to be implemented.
Models reviewed by HomVEE must serve pregnant women or families with children from birth to kindergarten entry (that is, through age 5), and the primary service delivery strategy must be home visiting. In addition, the model must have research that examines its effects in at least one of eight outcome domains.
HomVEE’s Reporting Guide for Authors, provides guidance about how to describe each eligible research design and how to report findings in a way that is clear, complete, and transparent. Reporting the information described in the Reporting Guide for Authors is considered a best practice in general, but it can also help HomVEE reviewers assess the appropriate rating to assign to the manuscript.
HomVEE focuses its resources on reviewing manuscripts about impact studies that answer the review’s core question of whether an early childhood home visiting model is effective. Research that answers other questions is not eligible for review by HomVEE. For example, questions about for whom and under what circumstances a model is effective are valuable to the field but beyond the scope of the HomVEE review.
Research evaluating the impact of an isolated model feature or group of features is generally ineligible for HomVEE’s annual review. Knowing that a certain feature of a model is effective in isolation does not establish that a model consisting of multiple features (including the effective one) is effective overall. Therefore, research on single features should not contribute to the evidence base for an entire model.
Analyses designed to answer questions other than whether a model is effective are generally ineligible for review. Once HomVEE has confirmed that a manuscript reports on a study that uses an eligible design, a reviewer examines whether any analysis in the manuscript uses an eligible analytic approach. Specifically, two types of approaches are generally ineligible for review by HomVEE: most mediating and moderating analyses and all analyses that control for endogenous characteristics. A longer discussion of these types of analyses and exceptions in which HomVEE would review them can be found in Chapter 3 of the Version 2 Handbook of Procedures and Evidence Standards located on the Methods and Standards section of the website.
HomVEE may explore questions of which models are effective and for whom in other products beyond the annual review, such as Evidence Says briefs on special topics and HomVEE’s review of research on tribal populations.
Study designs vary in their relative risk of bias – that is, they vary in the extent to which researchers can be confident that impacts are due to home visiting rather than other factors. HomVEE accounts for risk of bias when assigning ratings to research. Research with a higher risk of bias receives a lower rating. For more details, please see Chapter 3 of the Version 2 Handbook of Procedures and Evidence Standards located on the Methods and Standards section of the website. HomVEE routinely contacts authors if additional details are needed to assess the appropriate rating to assign to the manuscript.