Review Process

Prioritizing Models for Review

Each year, HomVEE releases new review results for models. This includes reviews of studies on models that have not previously been reviewed and/or updates to previously reviewed models. Decisions on the number of models to review depend on the number of studies that are identified for review about each model and available resources.

To help prioritize home visiting models for review, HomVEE reviews the title and abstract of each study that meets screening criteria, and assigns points to studies based on the following factors (as of 2018, when re-reviewing a model that is not evidence-based, studies already reviewed by HomVEE that received a high or moderate rating receive the same number of points as new studies):1

  • The number and design of impact studies (three points for each randomized controlled trial, single case design, or regression discontinuity design; two points for each matched comparison group design).
  • Sample sizes of impact studies (one point for each study with a sample size of 250 or more; before 2013, HomVEE used a cutoff of 50).
  • Studies that examined an outcome of interest. (Starting in 2013, one point for each impact study that had an outcome in: child maltreatment; juvenile delinquency, family violence, or crime; linkages and referrals; and family economic factors. These domains are of particular interest because, to date, fewer studies reviewed for HomVEE have focused on them.)
  • Factors of interest to the MIECHV Program. Starting in 2018, HomVEE also reviews impact studies and add points as follows:
    • 0.5 points if the study's sample is of a U.S. population or indigenous population.
    • 0.25 points if the study’s sample is of any priority population named in MIEHCV statute.2

After points are assigned, HomVEE groups the studies according to the home visiting model being tested and calculates a score for each model. Beginning in 2018, HomVEE applies up to 4 additional points for a series of model-level factors for specific MIECHV-relevant criteria, in order to more closely align HomVEE with the MIECHV Program. This information may be obtained from study abstracts, model websites, HHS partners, or other sources. The model-level factors and associated points are as follows:

  • The model is associated with a national organization (which may be outside of the United States) or institution of higher education.
  • The model is “active” – that is, currently serving or available to serve families.
  • The model has been implemented for at least three years, even if it is not active.
  • The model has implementation support available somewhere in the United States.

Beginning in 2019, HomVEE uses two review tracks: one for models that are not evidence-based (including unreviewed models as well as previously reviewed models that are not evidence-based) and another for evidence-based models.3

  • For models that HomVEE has not previously found to be evidence-based, HomVEE sorts the list so that models with the highest total prioritization score (including both study- and model-level points) are first on the list and models with the lowest score are last, and works in that order to allocate review resources.
  • For models that HomVEE has previously found to be evidence-based, HomVEE applies a weighting formula to the total prioritization score. This formula is applied so that evidence-based models that have not been reviewed recently have a higher likelihood of being prioritized for review than those that have been reviewed recently. Specifically, a model that is already evidence-based gets a weight of [1+0.1*(current year – prior report release date)]2. For example, a model being considered in 2018 that had a report released in 2014 would get a weight of [1 + 0.1*(2018 – 2014)]2 = 1.96. HomVEE sorts the list so that evidence-based models with the highest weighted score are first and those with the lowest weighted score are last, and works in that order to allocate review resources.4

If needed for prioritization, further information may be obtained by contacting study authors or model developers to confirm publicly available information.

The annual prioritization effort may yield more models in the highest point category than can be reviewed that year. Eligible models that are not reviewed will be returned to the pool for consideration in future years, following the same procedures stated above. The MIECHV program may coordinate with HomVEE to prioritize review of promising approaches5 implemented and evaluated under a MIECHV grant.

As of 2018, if resources are constrained in a given year and an evidence-based model is prioritized for updating, HomVEE will not review studies based on research conducted in international settings (except research involving indigenous communities outside the United States). In this event, HomVEE will clearly list the research that was included and the research that was not included when updating the report about that model on the HomVEE website. HomVEE will still review studies based on research conducted in international settings about any prioritized models that have not previously been found to be evidence-based.

1Studies HomVEE has already reviewed that earned a low rating will not receive any points.

2According to (42 U.S.C. § 711 (d)(4)), priority populations are as follows:

  • Low-income families.
  • Families who are pregnant women who have not attained age 21.
  • Families that have a history of child abuse or neglect or have had interactions with child welfare services.
  • Families that have a history of substance abuse or need substance abuse treatment.
  • Families that have users of tobacco products in the home.
  • Families that are or have children with low student achievement.
  • Families with children with developmental delays or disabilities.
  • Families who, or that include individuals who, are serving or formerly served in the Armed Forces, including such families that have members of the Armed Forces who have had multiple deployments outside of the United States.

3Originally, HomVEE randomly ordered models in the highest points category and worked through the list in that random order. In 2018, HomVEE applied a weighting scheme to the total prioritization points for all models and reviewed models in order from highest to lowest weighted points.

4As of 2013, results for previously reviewed models will not be updated every year. Models are only considered for updates every two years at the earliest. For example, if review results for a model were updated in 2019, that model will not be considered for additional updating until 2021 or later.

5Social Security Act, Title V, § 511 (d)(3)(A)(II).

View Revisions