Each year, HomVEE releases new review results for program models. This includes reviews of studies on additional models and/or updates to previously reviewed models. Decisions on the number of models to review depend on available resources.
To help prioritize home visiting models for review, HomVEE assigns points based on:
- The number and design of impact studies (three points for each randomized controlled trial, single case design, or regression discontinuity design; and two points for each matched comparison group design)
- Sample sizes of impact studies (one point for each study with a sample size of 250 or more; before 2013, HomVEE used a cutoff of 50)
- Studies that examined an outcome of interest (starting in 2013, one point for each impact study that had an outcome (as reported in the abstract) in: child maltreatment; juvenile delinquency, family violence, or crime; linkages and referrals; and family economic factors. These domains are of particular interest because, to date, fewer studies reviewed for HomVEE have focused on them.)
During the prioritization process the HomVEE team also considers whether the program appears to: (1) be an active model that has existed for at least 3 years, (2) be associated with a national organization or institution of higher education, and (3) have available implementation support. This information may be gleaned from websites, HHS partners or other sources.
The annual prioritization effort may yield more models in the highest point category than can be reviewed that year. Beginning in 2017, HomVEE will apply a weighting formula to the prioritization score1. The weighting scheme places more emphasis on identifying additional models that could rate as evidence-based while still making sure that models with older reports get occasional updates. Specifically:
- A model that is not yet evidence-based (regardless of whether previously reviewed) gets a weight of 2.
- A model that is already evidence-based gets a weight of [1+0.1*(current year- prior report release date)]2. For example, a model being considered in 2017 that had a report released in 2013 would get a weight of [1 + 0.1*(2017 – 2013)]2 = 1.96.
HomVEE then sorts the list so that models with the highest weighted score are first on the list and models with the lowest weighted score are last, and works in that order to allocate review resources. This effort may include contacting study authors or model developers to confirm publicly available information. The team will review information on as many eligible models as possible each year. Eligible models that are not reviewed will be returned to the pool for future consideration, following the same procedures stated above. Also, to support policy or programmatic needs, the Department of Health and Human Services (HHS) may direct HomVEE to prioritize a certain model in a certain year.
As of 2013, results for previously reviewed models will not be updated every year. Models are only considered for updates every two years at the earliest. For example, if review results for a model were updated in 2016, that model will not be considered for additional updating until 2018.
1Earlier, HomVEE randomly ordered models in the highest points category and worked through the list in that random order.