Frequently Asked Questions

Models

Were maternal/infant home visiting models, not just early childhood, included in the review?

Yes. Several models enroll women during pregnancy and aim to improve maternal health, infant health, and birth outcomes. Detailed information about each model’s theoretical model, target population, and target outcomes can be found on the model overview page of the implementation profile. Profiles can be accessed through the Implementation tab. Detailed information about maternal and child health outcomes can be found in the Effects Shown in Research tab in each model report. The reports can be accessed through the Research tab. That tab also has a search function that allows the user to filter the models by target population.

View Revisions

Is there any consideration given to those models that may not demonstrate a large impact due to the sample being difficult to treat (parents with severe mental illness, etc.)?

The HHS criteria do not consider the magnitude of impacts in determining whether a model has evidence of effectiveness.

Even though magnitude was not considered in the HHS criteria, HomVEE reported standard effect sizes on the website when they were available in the studies or the HomVEE team had sufficient information to calculate them. Effect sizes can be found in the Effects Shown in Research section of each model report. The reports can be accessed through the Research tab.

View Revisions

Study Sample Diversity

What is the racial/ethnic and income diversity of study samples in the HomVEE review?

Overall, the high- and moderate-quality impact studies included in the HomVEE review have diverse study samples that include families from a range of racial/ethnic groups and families with low incomes. The majority includes families from multiple racial/ethnic groups and all of the samples include low-income families. More information on study characteristics can be found for each model report accessed through the Research tab.

View Revisions

What is the evidence of effectiveness of home visiting models for specific racial/ethnic groups?

Although study samples included in the HomVEE review are quite diverse, most of the high- and moderate-quality impact studies aggregate the results across racial/ethnic groups. Thus, the available evidence does not indicate whether these models are more or less effective with one group or another. Further, sample sizes of most studies do not provide sufficient statistical power to detect effects for specific racial/ethnic groups. In other words, even if there are sizeable effects of the intervention for a particular racial/ethnic group, these may not be detected because of small sample sizes for that group. HomVEE only reported subgroup results for findings that were replicated in two different analytic samples; replication provides greater confidence that findings were not observed by chance. None of the models reviewed by HomVEE had subgroup findings for specific racial/ethnic groups that were replicated in two different samples. More information on study characteristics can be found for each model report accessed through the Research tab.

View Revisions

Are there any studies specific to American Indian/Alaska Native populations?

In addition to the main HomVEE review, the HomVEE team conducted a review of models that were implemented in tribal communities or included substantial proportions of American Indian Alaska Native families in the study samples. The results of this review are presented in a separate reports entitled, “Assessing the Research on Home Visiting Program Models Implemented in Tribal Communities Part 1: Evidence of Effectiveness“ and “Assessing the Research on Home Visiting Program Models Implemented in Tribal Communities Part 2: Lessons Learned about Implementation and Evaluation.“ These reports can be accessed on the Publications tab.

View Revisions

Were studies in the HomVEE review conducted in both rural and urban communities?

Several of the models that meet the HHS criteria for an “evidence-based early childhood home visiting service delivery model” have been studied with families from a mix of urban and rural communities. Results for families from rural and urban areas were aggregated across groups. Thus, the available evidence does not indicate whether these models are more or less effective with families from rural or urban communities. More information on study characteristics can be found for each model report accessed through the Research tab.

View Revisions

Information available on the website

Where can I find a list of models that meet the HHS criteria for evidence-based models?

A full list of models reviewed by HomVEE, and whether they meet the HHS criteria, is available in a model evidence summary table. The information is also available in the executive summary, available on the About Us page.

View Revisions

Is there a list of models initially reviewed but that did not meet the requirements?

A full list of models reviewed by HomVEE is available in a model evidence summary table. The information is also available in the executive summary, available on the About Us page.

View Revisions

Will the information on implementation of models be available to states?

Information on implementation of models is available in the implementation profile for each model. The implementation profiles contain an overview of the model and information about developers’ recommendations regarding prerequisites for implementation, training, materials and forms, program costs, and model contact information. The profiles also include a section on implementation experiences; this information was extracted from stand-alone implementation studies and causal studies with a moderate or high rating. These profiles can be accessed through the Implementation tab.

View Revisions

Do you include training and other on-going training costs?

The implementation profiles on each model include information about training and training costs to the extent that the information was available. Specific topics include requirements for certification, pre-service training, in-service training, training materials, qualified trainers, technical assistance, and training and technical assistance costs. This information is available in the training and estimated costs sections of each implementation profile. Profiles can be accessed through the Implementation tab.

View Revisions

Were birth outcomes/newborn outcomes included, such as preterm birth, low birth weight?

Yes. Detailed information about preterm birth, low birth weight, and newborn outcomes that were measured can be found in the Child Health section of the Effects Shown in Research section of each model report. The reports can be accessed through the Research tab. All outcomes in the Child Health domain combined across models can be accessed through the Outcomes tab.

View Revisions

Are there any plans to categorize research that is in progress on these models? That would be very helpful for those of us considering submission of a promising approach as part of our state program.

No. The HomVEE project does not have consistent information about research underway on home visiting models. States seeking information about research in process on specific models should contact the model developer or purveyor directly.

View Revisions

Review Process

How does a model get on the HomVEE list?

The answer to this question has two parts: 1) how a model is selected for review; and 2) whether a model is determined to meet the criteria established by HHS for an “evidence-based early childhood home visiting service delivery model.”

HomVEE uses a systematic process to select models for review. We conduct a literature search and issue a call for papers. Using the identified research, models are then assigned points based on 1) the number and design of the impact studies and 2) sample sizes of the impact studies. Models are ranked by accumulated points and those with higher rankings are selected for review. For more information on the review process, please see the review page.

HomVEE assesses each selected model to determine if it meets the HHS criteria for an evidence-based model. Information on each reviewed model is included on the HomVEE website, regardless of whether the models meet the HHS criteria, which can be accessed on each model’s “In Brief” page. A full list of models, and whether they meet the HHS criteria, is available in the model evidence summary table.

View Revisions

May I submit research on my model for review?

Yes, HomVEE issues a call for papers each year between November and early January. The call is sent to relevant electronic mailing lists and also will be posted on the HomVEE website with submission instructions.

Our goals are to treat all models consistently and fairly and to have a complete picture of all observed effects for a model. Therefore, HomVEE has a systematic process for identifying all relevant research and ranking models for review based on the available empirical evidence. Because of this process, we cannot review new studies on request.

View Revisions

May I appeal the review results?

You may request a re-review if you think the HHS criteria has been misapplied, information is missing, or there are errors on the HomVEE website. These concerns should be described in detail and submitted to: hvee@mathematica-mpr.com. If HHS approves the request, a re-review team made up of members external to the original contractor who were not involved in the original review will conduct a new, independent review. For more information on the re-review process, see Requests for Reconsideration of Evidence Determinations.

View Revisions

Can you provide information regarding the timing of re-review of models that have submitted an appeal? For example, when is re-review expected to be complete and results disseminated?

HHS will issue a final decision within 60 days of the submission of the request for re-review. All requests must be submitted to hvee@mathematica-mpr.com. If the model is approved as meeting the HHS criteria for evidence of effectiveness, all states will be notified.

View Revisions