FAQ-Study Sample Diversity-221

Although study samples included in the HomVEE review are quite diverse, most of the manuscripts about high- and moderate-quality impact studies aggregate the results across racial/ethnic groups. Thus, the available evidence generally does not reveal whether these models are more or less effective with one group or another.

Further, the sample sizes of most studies may not provide sufficient statistical power to detect effects for specific racial/ethnic subgroups. In other words, even if there are sizeable effects of the intervention for a particular racial/ethnic group, these may not be detected because of small sample sizes for that subgroup.

Because HomVEE’s mission is to identify evidence-based models, and HHS’ criteria require replication before subgroup research can be part of the evidence base, the annual review only reports findings from impact research on replicated subgroups. Replication provides more confidence that findings were not observed by chance. HomVEE uses the same set of standards to rate the quality of research for all findings on a replicated subgroup. For example, if the subgroup replication straddled the timing of an update to HomVEE standards, HomVEE would review both sets of subgroup findings using the newest standards (even if other, full-sample findings from the older manuscript were reviewed using older HomVEE standards).

HomVEE recognizes that the effects of models for different populations is of interest to the home visiting field. Although HomVEE only reports findings for replicated subgroups, HomVEE lists the subgroups examined in each manuscript beginning with its 2021 annual review. HomVEE does not report details on the findings or the quality of the study design until the subgroup is replicated.

More information on study characteristics, including the race/ethnicity of a study sample, can be found for each manuscript about high- or moderate-rated study on a model via the Information about research reviewed tab in any Model Effectiveness Research Report. If a model has been tested with a replicated subgroup, HomVEE summarizes the subgroup findings as part of that model’s effectiveness research report.

Custom Title

Does HomVEE provide information on the effectiveness of early childhood home visiting models for specific racial/ethnic groups?