Manuscript Detail

View Revisions

Green, B. L., Sanders, M. B., & Tarte, J. (2017). Using administrative data to evaluate the effectiveness of the Healthy Families Oregon home visiting program: 2–year impacts on child maltreatment & service utilization. Children and Youth Services Review, 75, 77–86.

Manuscript screening details
Screening decision Screening conclusion HomVEE procedures and standards version
Passes screens Eligible for review Version 1
Study design details
Rating Design Attrition Baseline equivalence Compromised randomization Confounding factors Valid, reliable measure(s)
High Randomized controlled trial Low Established on race and ethnicity; established on SES; outcomes not feasible to assess at baseline None None Not assessed in manuscripts reviewed before 2021
Notes:

footnote445

Submitted by user on

This study reported outcomes based on analyses of unadjusted means and proportions, as well as outcomes calculated based on regression analyses. Among the outcomes reported based on analyses of unadjusted means and proportions, all outcomes that are not assessable at baseline rate high. Three outcomes from the analyses of unadjusted means and proportions are assessable at baseline. Two of those outcomes, the percentage of mothers who ever received employment services and the number of days mother received employment services, rate moderate because the study does not establish baseline equivalence or control for differences in the baseline outcome. The third outcome, the percentage of mothers receiving substance abuse treatment services, rates low because we could not calculate attrition, and the study does not demonstrate equivalence or control for differences in the baseline outcome. All outcomes from the regression analyses rate low because we could not calculate attrition and the study does not establish baseline equivalence.

Study characteristics
Study participants Healthy Families Oregon staff who visited hospitals and clinics serving pregnant women screened families for eligibility. Eligible families were English-speaking, first-time parents with a child under 90 days old and identified as high-risk on a standardized screening tool (the New Baby Questionnaire [NBQ]). Parents who scored positive for substance abuse or depression, or a combination of two other parent and child risks (such as lack of comprehensive prenatal care or single-parent status) were determined to be high risk. The NBQ was used to screen 8,520 first-time parents, and 2,727 of these parents were found eligible and randomized to either the home visiting program or a control group (1,438 and 1,289, respectively). Enrollment took place between February 2010 and February 2012. Families were randomized in seven home visiting program sites in Oregon. At randomization, between 57 and 60 percent of families were white, non-Hispanic; 24 to 27 percent were Hispanic/Latino; and 15 to 16 percent were another race. Parents were, on average, 22 years old, and between 31 and 33 percent lacked a high school diploma or equivalent. Both parents were unemployed in between 35 and 37 percent of families. Outcomes were assessed using state administrative data matched to mother and child at two years following study enrollment for an analytic sample of 1,427 families in the intervention group and 1,280 families in the comparison group.
Setting This study took place in seven Healthy Families Oregon sites that were oversubscribed, three of which served primarily rural communities and four of which served mixed urban and rural communities.
Intervention services Healthy Families Oregon is an accredited state home visiting program based on the HFA model. Families enroll prenatally or up to 90 days after birth and receive home visiting services through the child’s third birthday. Families receive weekly individualized home visits for at least six months that typically focus on child development and positive parent-child interactions. Families also receive case management services that identify their needs and link them to relevant services. Intervention uptake was low; of those assigned to the home-visiting group, only 44 percent received at least one home visit.In addition, all families in the study received brochures on parenting and child development, information about community resources, books, videos, and other small gifts.
Comparison conditions Families assigned to the comparison condition were not eligible to receive home visiting services through the Healthy Families Oregon program. However, they did receive brochures on parenting and child development, information about community resources, books, videos, and other small gifts.
Subgroups examined This field lists subgroups examined in the manuscript (even if they were not replicated in other samples and not reported on the summary page for this model’s report).

• Public benefit receipt (mother received Temporary Assistance to Needy Families, Supplemental Nutrition Assistance Program, child care subsidy, or employment services)

Funding sources The research was funded by the U.S. Department of Health and Human Services, Children’s Bureau (grant number 90CA1782).
Author affiliation None of the study authors are developers of the program model.
Peer reviewed Peer reviewed status is not listed for manuscripts reviewed before 2021.
Study Registration:

Clinicaltrials.gov Identifier: None found. Study registration was assessed by HomVEE beginning with the 2014 review.

Findings that rate moderate or high

Family economic self-sufficiency
Rating Outcome measure Effect Sample Timing of follow-up Sample size Intervention group Comparison group Group difference Effect size Statistical significance Notes
High Number of days on TANF, intent-to-treat analysis, unadjusted
FavorableUnfavorable or ambiguousNo Effect
Oregon, 2010-2012 2 years 2,707 mothers Unadjusted mean = 176.10 Unadjusted mean = 169.60 Mean difference = 6.50 HomVEE calculated = 0.03 Not statistically significant, p = 0.51

footnote318

Submitted by user on

Statistical significance is based on <abbr title="Home Visiting Evidence of Effectiveness">HomVEE</abbr> calculations.

High Number of days on SNAP, intent-to-treat analysis, unadjusted
FavorableUnfavorable or ambiguousNo Effect
Oregon, 2010-2012 2 years 2,707 mothers Unadjusted mean = 475.10 Unadjusted mean = 461.20 Mean difference = 13.90 HomVEE calculated = 0.05 Not statistically significant, p = 0.20

footnote318

Submitted by user on

Statistical significance is based on <abbr title="Home Visiting Evidence of Effectiveness">HomVEE</abbr> calculations.

High Number of days with child care subsidy, intent-to-treat analysis, unadjusted
FavorableUnfavorable or ambiguousNo Effect
Oregon, 2010-2012 2 years 2,707 mothers Unadjusted mean = 19.50 Unadjusted mean = 19.80 Mean difference = -0.30 HomVEE calculated = 0.00 Not statistically significant, p = 0.92

footnote318

Submitted by user on

Statistical significance is based on <abbr title="Home Visiting Evidence of Effectiveness">HomVEE</abbr> calculations.

High Percentage ever received TANF, intent-to-treat analysis, unadjusted
FavorableUnfavorable or ambiguousNo Effect
Oregon, 2010-2012 2 years 2,707 mothers Unadjusted proportion = 0.42 Unadjusted proportion = 0.41 Mean difference = 0.01 HomVEE calculated = 0.02 Not statistically significant, p = 0.56

footnote318

Submitted by user on

Statistical significance is based on <abbr title="Home Visiting Evidence of Effectiveness">HomVEE</abbr> calculations.

High Percentage ever received child care subsidy, intent-to-treat analysis, unadjusted
FavorableUnfavorable or ambiguousNo Effect
Oregon, 2010-2012 2 years 2,707 mothers Unadjusted proportion = 0.11 Unadjusted proportion = 0.11 Mean difference = -0.01 HomVEE calculated = -0.04 Not statistically significant, p = 0.33

footnote318

Submitted by user on

Statistical significance is based on <abbr title="Home Visiting Evidence of Effectiveness">HomVEE</abbr> calculations.

High Percentage ever received SNAP, intent-to-treat analysis, unadjusted
FavorableUnfavorable or ambiguousNo Effect
Oregon, 2010-2012 2 years 2,707 mothers Unadjusted proportion = 0.85 Unadjusted proportion = 0.82 Mean difference = 0.02 HomVEE calculated = 0.10 Statistically significant, p = 0.01

footnote318

Submitted by user on

Statistical significance is based on <abbr title="Home Visiting Evidence of Effectiveness">HomVEE</abbr> calculations.

Moderate Number of days with employment services, intent-to-treat analysis, unadjusted
FavorableUnfavorable or ambiguousNo Effect
Oregon, 2010-2012 2 years 2,707 mothers Unadjusted mean = 98.90 Unadjusted mean = 94.40 Mean difference = 4.50 HomVEE calculated = 0.02 Not statistically significant, p = 0.53

footnote318

Submitted by user on

Statistical significance is based on <abbr title="Home Visiting Evidence of Effectiveness">HomVEE</abbr> calculations.

Moderate Percentage ever received employment services, intent-to-treat analysis, unadjusted
FavorableUnfavorable or ambiguousNo Effect
Oregon, 2010-2012 2 years 2,707 mothers Unadjusted proportion = 0.31 Unadjusted proportion = 0.30 Mean difference = 0.00 HomVEE calculated = 0.01 Not statistically significant, p = 0.77

footnote318

Submitted by user on

Statistical significance is based on <abbr title="Home Visiting Evidence of Effectiveness">HomVEE</abbr> calculations.

Reductions in child maltreatment
Rating Outcome measure Effect Sample Timing of follow-up Sample size Intervention group Comparison group Group difference Effect size Statistical significance Notes
High Number of days in out-of-home care, intent-to-treat analysis, unadjusted
FavorableUnfavorable or ambiguousNo Effect
Oregon, 2010-2012 2 years 2,707 children Unadjusted mean = 15.21 Unadjusted mean = 12.74 Mean difference = 2.47 Not available Not reported
High Number of substantiated abuse allegations, intent-to-treat analysis, unadjusted
FavorableUnfavorable or ambiguousNo Effect
Oregon, 2010-2012 2 years 2,707 children Unadjusted mean = 0.01 Unadjusted mean = 0.01 Mean difference = 0.00 Not available Not reported
High Number of substantiated maltreatment reports, intent-to-treat analysis, unadjusted
FavorableUnfavorable or ambiguousNo Effect
Oregon, 2010-2012 2 years 2,707 children Unadjusted mean = 0.07 Unadjusted mean = 0.07 Mean difference = 0.00 Not available Not reported
High Number of substantiated neglect allegations, intent-to-treat analysis, unadjusted
FavorableUnfavorable or ambiguousNo Effect
Oregon, 2010-2012 2 years 2,707 children Unadjusted mean = 0.07 Unadjusted mean = 0.07 Mean difference = 0.00 Not available Not reported
High Percentage with at least one out-of-home placement, intent-to-treat analysis, unadjusted
FavorableUnfavorable or ambiguousNo Effect
Oregon, 2010-2012 2 years 2,707 children Unadjusted proportion = 0.04 Unadjusted proportion = 0.03 Mean difference = 0.01 HomVEE calculated = 0.10 Statistically significant, p = 0.01

footnote446

Submitted by user on

Statistical significance is based on <abbr title="Home Visiting Evidence of Effectiveness">HomVEE</abbr> calculations. Negative effect is favorable to the intervention.

High Percentage with at least one substantiated maltreatment report, intent-to-treat analysis, unadjusted
FavorableUnfavorable or ambiguousNo Effect
Oregon, 2010-2012 2 years 2,707 children Unadjusted proportion = 0.06 Unadjusted proportion = 0.06 Mean difference = 0.00 HomVEE calculated = 0.03 Not statistically significant, p = 0.41

footnote446

Submitted by user on

Statistical significance is based on <abbr title="Home Visiting Evidence of Effectiveness">HomVEE</abbr> calculations. Negative effect is favorable to the intervention.

High Percentage with at least one substantiated neglect report, intent-to-treat analysis, unadjusted
FavorableUnfavorable or ambiguousNo Effect
Oregon, 2010-2012 2 years 2,707 children Unadjusted proportion = 0.06 Unadjusted proportion = 0.06 Mean difference = 0.00 HomVEE calculated = 0.03 Not statistically significant, p = 0.40

footnote446

Submitted by user on

Statistical significance is based on <abbr title="Home Visiting Evidence of Effectiveness">HomVEE</abbr> calculations. Negative effect is favorable to the intervention.

High Percentage with at least one substantiated physical or sexual abuse report, intent-to-treat analysis, unadjusted
FavorableUnfavorable or ambiguousNo Effect
Oregon, 2010-2012 2 years 2,707 children Unadjusted proportion = 0.01 Unadjusted proportion = 0.01 Mean difference = 0.00 HomVEE calculated = -0.29 Statistically significant, p = 0.00

footnote446

Submitted by user on

Statistical significance is based on <abbr title="Home Visiting Evidence of Effectiveness">HomVEE</abbr> calculations. Negative effect is favorable to the intervention.