Manuscript Detail

View Revisions

Green, B. L., Sanders, M. B., & Tarte, J. (2017). Using administrative data to evaluate the effectiveness of the Healthy Families Oregon home visiting program: 2–year impacts on child maltreatment & service utilization. Children and Youth Services Review, 75, 77–86.

Model(s) Reviewed: Healthy Families America (HFA)®
Manuscript screening details
Screening decision Screening conclusion
Passes screens Eligible for review
Study design details
Rating Design Attrition Baseline equivalence Reassignment Confounding factors
High Randomized controlled trial Low Established on race and ethnicity; established on SES; outcomes not feasible to assess at baseline None None
Notes:

footnote445

Submitted by user on Fri, 03/15/2019 - 14:29

This study reported outcomes based on analyses of unadjusted means and proportions, as well as outcomes calculated based on regression analyses. Among the outcomes reported based on analyses of unadjusted means and proportions, all outcomes that are not assessable at baseline rate high. Three outcomes from the analyses of unadjusted means and proportions are assessable at baseline. Two of those outcomes, the percentage of mothers who ever received employment services and the number of days mother received employment services, rate moderate because the study does not establish baseline equivalence or control for differences in the baseline outcome. The third outcome, the percentage of mothers receiving substance abuse treatment services, rates low because we could not calculate attrition, and the study does not demonstrate equivalence or control for differences in the baseline outcome. All outcomes from the regression analyses rate low because we could not calculate attrition and the study does not establish baseline equivalence.

Study characteristics
Study participants Healthy Families Oregon staff who visited hospitals and clinics serving pregnant women screened families for eligibility. Eligible families were English-speaking, first-time parents with a child under 90 days old and identified as high-risk on a standardized screening tool (the New Baby Questionnaire [NBQ]). Parents who scored positive for substance abuse or depression, or a combination of two other parent and child risks (such as lack of comprehensive prenatal care or single-parent status) were determined to be high risk. The NBQ was used to screen 8,520 first-time parents, and 2,727 of these parents were found eligible and randomized to either the home visiting program or a control group (1,438 and 1,289, respectively). Enrollment took place between February 2010 and February 2012. Families were randomized in seven home visiting program sites in Oregon. At randomization, between 57 and 60 percent of families were white, non-Hispanic; 24 to 27 percent were Hispanic/Latino; and 15 to 16 percent were another race. Parents were, on average, 22 years old, and between 31 and 33 percent lacked a high school diploma or equivalent. Both parents were unemployed in between 35 and 37 percent of families. Outcomes were assessed using state administrative data matched to mother and child at two years following study enrollment for an analytic sample of 1,427 families in the intervention group and 1,280 families in the comparison group.
Setting This study took place in seven Healthy Families Oregon sites that were oversubscribed, three of which served primarily rural communities and four of which served mixed urban and rural communities.
Intervention services Healthy Families Oregon is an accredited state home visiting program based on the HFA model. Families enroll prenatally or up to 90 days after birth and receive home visiting services through the child’s third birthday. Families receive weekly individualized home visits for at least six months that typically focus on child development and positive parent-child interactions. Families also receive case management services that identify their needs and link them to relevant services. Intervention uptake was low; of those assigned to the home-visiting group, only 44 percent received at least one home visit.In addition, all families in the study received brochures on parenting and child development, information about community resources, books, videos, and other small gifts.
Comparison conditions Families assigned to the comparison condition were not eligible to receive home visiting services through the Healthy Families Oregon program. However, they did receive brochures on parenting and child development, information about community resources, books, videos, and other small gifts.
Staff characteristics and training Not specified.
Funding sources The research was funded by the U.S. Department of Health and Human Services, Children’s Bureau (grant number 90CA1782).
Author affiliation None of the study authors are developers of the program model.
Study Registration:
Clinicaltrials.gov Identifier: None found. Study registration was assessed by HomVEE beginning with the 2014 review.

Findings details

Family economic self-sufficiency
Rating Outcome measure Effect Sample Timing of follow-up Sample size Intervention group Comparison group Group difference Effect size Statistical significance Notes
High Number of days on TANF, intent-to-treat analysis, unadjusted
FavorableUnfavorable or ambiguousNo Effect
Oregon, 2010-2012 2 years 2,707 mothers Unadjusted mean = 176.10 Unadjusted mean = 169.60 Mean difference = 6.50 HomeVEE calculated = 0.03 Not statistically significant, p = 0.51

footnote318

Submitted by user on Fri, 03/15/2019 - 14:29

Statistical significance is based on <abbr title="Home Visiting Evidence of Effectiveness">HomVEE</abbr> calculations.

High Number of days on SNAP, intent-to-treat analysis, unadjusted
FavorableUnfavorable or ambiguousNo Effect
Oregon, 2010-2012 2 years 2,707 mothers Unadjusted mean = 475.10 Unadjusted mean = 461.20 Mean difference = 13.90 HomeVEE calculated = 0.05 Not statistically significant, p = 0.20

footnote318

Submitted by user on Fri, 03/15/2019 - 14:29

Statistical significance is based on <abbr title="Home Visiting Evidence of Effectiveness">HomVEE</abbr> calculations.

High Number of days with child care subsidy, intent-to-treat analysis, unadjusted
FavorableUnfavorable or ambiguousNo Effect
Oregon, 2010-2012 2 years 2,707 mothers Unadjusted mean = 19.50 Unadjusted mean = 19.80 Mean difference = -0.30 HomeVEE calculated = 0.00 Not statistically significant, p = 0.92

footnote318

Submitted by user on Fri, 03/15/2019 - 14:29

Statistical significance is based on <abbr title="Home Visiting Evidence of Effectiveness">HomVEE</abbr> calculations.

High Percentage ever received TANF, intent-to-treat analysis, unadjusted
FavorableUnfavorable or ambiguousNo Effect
Oregon, 2010-2012 2 years 2,707 mothers Unadjusted proportion = 0.42 Unadjusted proportion = 0.41 Mean difference = 0.01 HomeVEE calculated = 0.02 Not statistically significant, p = 0.56

footnote318

Submitted by user on Fri, 03/15/2019 - 14:29

Statistical significance is based on <abbr title="Home Visiting Evidence of Effectiveness">HomVEE</abbr> calculations.

High Percentage ever received child care subsidy, intent-to-treat analysis, unadjusted
FavorableUnfavorable or ambiguousNo Effect
Oregon, 2010-2012 2 years 2,707 mothers Unadjusted proportion = 0.11 Unadjusted proportion = 0.11 Mean difference = -0.01 HomeVEE calculated = -0.04 Not statistically significant, p = 0.33

footnote318

Submitted by user on Fri, 03/15/2019 - 14:29

Statistical significance is based on <abbr title="Home Visiting Evidence of Effectiveness">HomVEE</abbr> calculations.

High Percentage ever received SNAP, intent-to-treat analysis, unadjusted
FavorableUnfavorable or ambiguousNo Effect
Oregon, 2010-2012 2 years 2,707 mothers Unadjusted proportion = 0.85 Unadjusted proportion = 0.82 Mean difference = 0.02 HomeVEE calculated = 0.10 Statistically significant, p = 0.01

footnote318

Submitted by user on Fri, 03/15/2019 - 14:29

Statistical significance is based on <abbr title="Home Visiting Evidence of Effectiveness">HomVEE</abbr> calculations.

Moderate Number of days with employment services, intent-to-treat analysis, unadjusted
FavorableUnfavorable or ambiguousNo Effect
Oregon, 2010-2012 2 years 2,707 mothers Unadjusted mean = 98.90 Unadjusted mean = 94.40 Mean difference = 4.50 HomeVEE calculated = 0.02 Not statistically significant, p = 0.53

footnote318

Submitted by user on Fri, 03/15/2019 - 14:29

Statistical significance is based on <abbr title="Home Visiting Evidence of Effectiveness">HomVEE</abbr> calculations.

Moderate Percentage ever received employment services, intent-to-treat analysis, unadjusted
FavorableUnfavorable or ambiguousNo Effect
Oregon, 2010-2012 2 years 2,707 mothers Unadjusted proportion = 0.31 Unadjusted proportion = 0.30 Mean difference = 0.00 HomeVEE calculated = 0.01 Not statistically significant, p = 0.77

footnote318

Submitted by user on Fri, 03/15/2019 - 14:29

Statistical significance is based on <abbr title="Home Visiting Evidence of Effectiveness">HomVEE</abbr> calculations.

Reductions in child maltreatment
Rating Outcome measure Effect Sample Timing of follow-up Sample size Intervention group Comparison group Group difference Effect size Statistical significance Notes
High Number of days in out-of-home care, intent-to-treat analysis, unadjusted
FavorableUnfavorable or ambiguousNo Effect
Oregon, 2010-2012 2 years 2,707 children Unadjusted mean = 15.21 Unadjusted mean = 12.74 Mean difference = 2.47 Not available Not reported
High Number of substantiated abuse allegations, intent-to-treat analysis, unadjusted
FavorableUnfavorable or ambiguousNo Effect
Oregon, 2010-2012 2 years 2,707 children Unadjusted mean = 0.01 Unadjusted mean = 0.01 Mean difference = 0.00 Not available Not reported
High Number of substantiated maltreatment reports, intent-to-treat analysis, unadjusted
FavorableUnfavorable or ambiguousNo Effect
Oregon, 2010-2012 2 years 2,707 children Unadjusted mean = 0.07 Unadjusted mean = 0.07 Mean difference = 0.00 Not available Not reported
High Number of substantiated neglect allegations, intent-to-treat analysis, unadjusted
FavorableUnfavorable or ambiguousNo Effect
Oregon, 2010-2012 2 years 2,707 children Unadjusted mean = 0.07 Unadjusted mean = 0.07 Mean difference = 0.00 Not available Not reported
High Percentage with at least one out-of-home placement, intent-to-treat analysis, unadjusted
FavorableUnfavorable or ambiguousNo Effect
Oregon, 2010-2012 2 years 2,707 children Unadjusted proportion = 0.04 Unadjusted proportion = 0.03 Mean difference = 0.01 HomeVEE calculated = 0.10 Statistically significant, p = 0.01

footnote446

Submitted by user on Fri, 03/15/2019 - 14:29

Statistical significance is based on <abbr title="Home Visiting Evidence of Effectiveness">HomVEE</abbr> calculations. Negative effect is favorable to the intervention.

High Percentage with at least one substantiated maltreatment report, intent-to-treat analysis, unadjusted
FavorableUnfavorable or ambiguousNo Effect
Oregon, 2010-2012 2 years 2,707 children Unadjusted proportion = 0.06 Unadjusted proportion = 0.06 Mean difference = 0.00 HomeVEE calculated = 0.03 Not statistically significant, p = 0.41

footnote446

Submitted by user on Fri, 03/15/2019 - 14:29

Statistical significance is based on <abbr title="Home Visiting Evidence of Effectiveness">HomVEE</abbr> calculations. Negative effect is favorable to the intervention.

High Percentage with at least one substantiated neglect report, intent-to-treat analysis, unadjusted
FavorableUnfavorable or ambiguousNo Effect
Oregon, 2010-2012 2 years 2,707 children Unadjusted proportion = 0.06 Unadjusted proportion = 0.06 Mean difference = 0.00 HomeVEE calculated = 0.03 Not statistically significant, p = 0.40

footnote446

Submitted by user on Fri, 03/15/2019 - 14:29

Statistical significance is based on <abbr title="Home Visiting Evidence of Effectiveness">HomVEE</abbr> calculations. Negative effect is favorable to the intervention.

High Percentage with at least one substantiated physical or sexual abuse report, intent-to-treat analysis, unadjusted
FavorableUnfavorable or ambiguousNo Effect
Oregon, 2010-2012 2 years 2,707 children Unadjusted proportion = 0.01 Unadjusted proportion = 0.01 Mean difference = 0.00 HomeVEE calculated = -0.29 Statistically significant, p = 0.00

footnote446

Submitted by user on Fri, 03/15/2019 - 14:29

Statistical significance is based on <abbr title="Home Visiting Evidence of Effectiveness">HomVEE</abbr> calculations. Negative effect is favorable to the intervention.

Outcome measure summary

Family economic self-sufficiency
Outcome measure Description of measure Data collection method Properties of measure

Number of days on TANF, intent-to-treat analysis, unadjusted

Total number of days receiving Temporary Assistance for Needy Families (for the full sample, including those not receiving) from enrollment to two years after enrollment. Administrative records

Not applicable

Number of days on SNAP, intent-to-treat analysis, unadjusted

Total number of days receiving Supplemental Nutrition Assistance Program (for the full sample, including those not receiving) from enrollment to two years after enrollment. Administrative records

Not applicable

Number of days with child care subsidy, intent-to-treat analysis, unadjusted

Total number of days receiving child care subsidies or benefits (for the full sample, including those not receiving) from enrollment to two years after enrollment. Administrative records

Not applicable

Percentage ever received TANF, intent-to-treat analysis, unadjusted

Percentage of mothers who were enrolled in Temporary Assistance for Needy Families at any time from enrollment to two years after enrollment. Administrative records

Not applicable

Percentage ever received child care subsidy, intent-to-treat analysis, unadjusted

Percentage of mothers who received a child care subsidy or benefit at any time from enrollment to two years after enrollment. Administrative records

Not applicable

Percentage ever received SNAP, intent-to-treat analysis, unadjusted

Percentage of mothers who were enrolled in the Supplemental Nutrition Assistance Program at any time from enrollment to two years after enrollment. Administrative records

Not applicable

Number of days with employment services, intent-to-treat analysis, unadjusted

Total number of days receiving employment services provided through the Oregon Department of Human Services (primarily related to TANF program), at any time from enrollment to two years after enrollment. Administrative records

Not applicable

Percentage ever received employment services, intent-to-treat analysis, unadjusted

Percentage of mothers who received employment services provided through the Oregon Department of Human Services (primarily related to TANF program), at any time from enrollment to two years after enrollment. Administrative records

Not applicable

Reductions in child maltreatment
Outcome measure Description of measure Data collection method Properties of measure

Number of days in out-of-home care, intent-to-treat analysis, unadjusted

Average number of days in out-of-home placement (foster care) for the full sample of children in the two-year period after enrollment. Administrative records

Not applicable

Number of substantiated abuse allegations, intent-to-treat analysis, unadjusted

Average number of substantiated abuse reports (including physical or sexual abuse from administrative records and counts taken from medical records of encounters with health care providers for injuries or ingestions) for up to two years after enrollment. Administrative records

Not applicable

Number of substantiated maltreatment reports, intent-to-treat analysis, unadjusted

Average number of substantiated maltreatment reports (from administrative records and counts taken from medical records of encounters with health care providers for injuries or ingestions) for up to two years after enrollment. Administrative records

Not applicable

Number of substantiated neglect allegations, intent-to-treat analysis, unadjusted

Average number of substantiated neglect reports (from administrative records and counts taken from medical records of encounters with health care providers for injuries or ingestions) for up to two years after enrollment. Administrative records

Not applicable

Percentage with at least one out-of-home placement, intent-to-treat analysis, unadjusted

Percentage of children with at least one out-of-home placement (foster care) in the two-year period after enrollment. Administrative records

Not applicable

Percentage with at least one substantiated maltreatment report, intent-to-treat analysis, unadjusted

Percentage of children with at least one substantiated maltreatment report (from administrative records and counts taken from medical records of encounters with health care providers for injuries or ingestions) for up to two years after enrollment. Administrative records

Not applicable

Percentage with at least one substantiated neglect report, intent-to-treat analysis, unadjusted

Percentage of children with at least one substantiated neglect report (from administrative records and counts taken from medical records of encounters with health care providers for injuries or ingestions) for up to two years after enrollment. Administrative records

Not applicable

Percentage with at least one substantiated physical or sexual abuse report, intent-to-treat analysis, unadjusted

Percentage of children with at least one substantiated abuse report (including physical or sexual abuse based on administrative records and counts taken from medical records of encounters with health care providers for injuries or ingestions) for up to two years after enrollment. Administrative records

Not applicable