1 Statistical significance is based on the results of the authors’ analysis using a chi-square test.
1 Authors report whether the groups are significantly different or not, but do not report actual p-values.
2 Statistical significance is based on the results of the authors’ analysis using a chi-square test.
1 In contrast to the study-reported results, HomVEE calculations showed this difference to be statistically significant (p = 0.05). The HomVEE tests of statistical significance are based on the HomVEE calculated effect sizes, whereas authors may have used other techniques, such as regression models or analyses of variance (ANOVA), to determine statistical significance.
3 Authors report whether the groups are significantly different or not, but do not report actual p-values.
This table shows a study’s results by outcome measure. Rows for primary outcomes are shaded. Favorable outcomes are listed in green and unfavorable or ambiguous outcomes are listed in red.
This table describes in more detail the outcome measures used by a study. Rows for primary outcomes are shaded.
Primary outcome measure: For the HomVEE review, an outcome measured through direct observation, direct assessment, or administrative data; or self-reported data collected using a standardized (normed) instrument.
Secondary outcome measure: For the HomVEE review, most self-reported data, excluding self-reports based on a standardized (normed) instrument.
For a uniform approach, HomVEE calculates mean differences as the program group mean minus the comparison group mean. In some studies, authors did this calculation in the opposite direction.
The odds ratio is the odds of an event occurring in one group divided by the odds of an event occurring in another group. If the odds of an event occurring in group A equals 3 and the odds of an event occurring in group B equals 2, the odds ratio equals 1.5. In other words, there is a 1.5 times greater likelihood the event will occur in group A than in group B.
HomVEE reports author-calculated effect sizes when available, as labeled in the table. If the author did not report effect sizes, but sufficient information was available, HomVEE calculated effect sizes.
The HomVEE review calculates effect sizes using the same computations as the What Works Clearinghouse (WWC). The WWC, established by the Institute for Education Sciences in the U.S. Department of Education, reviews education research. For more information, please read WWC effect size calculations.
Statistical significance is an indication of the probability that the observed finding was obtained by chance (when there is not a real relationship in the population). If the p-value is equal to or less than a predetermined cutoff (in the HomVEE review, 0.05), then the finding is considered statistically significant or probably not obtained by chance.
Home | Research | Outcomes | What's New | Publications | Review Process | About Us | Help