Holt and colleagues (2008) evaluated the Achievement Mentoring Program (AMP). Study participants were chosen from a group of 97 students who were receiving a universal prevention program entitled "Peer Group Connection" (PGC; Johnson, Holt, Bry, and Powell, 2008) in an urban public high school. PGC is a year-long, peer outreach program focused on facilitating all 9th graders’ transitions to high school. Study participants were students in PGC who were deemed to be at risk for academic failure after 1 semester of high school. Students were characterized as at risk for academic failure if they exhibited at least two of the following risk factors: low grades and/or academic motivation, discipline problems, and frequent tardiness or absence from school. Using these criteria, 44 students were selected for the evaluation of AMP.
These 44 students were grouped by gender and race/ethnicity, and then randomly assigned in equal numbers to either a treatment group to receive the 5-month AMP intervention or to a control group with no mentoring. Four students were excluded from the sample, resulting in a final sample of 40 students. The composition of the final sample was 58 percent male, 47 percent Latino, 38 percent African American, 5 percent white, and 10 percent other race/ethnicity. There were no observed differences between the treatment and control groups on the outcome measures of interest at the baseline.
Three outcome measures of interest were gathered from school records: 1) number of discipline referrals, 2) number of absences, and 3) grade point average. These measures were obtained for the first semester of the participants' freshman year (before the AMP intervention began), at the end of their freshman year, and at the end of the first semester of their second year of high school (additional outcome measures included students’ self-reported sense of school belonging, decision making, academic efficacy, and perceptions of teacher support; these outcomes were not obtained at the final follow-up, however, and thus are not included in this review.). Analysis of variance (ANOVA) tests were used to test for effects of the intervention on the outcome measures. A Wilcoxon signed-rank test was used to determine whether there was a significant difference between the treatment and control groups in the number of students who received a discipline referral for the first time during the relevant period.
Clarke (2009) replicated and extended Holt and colleagues’ (2008) evaluation of the impact of AMP over a period of 2 academic years. Participants who were deemed at risk for academic failure were selected from a pool of students in an urban public high school who were receiving the PGC program. Researchers selected a sample of 39 participants who were 79 percent African American, 18 percent Hispanic American, 3 percent European American, and 44 percent male. Each student selected was paired with a similar selected student based on gender, ethnicity, grades, and attendance and discipline records, in that order of priority. The two members of each pair were then randomly assigned to either a treatment group to receive AMP or to a control group (20 students were assigned to the treatment group, and 19 to the control group).
Treatment and control groups were surveyed at 3 time points: during the first semester of their freshman year (prior to the beginning of the AMP intervention), at the end of their freshman year, and at the end of their second year of high school. School records that reported information on grades and discipline referrals were obtained for each of the 4 semesters of the 2-year study period. By the end of the study, data was available for between 10 and 11 participants in the control group (depending on the outcome measure) and for between 13 and 15 participants in the treatment group. Lack of availability of data at the end of the study was due to various reasons, including dropping out of school, being sent to an alternative school, or being placed out of district. No differences were observed at the baseline comparison between the treatment and control groups on the outcome measures.
Outcome measures included students’ self-reported decision-making efficacy, goal-setting efficacy, perceptions of teacher support, perceptions of classmate acceptance, and negative school behavior as well as school records of grades in mathematics and language arts, and discipline referrals. Researchers tested for effects of the intervention by comparing the treatment and control groups at each available follow-up point. T-tests were used to examine the self-reported outcome measures, and Mann-Whitney U tests were used to examine the discipline referrals. To evaluate effects on academic grades, repeated-measures ANOVA was used to test for group differences in changes of grades over the 4 periods of assessment.