National Institute of Justice National Institute of Justice. Research. Development. Evaluation. Office of Justice Programs
Crime Solutions.gov
skip navigationHome  |  Help  |  Contact Us  |  Site Map   |  Glossary
Reliable Research. Real Results. skip navigation
skip navigation Additional Resources:

skip navigation

Program Profile: Challenging Horizons Program – After-School Version (CHP-After School)

Evidence Rating: No Effects - One study No Effects - One study

Date: This profile was posted on November 07, 2016

Program Summary

This is an after-school intervention designed to help students with attention deficit hyperactivity disorder (ADHD) develop, practice, and generalize academic and social skills by using volunteer mentors to deliver skills training to students. This program is rated as No Effects. Academic functioning and parent/teacher ratings of student behavior reflecting ADHD symptoms did not differ significantly between youths in the intervention group and those in the control group.

Program Description

Program Goals/Target Population
Challenging Horizons Program – After School (CHP-After School) is an after-school intervention (located at the school attended by participating students) for young adolescents with ADHD, which aims to improve their academic and social functioning through a relationship with an assigned Personal Counselor, or PC. The PC’s role is to develop a therapeutic relationship with each assigned student and to coordinate CHP interventions that help these students develop, practice, and generalize academic and social skills. Eligible participants are male and female middle school students (6th to 8th grade) who have a prior diagnosis of ADHD or exhibit at least four of nine symptoms of ADHD, according to the criteria from the Diagnostic and Statistical Manual of Mental Disorders (DSM).
 
Program Components
The CHP-After School intervention is based on the CHP model. CHP includes organizational interventions to address disorganization of materials and assignments, study and note-taking skills training to help students improve comprehension of written and spoken information, and an interpersonal skills group to help students improve their relations with peers and adults.
 
The CHP-After School version uses assigned PCs (trained undergraduate students) who work with their students in an after-school program. At the beginning of the school year, each student in the program is assigned to a PC, who focuses on establishing therapeutic relationships with youths while implementing CHP interventions and monitoring progress. PCs meet with students for a 2-hour and 15-minute program session twice a week. Each program session is composed of five activities: a meeting between the student and his or her PC, a group intervention targeting social impairment (ISG), recreation/game time, education/study skills group, and individual education time for homework completion. In the ISG component, students identify and work on personal social goals with their PC. In the education group, students focus on skills for studying, notetaking, summarizing, and writing. Skills are introduced with direct instruction followed by application of the skill at home or in class.
 
PCs track student data, including accuracy of assignment recording and adherence to the program’s criteria for student organization. The work of PCs is overseen by a site supervisor who also leads group activities. PCs are provided with a CHP-After School manual. The manual includes information on all procedures for each intervention, decision rules for modifying and progressing through interventions, and forms for tracking progress. PCs also check lockers, binders, book bags, and student planners to monitor student adherence to the organization criteria.
 
Key Personnel
The program is staffed primarily by the PCs. These are undergraduate students who receive 9 hours of pre-program training, and 15–30 minutes of individual and 60–90 minutes of group supervision weekly throughout the school year. The site supervisor, typically a graduate student or postdoctoral fellow, supervises the PCs and implements some of the program components, as noted above.
 
Program Theory
The CHP is designed to strengthen academic and social behaviors in students with ADHD. The program is identified as a training rather than a behavior-management intervention. This is reflected in the intervention’s focus on teaching skills, enabling practice of the learned skills, and promoting generalization of skills (but without manipulating behavioral contingencies in the school setting). Furthermore, schools serve as the host setting for the program, as this allows direct application of the skills that are the focus of the program, in both academic and social interactions.  

Evaluation Outcomes

top border
Study 1
Overall, Evans and colleagues (2016) reported mixed findings in their evaluation of the Challenging Horizons Program – After School (CHP-After School) program. No significant differences were found in change over time (i.e., from baseline to follow up) for the majority of the outcomes reviewed. Although the youths assigned to the program did have significantly more favorable change on some measures (including task planning, inattention/avoidance, and productivity), the preponderance of evidence suggests that  CHP-After School did not have a significant overall impact.
 
COSS: Task Planning
Students in the CHP-After School program differed significantly in change over time from students in the control group in parent-reported scores on the Task Planning subscale of the Children’s Organizational Skills Scale (COSS). Change from baseline to follow up was favorable for students in the intervention program. This difference was moderate in magnitude.
 
HPC: Inattention/Avoidance
Students in the CHP-After School program differed significantly in change over time from students in the control group in parent-reported scores on the Inattention/Avoidance subscale of the Homework Problem Checklist (HPC). Change from baseline to follow up was favorable for students in the intervention program. This difference was moderate in magnitude.
 
HPC: Poor Productivity
Students in the CHP-After School program differed significantly in change over time from students in the control group in parent-reported scores on the Poor Productivity and Nonadherence with Homework Rules subscale of the HPC. Change from baseline to follow up was favorable for students in the intervention program. This difference was moderate in magnitude.
 
CPS: Academic Competence
Students in the CHP-After School program did not differ significantly in change over time from students in the control group in teacher-reported scores on the Academic Competence subscale of the Classroom Performance Survey (CPS).
 
Percent of Assignments Turned In
Students in the CHP-After School program did not differ significantly in change over time from students in the control group in teacher-reported percentage of assignments turned in.
 
IRS: Academic Progress
Students in the CHP-After School program did not differ significantly in change over time from students in the control group in teacher-reported scores on the Academic Progress subscale of the Impairment Rating Scale (IRS). 
 
DBD: Inattention
Students in the CHP-After School program did not differ significantly in change over time from students in the control group in teacher-reported scores on the Inattention subscale of the Disruptive Behavior Disorders Rating Scale (DBD).
 
DBD: Hyperactivity/Impulsivity
Students in the CHP-After School program did not differ significantly in change over time from students in the control group in teacher-reported scores on the Hyperactivity/Impulsivity subscale of the DBD.
 
DBD: ODD Symptoms
Students in the CHP-After School program did not differ significantly in change over time from students in the control group in teacher-reported scores on the Oppositional Defiant Disorder Symptoms subscale of the DBD.
 
GPA
Students in the CHP-After School program showed a trend toward more favorable change over time in GPA, compared with students in the control group; however, this difference was not statistically significant.
bottom border

Evaluation Methodology

top border
Study 1
Evans and colleagues (2016) used a stratified random assignment design to evaluate the Challenging Horizons Program (CHP) in nine urban, suburban, and rural middle schools. Students in sixth to eighth grades were recruited in three cohorts over 3 academic years. Recruitment was conducted in the spring of the academic year prior to the start of the intervention by 1) sending letters home to all parents at participating schools, 2) having school staff directly inform parents of some students about the opportunity to participate, and 3) posting flyers in participating schools. Interested parents/caregivers underwent an initial telephone screening followed by an evaluation to determine the students’ eligibility for the study. To be eligible, students had to 1) attend a participating school; 2) meet Diagnostic and Statistical Manual of Mental Disorders, (4th edition, text revision [DSM-IV-TR]) diagnostic criteria for attention deficit hyperactivity disorder (ADHD); 3) demonstrate impairment in academic, social, or other relevant domains based on either parent or teacher report; 4) have an IQ of 80 or greater; and 5) not meet diagnostic criteria for pervasive developmental disorder or serious psychiatric disorders such as schizophrenia or bipolar disorder. 
 
After screening for eligibility and completing baseline assessment, students were randomized to one of three study conditions: CHP-After School, CHP-Mentoring, or a Community Care control condition. This review focuses on the comparison of the CHP-After School condition to the control condition. In the CHP-After School condition, each student was assigned to a Personal Counselor (PC) and participated in twice-weekly, after-school sessions involving the five program activities. Participants in the control condition received a list of locally available child and family psychosocial and pharmacological intervention options at the start of the school year. 
 
Participants were 112 sixth- to eighth-grade students in the CHP-After School program and 104 students in the control condition. No more than two students were assigned to one PC. Students attended an average of 31.85 after-school sessions, with 94 percent of the students attending at least once. 
 
Outcome measures were collected from parents at baseline, at four equally spaced times during the intervention year, and then 6 months after the end of the intervention (i.e., halfway through the following school year). These measures included 1) the Task Planning subscale of the Children’s Organizational Skills Scale (COSS), a 58-item scale assessing youth organization, time management, and planning difficulties; and 2) the Inattention and Avoidance of Homework and the Poor Productivity and Nonadherence with Homework Rules subscales of the 20-item Homework Problems Checklist. Outcome measures were also collected from teachers at all time points except at baseline. These measures included 1) the Academic Competence subscale of the Classroom Performance Survey (CPS), a 15-item rating scale assessing academic and interpersonal competence; 2) reports of the percentage of assignments turned in; 3) the Academic Progress subscale of the Impairment Rating Scale (IRS), a 7-item rating scale assessing areas of impairment at school; and 4) the Inattention, Hyperactivity/Impulsivity, and Oppositional Defiant Disorder (ODD) subscales of the Disruptive Behavior Disorders Rating Scale (DBD),  a 26-item checklist for symptoms of ADHD and ODD. Grades for each participating student were also collected for the last quarter of the previous school year, each quarter of both the intervention and follow-up school years, and converted into grade point averages (GPAs) for core subject areas (English/language arts, social studies, math, and science). Parents were also asked about their child’s use of medication and other treatment for ADHD at each assessment point. Descriptive information on demographic characteristics for the three groups at baseline was reported, but did not include significance tests for group differences. 
 
Intervention effects on youth functioning were assessed using hierarchical linear modeling (HLM) via a linear mixed effect model. All analyses used an intent-to-treat (ITT) approach (i.e., including data from all participants randomized into a condition regardless of level of participation in intervention activities). The basic models included group assignment, time, and group-by-time interaction as predictors of outcomes and included the following covariates: achievement scores, intelligence estimate, family income, and parental education (assessed at baseline), as well as medication use (included as a time-varying covariate).
bottom border

Cost

top border
There is no cost information available for this program.
bottom border

Implementation Information

top border
The Challenging Horizons Program – After School manual, which includes information on all procedures for each intervention, decision rules for modifying and progressing through interventions, and forms for tracking progress, is available to mentors and consultants and can be obtained from the program developer.
bottom border

Other Information

top border
Additional outcome measures in the study by Evans and colleagues (2016) included parent reports on the following measures: 1) Organized Actions and Memory Materials Manage subscales of the Children’s Organizational Skills Scale (COSS); 2) the Relation with Children and Academic Progress subscales of the Impairment Rating Scale; 3) the Inattention, Hyperactivity/Impulsivity, and ODD subscales of the Disruptive Behavior Disorders Rating Scale (DBD); and 4) the Social Skills, Problem Behavior, and Academic Competence subscales of the Social Skills Improvement System, a rating scale assessing the domains of social skills, problem behavior, and academic competence. Teachers also provided additional reports on the following measures: 1) the Interpersonal Competence subscale of the Classroom Performance Survey (CPS); and 2) the Relation with Peers subscale of the IRS. Group differences in change over time were generally not found between students in the CHP-After School and control groups, with the exception of parent-reported inattention on the DBD and the Materials Management subscale of the COSS, for which significant differences favoring the intervention group were found.
bottom border

Evidence-Base (Studies Reviewed)

top border
These sources were used in the development of the program profile:

Study 1
Evans, Steven W., Joshua M. Langberg, Brandon K. Schultz, Aaron Vaughn, Mekibib Altaye, Stephen A. Marshall, and Allison K. Zoromski. 2016. “Evaluation of a School-Based Treatment Program for Young Adolescents with ADHD.” Journal of Consulting and Clinical Psychology 84(1): 15–30. doi:10.1037/ccp0000057

bottom border

Additional References

top border
These sources were used in the development of the program profile:

Evans, Steven W., Brandon K. Schultz, Christine E. DeMars, and Heather Davis. 2011. “Effectiveness of the Challenging Horizons After-School Program for Young Adolescents with ADHD.” Behavior Therapy 42(3): 462–74. doi:10.1016/j.beth.2010.11.008 (This study was reviewed but did not meet Crime Solutions' criteria for inclusion in the overall program rating.)


Molina, Brooke S. G., Kate Flory, Oscar G. Bukstein, Andrew R. Greiner, Jennifer L. Baker, Vicky Krug, and Steven W. Evans. 2008. “Feasibility and Preliminary Efficacy of an After-School Program for Middle Schoolers with ADHD.” Journal of Attention Disorders 12(3): 207–17. doi:10.1177/1087054707311666 (This study was reviewed but did not meet Crime Solutions' criteria for inclusion in the overall program rating.)

bottom border

Related Practices

top border
Following are CrimeSolutions.gov-rated practices that are related to this program:

Mentoring
This practice provides at-risk youth with positive and consistent adult or older peer contact to promote healthy development and functioning by reducing risk factors. The practice is rated Effective in reducing delinquency outcomes; and Promising in reducing the use of alcohol and drugs; improving school attendance, grades, academic achievement test scores, social skills and peer relationships.

Evidence Ratings for Outcomes:
Effective - One Meta-Analysis Crime & Delinquency - Multiple crime/offense types
Promising - More than one Meta-Analysis Drugs & Substance Abuse - Multiple substances
Promising - One Meta-Analysis Education - Multiple education outcomes
Promising - One Meta-Analysis Mental Health & Behavioral Health - Psychological functioning
bottom border


Program Snapshot

Age: 10 - 14

Gender: Both

Race/Ethnicity: Black, Hispanic, White, Other

Geography: Rural, Suburban, Urban

Setting (Delivery): School

Program Type: Academic Skills Enhancement

Current Program Status: Active

Listed by Other Directories: Model Programs Guide, National Registry of Evidence-based Programs and Practices