Study 1
The study by Harrell, Cavanagh, and Sridharan (1999) compared CASASTART participants with two different groups. The first was a randomly assigned control group within target neighborhoods (hereafter referred to as the control group). The second was a quasi-experimental comparison group selected from matched high-risk neighborhoods in four of the five cities (hereafter referred to as the comparison group). There were no statistically significant differences between CASASTART youths and youths in the quasi-experimental comparison group in outcome measures, and only some significant differences were found between CASASTART youths and the randomly assigned control group.
Drug Use and Selling
There were no statistically significant differences between CASASTART youths and youths in the quasi-experimental comparison group in drug use or selling outcomes. After the program, CASASTART youths had lower rates of drug use—a difference that was statistically significant compared with the randomly assigned control group youths. However, this reduction was primarily in gateway drug use. No significant differences were found for strong drug use. CASASTART participants were significantly less likely to have sold drugs in the past month or at any time compared with control group youths.
Violent and Property Crimes
Similar to the drug use and selling outcomes, there were no statistically significant differences between CASASTART youths and youths in the quasi-experimental comparison group in any violent or property crime outcome measures. At the 1-year follow-up, CASASTART youths had committed fewer violent crimes after the program’s end than control youths had committed, and the difference was statistically significant. However, there were no statistically significant differences between CASASTART youths and control group youths in regard to property crime. Reductions in property crime were not evident in either group.
Prosocial Activities
There were no statistically significant differences between CASASTART youths and youths in the quasi-experimental comparison group in prosocial activity outcome measures. At the 1-year follow-up, CASASTART youths reported having significantly more positive peer support and significantly less association with delinquent peers than control group youths reported.
Study 2
Mihalic, Huizinga, Ladika, Knight, and Dyer (2011) found harmful programmatic effects, especially when examining outcomes by gender. Female participants in CASASTART achieved no measurable benefits from the program and showed negative effects in prevalence and frequency behavioral measures.
Prevalence of Delinquency and Violence
Prevalence outcomes showed CASASTART youths were significantly more likely to be status offenders, truants, and sent to the principal’s office than control group youths. CASASTART youths also had significantly higher prevalence rates of disciplinary incidents compared with control group youths.
When the analysis was examined by gender, there were no significant differences in prevalence measures between CASASTART males and control group males. However, CASASTART females reported higher prevalence measures than control group females reported for total, serious, and minor delinquency, as well as for serious violence, status offenses, truancy, and arrests, and the difference was statistically significant. In other words, treatment group females had significantly poorer outcome results than control group females.
Frequency of Delinquency and Violence
Frequency outcomes showed CASASTART youths exhibited significantly less serious delinquency and total violence and had fewer serious property offenses and drug sales compared with control group youths. However, CASASTART youths had significantly more truancy incidents than control group youths.
When the analysis was examined by gender, several significant differences were found favoring treatment group males. CASASTART males reported significantly less total and serious delinquency and violence and fewer serious property offenses and arrests than control group males. However, it appears CASASTART had a negative effect on females in the program. CASASTART females reported significantly higher measures of total and minor delinquency, drug use, status offenses, sexual activity, arrests, truancy, and number of days suspended.
Effect of Fidelity on Problem Behaviors
When accounting for implementation fidelity, negative program effects appear to be heightened. Youths who stayed in the program for a longer period of time had higher measures of total and serious delinquency, and the difference was statistically significant. Girls who stayed in the CASASTART program for a longer period of time had higher measures of total, minor, and serious delinquency compared with girls in the control group. CASASTART boys had higher measures of serious delinquency and other drug use. Negative program effects were stronger (e.g., treatment boys had higher measures of serious delinquency and other drug use) at sites with greater fidelity adherence.
Prosocial Activities and Risk Factors
Differences between CASASTART youths and control group youths were not statistically significant in any prosocial activity or risk factor outcome measures.
Study 1
Harrell, Cavanagh, and Sridharan (1999) conducted an impact evaluation using experimental and quasi-experimental comparison groups in five cities—Austin, Texas; Bridgeport, Conn.; Memphis, Tenn.; Savannah, Ga.; and Seattle, Wash. These sites were competitively selected (candidate cities developed proposals to implement the model) following an extensive planning phase, and received funding for at least 3 years. CASASTART (Striving Together to Achieve Rewarding Tomorrows) participants were compared with two different groups. The first was a randomly assigned control group within target neighborhoods (hereafter referred to as the control group). The second was a quasi-experimental comparison group selected from matched high-risk neighborhoods in four of the five cities (hereafter referred to as the comparison group). Seattle’s program stopped operating after 2 years and was not included in the quasi-experimental condition. The sample consisted of 338 CASASTART participants (the treatment group), 333 control group youths, and a quasi-experimental comparison group of 203 youths.
At the beginning of the evaluation, the average age of participating youths was 12.4 years. The sample was fairly evenly distributed in terms of gender (52 percent male) and consisted almost entirely of minority youths (58 percent black, 34 percent Hispanic, and 8 percent white or Asian). The mother was the primary caregiver (80 percent) for most participants. Caregivers in the sample had low levels of education, with more than half not graduating from high school. Additionally, less than half of caregivers were employed, and most relied on family support and public assistance. An extensive attrition analysis showed no differential response rates by group, city, demographic characteristics, or baseline risk factor assessments. As such, differences between the treatment and control groups were not statistically significant.
Outcome data was obtained with several different measures. In-person interviews were conducted in the respondent’s home at baseline and at the end of the program 2 years later. An additional follow-up survey was conducted with youths 1 year after the end of the CASASTART program. Police and court records were collected for participating youths each year of the evaluation. In addition, school records of grades, promotions, and attendance were collected for participating youths at each evaluation site. Drug use was measured using self-reported data. Specifically, adolescents were asked whether they had used drugs in the past month, past year, or at any time in their lives.
Study 2
Mihalic, Huizinga, Ladika, Knight, and Dyer (2011) conducted a full replication of the CASASTART evaluation using random assignment of eligible youths to the CASASTART program or a control group. Seven sites were selected: San Antonio, Texas (four schools); Baltimore, Md. (two schools); Bridgeport, Conn. (two schools); McKeesport, Pa. (four schools); Livingston, Ky. (two schools); Portland, Ore. (one school); and Trenton, N.J. (six schools). Almost all of the schools were new to the CASASTART program. Half of the schools in San Antonio and all of the schools in Bridgeport had previous experience with CASASTART. Eligible youths at each site were between the ages of 11 and 13 and had at least two risk factors in the school domain and at least one risk factor in both the family and personal domains. School risk factors included grade retention, poor academic performance, truancy, suspensions, or disruptive behavior; family risk factors included a history of family violence or drug use; and personal risk factors included involvement in drug use or sales, delinquency, gang membership, or being the victim of abuse or neglect.
The total evaluation sample consisted of 382 youths. These youths were randomly selected to receive the CASASTART program (197 youths) or assigned to a control group (185 youths). Due to the nationwide economic downturn, several sites lost funding and only 1 year of data was available instead of 2. Additionally, nine youths were below the required minimum age of 11 and another nine youths left school prior to baseline assessment, leaving a final sample size of 364 youths for analysis. Youths in the control group could receive services as usual, but could not receive all CASASTART services. Control group participants did not receive the intensive CASASTART case management service component that was received by the treatment group.
The sample consisted mostly of minorities (39.8 percent black, 31 percent Hispanic, 17 percent “Other Ethnicity,” and 12.1 percent white), and included slightly more males (60.4 percent) than females. The average age of youths in the evaluation was 12 years. Even though both treatment and control group youths showed no differences in measures of risk factors for delinquency (e.g. poor academic performance, suspensions, history of family violence, gang membership), baseline assessments revealed significant differences in measures of delinquent behavior. Despite having the same risk factors present at baseline, CASASTART youth had significantly higher prevalence and frequency measures for delinquency, violence, and drug sales compared with control youths. CASASTART youth also evidenced higher frequency measures for serious property offenses, other drug use, and sexual activity compared to control youths. Baseline differences in problem behaviors were controlled for in the analyses.
Data was collected three times during the evaluation: at pretest after randomization, in fall of the following school year, and during a posttest survey after the second school year. Delinquency was measured by the Self-Reported Delinquency Scale, adapted from the National Youth Survey and Denver Youth Survey. Youths were asked how many times in the last year they had engaged in a wide variety of illegal behavior (e.g., serious and minor violence, property crimes, drug use, etc.). Additional self-report survey measures included personal characteristics, peer association, and family risk and protective factors, as well as neighborhood risk and protective factors. Official school records were also collected to assess academic performance, absences, disciplinary incidents, and suspensions.
Two kinds of analysis were used to determine CASASTART’s effectiveness. The first tested mean differences between treatment and control participants; the second estimated treatment effects on outcomes in linear models that control for other variables. Multilevel regression models were used to account for the nested nature of students within sites. At the end of the 2-year program, posttest interviews were completed with 272 of the original 364 eligible students. Similar to checking for baseline differences, analyses were completed to detect the differences in treatment and control groups due to attrition. When compared with overall samples at baseline, the distributions in outcome variables were maintained in posttest results. Despite the attrition of subjects, analyses and outcome measures were not differentially affected by attrition.
Additional Resources:
