National Institute of Justice National Institute of Justice. Research. Development. Evaluation. Office of Justice Programs
Crime Solutions.gov
skip navigationHome  |  Help  |  Contact Us  |  Site Map   |  Glossary
Reliable Research. Real Results. skip navigation
skip navigation Still have questions?

skip navigation

FAQs

Following are some frequently asked questions associated with CrimeSolutions.gov:

General Questions
Programs and Practices on CrimeSolutions.gov
The Review Process and Becoming a Reviewer
Using Information on CrimeSolutions.gov


  • What is the difference between CrimeSolutions.gov and the What Works in Reentry Clearinghouse?

    CrimeSolutions.gov and the What Works in Reentry Clearinghouse projects serve as complementary resources for the field, with CrimeSolutions.gov providing topical information on effective justice programs and practices, and the What Works in Reentry Clearinghouse providing topical information on effective reentry programs. Despite the different topic areas, the projects share many common goals and characteristics. The projects are both funded by the U.S. Department of Justice. What Works in Reentry is a project of the National Reentry Resource Center, a project of the Bureau of Justice Assistance funded through the Second Chance Act.

     

    CrimeSolutions.gov and the What Works in Reentry Clearinghouse serve a similar function of providing users with online access and analysis of research studies from social science evaluations. To this end, both resources rely on evidence from high quality evaluations that use either experimental or quasi-experimental designs. Furthermore, both employ systematic review techniques and multiple reviewers to assess criteria related to the quality of evaluation design, evaluation outcomes, and program implementation. Both strive to improve practitioner decision making by providing a range of practical information related to each intervention, program, or practice.

     

    The primary difference between the two resources is that the What Works in Reentry Clearinghouse was developed to address the evidence base related to reentry and the specific information needs of those working in this field. CrimeSolutions.gov has a more general focus that includes all of criminal justice, juvenile justice, and crime victim services. In large part because of this difference, these two resources employ slightly different evidence review procedures, apply somewhat different weighting criteria, and use different methods to display information.

     

    Learn more about specific evidence review procedures and methods for:

  • What is the purpose of CrimeSolutions.gov?

    CrimeSolutions.gov is intended to be a central, reliable resource to help practitioners and policymakers understand what works in justice-related programs and practices. Its purpose is to assist in practical decision making and program implementation by gathering information on specific justice-related programs and practices and reviewing the existing evaluation and meta-analysis research against standard criteria.

    It is important to note the CrimeSolutions.gov Web site does not constitute an endorsement of particular programs or practices. The programs reported upon favorably are being recognized for their accomplishments in support of the mission of the Office of Justice Programs (OJP). Furthermore, it is not intended to replace or supersede informed judgment and/or innovation. CrimeSolutions.gov recognizes that rigorous evaluation evidence is one of several factors to consider in justice programming, policy, and funding decisions. OJP also recognizes the importance of encouraging and supporting innovative approaches that may not yet have extensive evidence of effectiveness.

    Read more about Tips for Using CrimeSolutions.gov.

  • What is the relationship between CrimeSolutions.gov and the Office of Juvenile Justice and Delinquency Prevention (OJJDP)’s Model Programs Guide?

    OJJDP’s Model Programs Guide (MPG), which predates the June 2011 launch of CrimeSolutions.gov, focuses specifically on programs related to the juvenile justice system and covers the entire continuum of youth services from prevention through sanctions to reentry. CrimeSolutions.gov serves a broader scope focusing on what works in criminal justice, juvenile justice, and crime victim services.

    After the launch of CrimeSolutions.gov, OJJDP began a re-review of MPG programs using the evidence standards and criteria developed for CrimeSolutions.gov (see Program Review and Rating from Start to Finish and Scoring Instruments to learn more about this review process). Beginning in October 2013, the juvenile program information contained on CrimeSolutions.gov and the MPG site will be completely aligned and will share program profiles. The existence of the two separate sites remains, though, because MPG also captures additional resources and tools that are specifically tailored to meet the needs of the juvenile justice field, including literature reviews and implementation guides.

    OJJDP and the National Institute of Justice (NIJ), the sponsor of CrimeSolutions.gov, are both components of the Office of Justice Programs (OJP), U.S. Department of Justice.

  • Who sponsors and operates CrimeSolutions.gov?

    CrimeSolutions.gov is an initiative of the Office of Justice Programs (OJP) within the U.S. Department of Justice. It is operated by the National Institute of Justice (NIJ). NIJ contracts with Development Services Group, Inc. (DSG) to coordinate the evidence review process and provide content and with Lockheed Martin to provide Web site and technical support.   

    Read more information about the Office of Justice Programs.

  • Will the Office of Justice Programs (OJP) only fund programs that appear on this site?

    No! CrimeSolutions.gov is a resource to assist justice practitioners and policy makers in using evaluation evidence for practical decision making and program implementation. CrimeSolutions.gov is not intended to be an exhaustive list of worthy and unworthy investments. OJP and its components also recognize the importance of supporting innovative approaches and practices that may not yet have extensive evidence of effectiveness. Please refer to the What is the purpose of CrimeSolutions.gov? FAQ for additional information.

    Read more information about the Office of Justice Programs.



  • How do I provide feedback or express concerns about an evidence rating or any other aspect of CrimeSolutions.gov?

    The information and evidence ratings included on CrimeSolutions.gov are not static. As additional programs and practices are identified and new research becomes available, CrimeSolutions.gov content will be updated and supplemented to reflect the most current programmatic and research information available. We also rely on users to provide us with critical feedback about the CrimesSolutions.gov Web site itself. What is useful and what is not? What additional features would you like to see on the site in the future? CrimeSolutions.gov users are welcome to Submit Feedback.

    Specific concerns about evidence ratings or information contained within CrimeSolutions.gov may also be submitted via the Submit Feedback online form. If necessary, changes to the information presented on the site will be made, per the processes outlined in Updating an Evidence Rating and Inquiring About or Appealing an Evidence Rating.

  • If a program or practice is rated as “Effective” on CrimeSolutions.gov, does that mean the Department of Justice (DOJ) or the Office of Justice Programs (OJP) is endorsing this program or practice for use by others?

    No. It is important to note the CrimeSolutions.gov Web site does not constitute an endorsement of particular programs or practices. The programs and practices reported upon favorably are being recognized for their accomplishments in support of the mission of the Office of Justice Programs (OJP). Furthermore, it is not intended to replace or supersede informed judgment and/or innovation. CrimeSolutions.gov recognizes that rigorous evaluation evidence is one of several factors to consider in justice programming, policy, and funding decisions. OJP also recognizes the importance of encouraging and supporting innovative approaches that may not yet have extensive evidence of effectiveness. Please refer to the What is the purpose of CrimeSolutions.gov? FAQ for additional information.

  • What do the evidence ratings used on CrimeSolutions.gov mean?

    CrimeSolutions.gov classifies programs  and practices in three levels: “Effective,” “Promising” and “No Effects” based on the strength of the evaluation research that indicates a program or practice achieves its goals (i.e., its intended outcomes).

    Evidence Ratings

    Evidence Rating

    Icon*

    Description

    One Study

    More than One Study or Meta-Analysis

    Effective

    Effective icon

    Effective multi-study icon

    Programs or Practices have strong evidence indicating they achieve their intended outcomes.

    Promising

    Promising icon

    Promising multi-study icon

    Programs or Practices have some evidence indicating they achieve their intended outcomes.

    No Effects

    Ineffective icon

    Ineffective multi-study icon

    Programs or Practices have strong evidence indicating that they did not achieve their intended outcomes.

    * A single study icon is used to identify programs or practices that have been evaluated with only one study. A multiple studies icon is used to represent a greater extent of evidence supporting the evidence rating. The icon depicts programs and practices that have more than one study or meta-analysis in the evidence base demonstrating effects in a consistent direction.   

    Read more about evidence ratings at About CrimeSolutions.gov.

  • What is the difference between a program and a practice on CrimeSolutions.gov?

    A Program is a specified set of activities combined according to precise guidance in order to achieve a specific purpose. Program profiles on CrimeSolutions.gov tell us whether a specific program was found to achieve its goals when it was carefully evaluated. The results apply to the exact set of activities and procedures used for that one program as it was implemented at the time of evaluation. Thus the program profile tells us that a program is likely to produce the observed result if implemented in exactly the same way.

    A Practice is a general category of programs, strategies, or procedures that share similar characteristics with regard to the matters they address and how they do it. Practice profiles tell us about the average results from multiple evaluations of similar programs, strategies, or procedures. The programs, strategies, or procedures within a practice are similar because they share certain defining characteristics that are described for each practice profile on CrimeSolutions.gov. Thus, practice profiles tell us the most typical results across multiple evaluations.

  • Why do practices sometimes have more than one evidence rating, but programs only have one?
    Sometimes a practice on CrimeSolutions.gov will receive multiple evidence ratings because there is sufficient evidence available to draw conclusions about multiple outcomes addressed by that practice. This is because the ratings for practices are based on a larger amount of evidence using techniques of meta-analysis to examine the findings of numerous studies. With more evidence, it is possible to draw conclusions at a more detailed level of analysis. Programs on CrimeSolutions.gov receive a single overall evidence rating because they are often based on only one evaluation study (although up to three may be included). With a smaller evidence base to inform conclusions, CrimeSolutions.gov does not provide outcome-level ratings for programs.
  • Why does CrimeSolutions.gov include “No Effects” programs and practices?

    "No Effects” programs and practices have strong evidence to indicate they do not achieve their intended outcomes. Programs and practices with "No Effects" evidence ratings either failed to produce any intended change or they produced negative effects. In cases where negative effects were found, CrimeSolutions.gov profiles for programs and practices will idntify and describe observed negative effects. CrimeSolutions.gov includes “No Effects” programs and practices to inform policy makers and practitioners about the current status of available evaluation evidence before planning or implementing similar efforts.

    Read more about: Program Review and Rating from Start to Finish or Tips for using CrimeSolutions.gov.



  • Are programs and practices implemented and evaluated outside the United States eligible for CrimeSolutions.gov?

    Yes. Programs and practices from other countries are included in CrimeSolutions.gov. These must have English language citations and are otherwise subject to the same criteria used for programs and practices located within the United States.

  • How can I become a CrimeSolutions.gov Study Reviewer?

    CrimeSolutions.gov Study Reviewers are subject-matter and research methodology experts who must complete training and certification prior to reviewing studies. Those interested in becoming Study Reviewers must first meet the minimum qualifications: (1) Study Reviewers must possess a Ph.D. (or other comparable advanced degree), preferably in a social science-related field of study such as sociology or criminal justice, but other fields of study will be considered; (2) Study Reviewers must be subject-matter experts and have substantial knowledge of research in at least one or more of the following seven topic areas: corrections and reentry, courts, crime and crime prevention, drugs and substance abuse, juvenile justice, law enforcement, and/or crime victims and victimization; (3) Study Reviewers must have extensive knowledge about research methodology. Potential Reviewers must be available to complete reviews up to about 5 days per year. Practitioners in the field are welcome to apply, as long as they meet the requisite education and background requirements.

     

    Separate training is required to review programs and practices. To become certified in reviewing practices, Study Reviewers must exhibit substantional knowledge and understanding of mea-analysis.

     

    To learn more about the qualifications and responsibilities of Study Reviewers, visit the CrimeSolutions.gov Researchers and Reviewers page. To submit your request to become a CrimeSolutions.gov Study Reviewer, please Contact Us. Upon receipt of your request, your message will be shared with the appropriate personnel at the Office of Justice Programs and Development Services Group, Inc., who will contact you for more information.

  • How does CrimeSolutions.gov determine if a practice works?

    For practices, Study Reviewers analyze all of the eligible and applicable meta-analysis studies to determine whether there is evidence that the practice achieves its goal(s). The Study Reviewers use a standard Scoring Instrument to determine the quality of each meta-analysis and to determine the internal validity of each outcome within the meta-analyses. The quality rating for each meta-analysis is combined with the internal validity rating of each outcome and the statistical significance of the effect size to determine a final evidence rating. If there are multiple meta-analyses that assess a single outcome, the individual outcome ratings from each meta-analysis are then aggregated to determine the final rating for that outcome.

    For practices, individual evidence ratings are assessed for each outcome.  Therefore, a single practice may have multiple evidence ratings if it addresses multiple outcomes. That is, one practice may be rated “Effective” for achieving one outcome, and “Promising” for achieving another outcome.

  • How does CrimeSolutions.gov determine whether a program works?

    For programs, Study Reviewers analyze the most rigorous evaluation research available to determine whether there is evidence that the program achieves its goal(s). Up to three studies, representing the most rigorous evaluation research available, are selected to comprise the program’s evidence base. In some cases, a fourth study may be included in the review if Study Reviewers feel it is necessary to accurately assess a program. The reviewers use a standard Scoring Instrument for each study and assign scores across four dimensions:

    ·         Program’s Conceptual Framework

    ·         Study Design Quality

    ·         Study Outcomes

    ·         Program Fidelity

    The individual study scores are then aggregated to determine the final evidence rating.

    Read more about the Program Review and Rating from Start to Finish or see the Scoring Instrument.

  • How does CrimeSolutions.gov identify programs and practices to be rated?

    Potential justice-related programs and practices are identified through:

     

    ·         Literature searches of relevant databases, journals, and publications; and

    ·         Nominations from experts, practitioners, or others.

     

    Read more about how to Nominate a Program or Practice for CrimeSolutions.gov.

  • How does CrimeSolutions.gov treat programs and practices for which there is insufficient evidence about whether they work?

    Programs with evidence that is insufficient to determine whether they achieve their intended outcomes do not receive an evidence rating on CrimeSolutions.gov. However, CrimeSolutions.gov periodically updates a static list of programs that have been reviewed by Study Reviewers, but not assigned an evidence rating due to lack of evidence. A program is placed on the insufficient evidence list if the study (or studies) reviewed received only Class 5 study ratings indicating that there were significant limitations in the study design such that it was not possible to establish a causal relationship to the program’s justice-related outcomes.

    A practice is placed on the insufficient evidence list if the meta-analysis (or meta-analyses) reviewed received only Class 5 outcome ratings indicating there were significant limitations in the study design of the included evaluations that it was not possible to establish a causal relationship to the practice's justice-related outcomes.

    See the List of Programs and Practices with Insufficient Evidence or Program Review and Rating from Start to Finish or Practice Review and Rating from Start to Finish.

  • What is considered before a program or practice is rated on CrimeSolutions.gov?

    Before the evidence for a program or practice is reviewed and rated, the program or practice is screened to determine whether it meets CrimeSolutions.gov’s criteria for inclusion on the Web site. This includes a thorough review of the purpose of the program or practice and the strength of the available evidence. Read more about: Program Review and Rating from Start to Finish or Practice Review and Rating from Start to Finish.

  • Who reviews the evidence used to rate the programs and practices on CrimeSolutions.gov?

    At least two Study Reviewers independently assess the evaluation research for each program and the meta-analysis research for each practice under consideration by CrimeSolutions.gov. All Study Reviewers are subject matter and research methodology experts who complete training and certification prior to reviewing studies. Lead Researchers select the evidence base, coordinate the review process, match programs with Study Reviewers based on expertise, and resolve any scoring discrepancies prior to a program being assigned a final evidence rating. Study Reviewers use a standard Scoring Instrument to determine whether a program achieves its goals.

    Read more about CrimeSolutions.gov Researchers and Reviewers or the Program Review and Rating from Start to Finish or the Practice Review and Rating from Start to Finish.

  • Why does CrimeSolutions.gov use a maximum of three evaluations to determine an evidence rating for a program?

    CrimeSolutions.gov Lead Researchers review the available research about a program and select up to three of the most rigorous studies to determine the evidence rating. In many cases, there are fewer than three evaluations available for a particular program. The studies selected are those with the most rigorous methods and study designs and are expected to produce the most reliable and credible results. Study Reviewers are subject matter experts familiar with the body of research in the topic area, so they may, in consultation with the Lead Researcher, request to review additional studies if they can demonstrate a compelling need. Study Reviewers also use additional studies, program materials, process evaluations, and curricula outside the evidence-base as supplemental material to inform the overall assessment process.

    To see more about the entire review process: Program Review and Rating from Start to Finish.



  • Can I cite the CrimeSolutions.gov review in my materials and, if so, how should it be cited?

    Intervention materials, including Web sites, may state that a program or practice has been reviewed and posted on CrimeSolutions.gov. After a profile has been posted on CrimeSolutions.gov, interested parties may Submit Feedback to request the CrimeSolutions.gov logo graphic. The logo may be used as a link to CrimeSolutions.gov or in the materials related to the program or practice. However, the posting of a program summary on CrimeSolutions.gov does not constitute an endorsement, promotion, or approval of the intervention by CrimeSolutions.gov or OJP. Please refer to the What is the purpose of CrimeSolutions.gov? FAQ for additional information.

    CrimeSolutions.gov recommends the following citation format for program/practice profiles and summaries: Office of Justice Programs. Name of program or practice. Retrieved [month, date, year profile was accessed], from CrimeSolutions.gov, [URL of summary].

  • Can I establish a link to the CrimeSolutions.gov Web site and if so, do you have any CrimeSolutions.gov graphics I can display on my site?

    The CrimeSolutions.gov Web site is sponsored by the U.S. Federal Government and is in the public domain. As such, you can link to the site.

    If you would like to provide your site visitors with access to the program profiles and ratings most recently posted to CrimeSolutions.gov, you are encouraged to add one of the available shareable program record widgets to your Web site. Utilizing CrimeSolutions.gov widgets is a great way of keeping your Web site users informed about what works in criminal justice, juvenile justice, and crime victim services!

    A "Latest Programs" widget is available which presents information about and access to details for the most recently posted CrimeSolutions.gov program profiles. Additionally, topical widgets are also available which provide information about recently posted programs associated with the following topics: Corrections & Reentry, Courts, Crime & Crime Prevention, Drugs & Substance Abuse, Juveniles, Law Enforcement, Technology & Forensics, and Victims & Victimization. The shareable program record widgets section of our site provides easy to follow instructions on how to incorporate these widgets onto your site. If you have any questions, please contact us for assistance.

  • Can I share information with the Office of Justice Programs (OJP) about how I was able to successfully use program or other content from the CrimeSolutions.gov site in my community?
    Yes! We are very interested in receiving success stories related to the use of CrimeSolutions.gov program profiles, practice profiles, or other content from the site. Using our Submit Feedback form, please provide us with the content you found useful (for example, a specific program profile), how it was used, what you found useful or beneficial about the information, and what the successful outcome was related to your use of the content. The information you provide will be shared with personnel at OJP. Upon review of your submission, OJP personnel may reach out to you directly to gather further information and feedback about your experience.
  • How can I reference the CrimeSolutions.gov review of my program or practice?

    Please use the following text: This program [or practice] received the rating of ['Effective' or 'Promising'] by CrimeSolutions.gov. For more information, visit [URL of CrimeSolutions.gov program or practice profile].