Glossary

A  B  C  D  E  F  G  H  I  J  K  L  M  N  O  P  Q  R  S  T  U  V  W  X  Y  Z  

A

A method for analyzing the differences in the means of two or more groups. Specifically, this procedure partitions the total variation in the dependent variable into two components: between-group variation and within-group variation. It allows researchers to determine if the differences between a control group and a treatment group are attributed to the independent variable or treatment.

Occurs when the effects of a program are observed prior to the implementation of the program, generally because the target population believes the program has already started. This element is reviewed along with diffusion and displacement on the CrimeSolutions.gov Scoring Instrument. These elements are typically considered in evaluations of community-level crime prevention efforts. See Program Review and Rating from Start to Finish for more information.

The loss of participants during the course of a study, which often occurs because subjects move or they refuse to participate in the study. This may be a threat to the study’s Internal Validity. See Program Review and Rating from Start to Finish for more information.

Back to Top

B

An analysis of the relationship between two variables, such as correlations and one-way analysis of variance (ANOVA).

Back to Top

C

Evidence that documents a relationship between an activity, treatment, or intervention (including technology) and its intended outcomes, including measuring the direction and size of a change, and the extent to which a change may be attributed to the activity or intervention. Causal evidence depends on the use of scientific methods to rule out, to the extent possible, alternative explanations for the documented change. This differs from descriptve evidence.

A statistical test used to compare differences between observed, categorical data and expected data (based on a specific hypothesis) to determine if any difference that occurred is the same as would occur by chance.

A group of individuals whose characteristics are similar to those of a treatment group. Comparison group individuals may not receive any services, or they may receive a different set of services, treatment, or activities as the treatment group. In no instance do they receive the same services as the individuals being evaluated (the treatment group). Comparison groups are used in quasi-experimental designs where random assignment is not possible or practical.

Occurs when members of the control group or the comparison group are inadvertently exposed to the intervention or treatment being studied. Contamination threatens the study’s Internal Validity. See Program Review and Rating from Start to Finish for more information.

A group of individuals whose characteristics should be almost identical to those of the treatment group but do not receive the program services, treatments, or activities being evaluated. In experimental designs, individuals are placed into control groups and treatment groups through random assignment.

A statistical term that measures the degree of the relationship between two variables. A correlation has two components, magnitude and direction. Magnitude is a measure of strength and ranges from 0, no correlation, to 1, perfect correlation. Direction determines whether a correlation is positive or negative. A positive correlation means that as one variable, X, increases so does another variable, Y. A negative correlation means that as one variable, X, decreases so does another variable, Y. An inverse correlation means that as one variable, X, increases the other variable, Y, decreases and vice versa. For example, if variables X and Y have a correlation of 0.7 this means they have a strong, positive relationship. Correlation does not imply a causal relationship between variables.

Back to Top

D

A variable whose outcome is influenced or changed by some other variable, usually the independent variable or the treatment. It is the “effect” or outcome variable in a cause and effect relationship.

Evidence used to characterize individuals, groups, events, processes, trends, or relationships using quantitative statistical methods, correlational methods, or qualitative research methods. This differs from causal evidence.

Occurs when the effects or benefits of a program extend beyond the places, individuals, problems, or behaviors directly or indirectly targeted. This element is reviewed along with anticipatory benefits and displacement on the CrimeSolutions.gov Scoring Instrument. These elements are often considered in evaluations of community-level crime prevention efforts. See Program Review and Rating from Start to Finish for more information.

One of four broad categories of information included in the CrimeSolutions.gov Scoring Instrument used to review and rate program evidence. The dimensions include: Program’s Conceptual Framework; Study Design Quality; Study Outcomes; and Program Fidelity, which consists of multiple Evidence Rating Elements. See Program Review and Rating from Start to Finish for more information.

Occurs when an intervention has the effect of moving the problem in question (such as crime) rather than producing an actual reduction in incidence. This element is reviewed along with diffusion and anticipatory benefits on the CrimeSolutions.gov Scoring Instrument. These elements are typically considered in evaluations of community-level crime prevention efforts. See Program Review and Rating from Start to Finish for more information.

Back to Top

E

An Evidence Rating on CrimeSolutions.gov that indicates a program with strong evidence that it achieves its intended outcomes when implemented with fidelity. Read more About CrimeSolutions.gov or about Program Review and Rating from Start to Finish. "Effective" programs are represented throughout the site with the "Effective" icon: .

The strength of the evidence demonstrating that a program achieves its intended outcomes.  

Information about a question that is generated through systematic data collection, research, or program evaluation using accepted scientific methods that are documented and replicable. Evidence may be classified as either descriptive or causal. 

Up to three studies reviewed and scored by CrimeSolutions.gov Study Reviewers the results of which are aggregated to determine a program’s Evidence Rating.  Read more About CrimeSolutions.gov or about Program Review and Rating from Start to Finish.

Subcategories within the four broad dimensions included in the CrimeSolutions.gov Scoring Instrument used to review and rate a program’s evidence. See Program Review and Rating from Start to Finish for more information.

The Office of Justice Programs (OJP) considers programs and practices to be evidence-based when their effectiveness has been demonstrated by causal evidence, generally obtained through high quality outcome evaluations.

A research design in which participants are randomly assigned to an intervention/treatment group or a control group. Many social scientists believe studies using random assignment lead to the highest confidence that observed effects are the result of the program and not another variable. See also Randomized Controlled Trial (RCT).

Back to Top

F

The degree to which a program’s core services, components, and procedures are implemented as originally designed. Programs replicated with a high degree of fidelity are more likely to achieve consistent results. See Program Review and Rating from Start to Finish for more information.

Back to Top

H

An event that takes place between the pretest (data collected prior to the treatment beginning) and the posttest (data collected after the treatment ends) that has nothing to do with the treatment but may impact observed outcomes.  History is a potential threat to Internal Validity. See Program Review and Rating from Start to Finish for more information.

Back to Top

I

The measures used in a study. The instrumentation quality is dependent on the measures’ reliability and validity. Reliability refers to the degree to which a measure is consistent or gives very similar results each time it is used, and validity refers to the degree to which a measure is able to scientifically answer the question it is intended to answer. Instrumentation is a component considered within Internal Validity. See Program Review and Rating from Start to Finish for more information.

An analysis based on the initial treatment intent, not on the treatment eventually administered. For example, if the treatment group has a higher attrition rate than the control or comparison group, and outcomes are compared only for those who completed the treatment, the study results may be biased. An intent-to-treat design ensures that all study participants are followed until the conclusion of the study, irrespective of whether the participant is still receiving or complying with the treatment.

Back to Top

L

Back to Top

M

When observed outcomes are a result of natural changes of the program participants over time rather than because of program impact. Maturation is a threat considered within Internal Validity. See Program Review and Rating from Start to Finish for more information.

The systematic quantitative analysis of multiple studies that address a set of related research hypotheses in order to draw general conclusions, develop support for hypotheses, and/or produce an estimate of overall program effects.

A statistical method that allows researchers to estimate separately the variance between subjects within the same setting, and the variance between settings. For example, when evaluating a school-based program it is important to know the variation of students within the same school as well as the variation of students between different schools. This ensures that when programs are evaluated, the effects are not attributed to the program when there could be underlying differences between schools or between the students in those schools.

Research strategy and analytic technique that involves the investigation of more than two variables at the same time or within the same statistical analysis. For example, in a multiple regression analysis, the effects of two or more independent variables are assessed in terms of their impact on the dependent variable.

Back to Top

N

An Evidence Rating on CrimeSolutions.gov that indicates a program with strong evidence that it does not achieve its intended outcomes when implemented with fidelity. Read more About CrimeSolutions.gov or Program Review and Rating from Start to Finish. "No Effects" programs are represented throughout the site with the "No Effects" icon: .

Refers to a research design in which participants are not assigned to treatment and control/comparison groups (randomly or otherwise). Such designs do not allow researchers to establish causal relationships between a program or treatment and its intended outcomes. Non-experimental designs are sometimes used when ethics or circumstances limit the ability to use a different design or because the intent of the research is not to establish a causal relationship. Examples of non-experimental designs include case studies, ethnographic research, or historical analysis.

Back to Top

O

The intended results of a program’s activities or operation and a dimension in the CrimeSolutions.gov Scoring Instrument. Primary outcomes refer to the primary or central intended effects of a program. Within the scope of CrimeSolutions.gov, those primary outcomes must also relate to criminal justice, juvenile justice, or victim services. Secondary outcomes are the ancillary effects of a program. Outcomes are considered and rated separately within this dimension because programs may target multiple outcomes. Examples of outcomes include: reducing drug use, increasing system response to crime victims, and reducing fear of crime.

Back to Top

P

A planned, coordinated group of activities and processes designed to achieve a specific purpose. A program should have specified procedures (e.g., a defined curriculum, an explicit number of treatment or service hours, and an optimal length of treatment) to ensure the program is implemented with fidelity to its model. It may have, but does not necessarily need, a “brand” name and may be implemented at single or multiple locations.

Back to Top

Q

A research design that resembles an experimental design, but in which participants are not randomly assigned to treatment and control groups. Quasi-experimental designs are generally viewed as weaker than experimental designs because threats to validity cannot be as thoroughly minimized. This reduces the level of confidence that observed effects may be attributed to the program and not other variables.

Back to Top

R

Refers to an experimental research design in which participants are randomly assigned to a treatment or a control group. Most social scientists consider random assignment to lead to the highest level of confidence that observed effects are the result of the program and not other variables. 

As a final step on the Scoring Instrument, Study Reviewers provide an assessment as to their overall confidence in the study design. If both Study Reviewers agree, and the Lead Researcher concurs, that there is a fundamental flaw in the study design (not captured in the Design Quality dimension) that raises serious concerns about the study’s results, the study is removed from the evidence base and not factored into the Evidence Rating. This final determination serves as an additional safeguard to ensure that only the most rigorous studies comprise the evidence base. The study citation will be listed among the program’s additional references. See Program Review and Rating from Start to Finish for more information.

Back to Top

S

A sample is the subset of the entire population that is included in a research study. Typically, all else being equal, a larger sample size leads to increased precision in estimates of various properties of the population. The sample size affects the statistical power of a study and the extent to which a study is capable of detecting meaningful program effects. It is included as an element with Statistical Power in the CrimeSolutions.gov Scoring Instrument. See Program Review and Rating from Start to Finish for more information.

A compilation of the dimensions and elements of a research study that are reviewed and assigned a numerical score by the CrimeSolutions.gov Study Reviewers in order to assess the evidence of a program’s effectiveness. The instrument provides a standard method to assess the quality of each program’s evidence base, while also reflecting Study Reviewers’ judgment and expertise. See Program Review and Rating from Start to Finish or Scoring Instrument for more information.

The use of statistical controls to account for the initial measured differences between groups. It is not applicable for all research designs. See Program Review and Rating from Start to Finish for more information.

The ability of a statistical test to detect meaningful program effects. It is a function of several factors, including: 1) the size of the sample; 2) the magnitude of the expected effect; and 3) the type of statistical test used. Statistical power is an element within Sample Size on the CrimeSolutions.gov Scoring Instrument. See Program Review and Rating from Start to Finish for more information.

Subject matter and research methodology experts who review and assess the individual evaluation studies that comprise a program’s evidence base used to rate the programs on CrimeSolutions.gov. All Reviewers must complete training and receive certification prior to becoming a Study Reviewer. Read more about CrimeSoultions.gov Researchers and Reviewers.

A process by which the research evidence from multiple studies on a particular topic is reviewed and assessed using systematic methods to reduce bias in selection and inclusion of studies. A systematic review is generally viewed as more thorough than a non-systematic literature review, but does not necessarily involve the quantitative statistical techniques of a meta-analysis.

Back to Top

T

An analytic technique that uses a sequence of data points, measured typically at successive, uniform time intervals, to identify trends and other characteristics of the data. For example, a time series analysis may be used to study a city’s crime rate over time and predict future crime trends.

The subjects or program participants of the set of services, treatment, or activities being studied or tested.

Back to Top

V

A statistical measure of how far a set of data points are dispersed from the mean or average for a population or a sample. It is the average deviation of outcomes from the mean of outcomes for a group. It is used as a step in determining the effect of an intervention or treatment on a population.

Back to Top