Can Evidence Be Used to Make It More Effective?
Although there is no general consensus on the definition of “evidence-based,” the term is usually used to describe a policy, law, or practice whose efficacy is supported by a body of research. Typically the research will be academic, and the most trustworthy source of research is often considered that which is published in a peer-reviewed journal.
PEER REVIEW
The process of “peer review” is considered rigorous, and because it is undertaken by “peers,” or those colleagues of the author who have proven expertise in a given field, it is considered to be a legitimate way to validate research findings.
The Mississippi State Legislature has embraced the use of evidence in its “revitalization” of performance-based budgeting, a process begun in the Mississippi Performance Budget and Strategic Planning Act of 1994. Part of this plan includes implementing the Pew-MacArthur Results First Initiative, a model of “evidence-based policymaking” that utilizes a cost-benefit analysis to determine the return on investment of state budget dollars. Section 27-103-159 (MS Code of 1972) codified certain definitions to be used in this pilot. The focus on program outcomes and research in this legislative performance-based budgeting effort signals a shift to implementing evidence-based programs in the state.
Definitions Used in Performance-Based Budgeting MS Code 1972 § 27-103-159
“Evidence-based program” shall mean a program or practice that has multiple site random controlled trials across heterogeneous populations demonstrating that the program or practice is effective for the population.
“Research-based program” shall mean a program or practice that has some research demonstrating effectiveness, but that does not yet meet the standard of evidence-based practices.
“Promising practices” shall mean a practice that presents, based upon preliminary information, potential for becoming a research-based or evidence-based program or practice.
“Other programs and activities” shall mean all programs and activities that do not fit the definition of evidence-based, research-based or promising practices program.
Application to Sex-Related Education
The National Conference of State Legislatures (NCSL) recommends that in order to address teen pregnancy, Mississippi should invest in evidence-based programs, particularly those approved by the Office of Adolescent Health, housed in the U.S. Department of Health and Human Services (HHS). HHS hired private firms to evaluate programs to determine if evidence supported their effectiveness in three areas: reducing teen pregnancy, reducing sexually transmitted infections, and reducing associated sexual risk behaviors. The firms used a rigorous protocol to review studies released from 1989 through July 2014, resulting in a list of 36 programs being identified as evidence-based. These 36 programs represent a wide variety of types of interventions, including sex-related education (SRE).
In contrast to the focus on outcomes promoted by NCSL, Pew, and the Mississippi Legislature’s performance-based budgeting initiative, the Mississippi law enacted in 2011 that mandates school districts provide sex-related education does not require the curriculum to be evidence-based. As shown in Figure 1, the most commonly used curriculum,“Choosing the Best,” is not one recommended by HHS as an evidence-based program. Out of 146 school districts reporting to MDE on which curriculum was used during school year 2014-2015, 112 districts (76%) were using a curriculum not on the HHS list, while 35 districts (24%) were using a curriculum on the HHS list. Of the programs approved under the current state law, only three of the nine abstinence-only curricula and two of the seven abstinence-plus curricula appear on the HHS list of evidence-based programs. The fact that a curriculum is not recognized as evidence-based, however, does not necessarily mean that it is not valid; it may be due to the absence of a study evaluating its effectiveness. Also, if a curriculum is modified to meet state standards or statutory mandates, it is no longer the same curriculum that was deemed evidence-based.
Summary
While the specific definitions of evidence-based may differ among users of the term, the emphasis on demonstrated outcomes based on sound research is a common goal. When applied to sex-related education, there is available research conducted by independent organizations that can be used to evaluate curricula, although the timeliness of the research can be an issue. Requiring this or other credible research to be used in selecting curricula purchased by school districts would bring Mississippi’s law more closely in alignment with the goals of performance-based budgeting.
Resources
- Kolbo, et. al. (2015). Sex-related education policy and practices in Mississippi public schools. Center for Mississippi Health Policy. Available at: http://www.mshealthpolicy.com/wp-content/uploads/2015/12/2015-SRE-Evaluation-Final-Report10-27-15.pdf.
- Legislative Budget Office, (2015). A legislator’s guide to revitalizing performance budgeting in Mississippi. Mississippi Legislature. Available at: http://www.lbo.ms.gov/pdfs/2015_guide_to_pbb2.pdf.
- Mathematica Policy Research. (2015). Identifying programs that impact teen pregnancy, sexually transmitted infections, and associated sexual risk behaviors: review protocol, version 4.0. Available at: http://tppevidencereview.aspe.hhs.gov/pdfs/Review_protocol_v4.pdf.
- Mississippi Department of Education, (2016). MS Code 37-13-171 sex-related education (2011 legislation HB999) list of approved resources. Office of Healthy Schools. Available at: http://www.mde.k12.ms.us/docs/healthy-schools/listing-of-approved-curriculanew.pdf?sfvrsn=2.
- Miss Code of 1972 Ann. §37-13-171 et seq. (2011).
- National Conference of State Legislatures. (2015). Mississippi: Teen pregnancy, state policy options. Available at: http://www.ncsl.org/documents/health/TPreMSStatePolicyOptions115.pdf.
- Office of Adolescent Health, (2016). Evidence-based teen pregnancy prevention programs at a glance. United States Department of Health and Human Services. Available at: http://www.hhs.gov/ash/oah/oah-initiatives/teen_pregnancy/training/Assests/ebp-table.pdf.
