Examining Statewide CCRPI Score Patterns

May 31, 2013

In February 2012, the U.S. Department of Education (US ED) granted a waiver from some accountability requirements of the Elementary and Secondary Education Act (ESEA), more commonly known as No Child Left Behind (NCLB), to Georgia and ten other states. The waiver included federal requirements, which GOSA summarized in a September 2012 e-bulletin, and a unique aspect of Georgia’s application, the College and Career Readiness Performance Index (CCRPI), which replaces Adequate Yearly Progress (AYP) as Georgia’s new statewide public school accountability measure.

On Tuesday, May 7, the Georgia Department of Education (GaDOE) released the first CCRPI scores based upon the 2011-12 school year. This e-bulletin provides an analysis of CCRPI summary statistics and relevant patterns across the state.

CCRPI Overview

The total possible CCRPI score is 110 points (100 points plus 10 Challenge Points). All calculations are based upon Full Academic Year (FAY) students, meaning students who are enrolled for at least 65% of the school year and during the testing window. Scores are split into Elementary (K-5), Middle (6-8), and High School (9-12) categories. Schools with grade configurations in multiple categories receive a separate CCRPI score for both categories. The index includes the following components:

Achievement (70 points, 63.7% of total possible score)

Content Mastery: CRCT/EOCT tests in core subjects (40% of achievement measure, 25.5% of total possible score)

Post High School Readiness: Includes career pathways, ACT/SAT/AP/IB exam performance, world language coursework, reading/writing skills, and attendance (30% of achievement measure, 19.1% of total possible score)

Graduation Rate: Four-year and five-year cohort graduation rates in high schools or a Graduation Rate Predictor for elementary and middle schools, which is an additional, different look at CRCT performance (30% of achievement measure, 19.1% of total possible score)

Progress (15 points, 13.6% of total possible score))

The percentage of a school’s students showing typical or high growth in their Student Growth Percentiles (SGPs), which compare a student’s score growth in a tested subject with other students from across the state who have similar past achievement (13.6% of total possible score).

Achievement Gap (15 points, 13.6% of total possible score))

Based upon schools’ achievement gap size and ability to close it. The gap is measured between the schools’ lowest 25% of achievers and the state average (13.6% of total possible score).

Challenge Points (10 possible points, 9.1% of total possible score) as a sum of the following:

Exceeding the Bar (ETB) Points: Ten items worth 0.5 point each that focus on innovative practices and career-related outcomes. Only the top 5% in each category qualify for points, with the exception of the innovative practice point.

EL/ED/SWD Performance Points: Based upon subgroup performance of economically disadvantaged students (ED), students with disabilities (SWD), and English Language Learners (EL) on performance flags set forth in the ESEA waiver. Schools can only receive points proportionally for the percentage of students in these subgroups in their school. For example, schools with 50% of students in one or more of these groups can receive a maximum of 5 points (50% of 10 points) if every subgroup meets all targets.

Across the state, 148 schools lack a Progress and/or Achievement Gap score because those schools have fewer than 15 students in relevant reference groups. Small schools are more likely to lack either a Progress or Achievement Gap Score for this reason. In addition, primary schools (K-2) do not have either score because CRCT tests are not given until 3rd grade. As a result, their Achievement Scores, which are based on proficiency, weigh more in the final scores relative to schools with scores in each of the three categories.

For more details regarding the contents of each component, visit the GaDOE’s CCRPI website at http://ccrpi.gadoe.org/2012/.

Overall Summary Statistics

The average CCPRI score for both elementary and middle schools is 80.8, whereas the average for high schools is 70.8. [1]  High schools received lower scores largely because middle and elementary schools earn more Achievement and Challenge points. The distribution of Progress Points and Achievement Gap points is similar for all school types. Click on this link to view the score distributions of each component.

Relationships among Score Components

The graphs below demonstrate the relationship between Achievement Points and the other components of the CCRPI score. In general, schools that receive high scores in Achievement also receive high Progress and Achievement Gap scores. Thus, the inclusion of a growth component does not seem to provide strong benefit to schools with low proficiency.

As seen in the graph below, the relationship between Achievement points and Challenge points is less straightforward.  Schools at all levels of Achievement points receive no or few Challenge points.  However, schools receiving a high number of Challenge points tend to earn higher Achievement points.

School Characteristics and CCPRI Scores

Urban Locale

Schools located in cities performed significantly lower than schools in suburbs, towns, or rural areas.  Suburbs, towns, and rural areas performed similarly in each category. [2] High schools received the lowest scores regardless of locale. Click here to see the supplementary tables with scores by urban locale.

Percent of Students in Poverty

In all school types, as the percent of students eligible for free/reduced price lunch increases, the school’s CCRPI score decreases.  While this relationship exists, schools at nearly all poverty levels were able to achieve high CCRPI scores.

WYSIWYG: EMBEDDED IMAGE

The Impact of Challenge Points on Overall Score

In general, schools receive a boost from Challenge points.  On average, elementary schools receive 4.1 Challenge points, middle schools receive 3.7 Challenge points, and high schools receive 1.9. [3] However, as seen below, schools with higher CCRPI scores earned more Challenge Points than schools receiving lower CCRPI scores. As such, Challenge Points tend to help schools that already score at the high end of the distribution. The chart below details the effect of Challenge points on the CCPRI score. 

In general, Challenge Points appear help high-performing schools and districts with high proportions of students in poverty.  Overall, the EL/ED/SWD Performance Points appear to be helping high-performing schools with a proportion of students in those subgroups, while the ETB Points are more evenly distributed. Click here for supplemental graphs about this relationship.

Conclusion

The CCRPI provides a comprehensive look at the school performance across the state. In the weeks ahead, GOSA will produce another e-bulletin with indicator-level analysis. In addition, GOSA will examine how the CCRPI can be used for new criteria for Single Statewide Accountability System (SSAS) awards that were previously based upon AYP.

Link to Supplementary Tables and Figures.

[1] The averages given in this e-bulletin differ from averages in the GaDOE’s press release because the GaDOE rolled up the scores to the state level, meaning that the CCRPI formula was applied at the state level for all students. The calculations in this report represent a mean score that counts every school as one entity, regardless of size.

[2] Locales as defined by the National Center for Education Statistics (NCES): http://nces.ed.gov/ccd/commonfiles/localedescription.asp

[3] The elementary school average falls from 80.8 to 76.7 when Challenge points are removed.  The middle school average falls from 80.8 to 77.1, and the high school average falls from 70.8 to 68.9.