A Snapshot of K-8 Academic Achievement in Georgia

August 30, 2013

This month’s education update summarizes academic achievement of Georgia’s elementary and middle school students from 2009-10 to 2012-13.  While performance on the Criterion-Referenced Competency Tests (CRCT) has improved modestly during this time, a comparison with national tests shows that Georgia sets a low bar for proficiency on the CRCT and still ranks in the bottom half of national comparisons on the National Assessment of Educational Progress (NAEP).  This education update was originally published in August and has been revised to include numbers from the 2012-13 academic year.

Georgia CRCT Performance

The CRCT are Georgia-specific standardized tests that measure student proficiency on the CCGPS and the Georgia Performance Standards (GPS) in core subjects. All Georgia students in third through eighth grade take CRCT tests in reading, English, mathematics, science, and social studies.

With the guidance of the Georgia Department of Education and the Governor’s Office of Student Achievement (GOSA), the State Board of Education sets what scores are considered below, meeting, and exceeding the established standards.  Students who either “meet expectations” or “exceed expectations” are considered proficient on Georgia assessments.

Over the past four years, the percentage of students statewide who are meeting or exceeding expectations has risen in all subject areas.[1]  As seen in Figure 1, over 90% of students tested either met or exceeded expectations in English and reading in 2012-13.  86% were proficient in mathematics, while 80% and 79% of students met or exceeded expectations in social studies and science, respectively.

As seen in Figure 2, the percentage of students exceeding expectations has also increased in each subject area. In 2012-13, 39% percent of students tested exceeded expectations in English, as did 44% in reading and 38% in mathematics. In science and social studies, 35% and 34% exceeded expectations, respectively.

Are Georgia’s CRCT Cut Scores Too Low?

While CRCT performance over the last four years shows modest improvement across all core subjects, Georgia’s assessments may provide an inflated picture of the number of proficient students. A recent Education Next article examining the rigor of states’ cut scores, or thresholds for determining proficiency on assessments, finds that Georgia has the lowest cut scores of any state in the nation on its 4th and 8th grade mathematics and reading assessments. Written by Harvard professor Paul Peterson and student Peter Kaplan, the analysis compares the percentage of students each state identified as “proficient” on 2011 state-developed assessments of mathematics and reading with the percentage of students identified as proficient on the 2011 National Assessment of Educational Progress (NAEP) for mathematics and reading. The NAEP is a biannual assessment given to a representative sample of 4th and 8th grade students in each state.[2]

The authors rate each state on an A-F scale according to the alignment of the proficiency rates on state assessments with the NAEP. An “A” or “B” indicates that the state and NAEP proficiency rates are closely aligned, and the state cut scores set high expectations for student performance, comparable to those set by NAEP. As such, the analysis does not compare states’ relative performance on the NAEP.

Georgia received an “F” and is ranked last among the 50 states and the District of Columbia, meaning Georgia sets low expectations for proficient performance on the CRCT. Alabama was the only other “F” state.[3]  Tennessee jumped from an “F” in 2009 to an “A” in 2011 by increasing the rigor of the cut scores on its standardized assessments.

The NAEP and Georgia’s CRCT have differences that should be considered when comparing results. NAEP assesses a representative sample of Georgia students that allows for comparisons between states and the nation, while the CRCT is designed to measure all Georgia students’ mastery of grade-level performance standards.[4]  A 2010 report published by the Center on Education Policy describes NAEP’s definition of proficient as a challenging achievement goal for students, an indication of what “students should know and be able to do.” The report indicates that proficiency levels on most states’ assessments are a closer comparison with the basic level on NAEP, which is defined as “partial mastery of prerequisite knowledge and skills that are fundamental for proficient work at each grade.”[5]

The Education Next article indicates that several states have made progress in closing the gap between the high level of NAEP expectations and state accountability-driven expectations. Tennessee took the difficult step of implementing more rigorous state assessments, a move which resulted in a sharp decrease in the number of students deemed proficient on those assessments. Massachusetts and Missouri have established similar expectations. Making adjustments in state expectations through the adoption of more rigorous assessments and cut scores can be challenging. However, the Georgia Department of Education has indicated that it will be developing new state assessments and cut scores in response to the need for just such rigor and high expectations. The comparison in the next section further indicates the need for this move in Georgia.[6]

Comparison of CRCT and NAEP Results

GOSA compared the NAEP proficiency and basic or above rate with both the CRCT meets/exceeds and exceeds rate in 4th grade math and reading as well as 8th grade math, reading, and science.[7]  On all five tests over multiple years, the exceeds rate on Georgia’s CRCT was more consistent with NAEP proficiency or above rate than was the CRCT meets/exceeds rate. The CRCT meets/exceeds rate was still 3 to 26 percentage points higher than the NAEP basic or above rate, indicating that Georgia’s cut scores over-represent the number of proficient students even when compared with the less rigorous NAEP basic or above rate.

Reading

In 2012-13, the CRCT meets/exceeds rate was 59 percentage points higher in 4th grade and 66 percentage points higher in 8th grade than the NAEP proficiency or above rate. When compared to the NAEP basic or above rate, that gap narrowed to 26 and 23 points, respectively.  The CRCT exceeds rate more closely aligned with the NAEP proficiency or above rate, but it was still 13 percentage points higher in 8th grade and 15 percentage points higher in 4th grade in 2012-13.

In 4th grade reading in 2012-13, Georgia scored in the middle nationally.  Compared to Georgia, 16 states had lower average scores, and 15 states had higher average scores.  The average scores of 19 states were statistically indistinguishable from Georgia’s score.  In 8th grade, Georgia students scored higher than students in only 9 other states but lower than students in 22 states.  Nineteen had average scores that were not statistically different from Georgia’s score.

Figure 3: Performance on Reading Tests

Mathematics

In 2012-13, the CRCT meets/exceeds rate was more than two times the NAEP proficiency or above rate in 4th grade math and more than three times the NAEP proficiency or above rate in 8th grade math. The CRCT meets/exceeds rate was only 3 percentage points higher than the NAEP basic or above rate  in 4th grade math, but it was 20 percentage points higher in 8th grade math.  The CRCT exceeds rate closely corresponded to the NAEP proficiency or above rate.

In 2012-13, Georgia’s math scores were in the bottom half nationally.  On 4th grade math, Georgia students scored higher than students in 11 states and lower than students in 24 states. Georgia’s average score was statistically indistinguishable from 15 states.  In 8th grade math, Georgia’s average score was higher than 7 states’ average score and lower than 31 states’ average score.  Its score was not statistically different from 12 states’ scores.

Figure 4: Performance on Mathematics Tests

Science

The U.S. Department of Education did not administer the NAEP science assessment in 2012-13.[8]  In 2010-11, the CRCT meets/exceeds rate was more than double the NAEP proficiency or above rate for 8th grade science, while the CRCT exceeds rate fell 10 percentage points below the NAEP proficiency or above rate.  The CRCT meets/exceeds rate was five percentage points above the NAEP basic or above rate.

On the 2010-11 NAEP, Georgia students outscored students in 9 states but scored lower than students in 25 states.  Sixteen other states’ average scores were statistically indistinguishable from Georgia’s average score.

WYSIWYG: EMBEDDED IMAGE

Summary

Georgia’s CRCT scores have increased modestly from 2009-10 to 2012-13.  However, the cut scores used to determine proficiency (the CRCT meets/exceeds rate) over-represent the percent of Georgia students who are proficient in mathematics, reading, and science relative to NAEP results.  Instead, the CRCT exceeds rate more closely aligned with Georgia’s performance on the NAEP, both of which have increased in recent years.  Still, 49% or fewer of Georgia elementary and middle school students were proficient in each subject when using higher expectations (CRCT exceeds rate) comparable to those set by NAEP.  As the state looks to redesigning state assessments in the coming year to align with Georgia’s academic standards (CCGPS), it should pursue more rigorous assessments with cut scores that raise the bar for student mastery of standards.

[1] CRCT numbers are from GOSA’s Report Card: http://gosa.georgia.gov/report-card.

[2] NAEP is also given periodically in other subjects and grade levels. For more information, click on this link: http://nces.ed.gov/nationsreportcard/about/

[3] Georgia moved from a “C-” in 2009 to an “F” in 2011. Georgia’s previous ratings were “D-” in 2003, “D-” in 2005, and “F” in 2007.

[4] For more information, click on the following link: http://nces.ed.gov/nationsreportcard/about/comparing_assessments.aspx

[5] For full definitions of each performance level, click on the following link: http://nces.ed.gov/nationsreportcard/achievement.aspx

[6] Additional NAEP information is also available on an earlier GOSA education update, Georgia vs. the Nation’s Report Card. This 2007 publication can be found at http://gosa.georgia.gov/sites/gosa.georgia.gov/files/related_files/document/E-Bulletin_NAEP_11_29_07.pdf .

[7]NAEP scores were obtained from the National Center for Education Statistics: http://nces.ed.gov/nationsreportcard/states/.  Rankings are provided by NAEP: http://nces.ed.gov/nationsreportcard/statecomparisons/.  

The 8th grade science NAEP was not administered in 2007. The 4th grade test was administered in 2009 but not 2011.

CRCT scores are reported from GOSA’s 2007, 2009, and 2011 Report Cards, accessible here: http://gaosa.org/report.aspx

[8] http://nces.ed.gov/nationsreportcard/about/current.aspx#naep2013