2009 Spring CRCT Analysis

February 10, 2010

The Governor’s Office of Student Achievement (GOSA) today released the results of a spring 2009 Criterion Referenced Competency Test (CRCT) erasure analysis. GOSA partnered with CTB-McGraw Hill (CTB), the state’s testing vendor in charge of developing and scoring CRCT exams, to conduct a comprehensive examination of all statewide CRCT answer documents for grades 1 through 8. The analysis focused on the number of wrong answers that had been changed to right answers on individual student answer sheets in Reading, English-Language Arts, and Mathematics.

“The analysis looked on average at 125,000 test takers in every subject and grade level at which the CRCT was administered and provided a clear picture of typical student test behavior against which all schools could be compared,” said GOSA Executive Director, Kathleen Mathers. “Our recommendations are intended to eliminate future problems and help students who have been adversely affected by test tampering.”

In the analysis, CTB psychometricians scanned answer documents to identify total erasures per classroom, flagging those classrooms in which the number of wrong-to-right changes proved to be three standard deviations (SDs) or more above the state average. Less than 0.15% of test takers would be expected to fall in that range naturally.

Based on the analysis, schools were placed in varying categories according to their percentage of flagged classrooms. 80% of Georgia’s elementary and middle schools fell into the “Clear of Concern” category, meaning less than 6% of the classes within a given school were flagged; 10% fell into the “Minimal Concern” category with 6%-10% of classes flagged; 6% were determined to be in the “Moderate Concern” category with 11%-24% of classes flagged; and only 4% were termed “Severe Concern” as defined by a school having 25% or more of its classes flagged for wrong-to-right changes.

Recommendations on which the State Board of Education will vote range from requiring local Superintendents to conduct internal investigations to determine the causes of testing irregularities, to schools rotating teachers during the 2010 CRCT test administration so that they administer the test to students they have not taught. In addition, state monitors will be placed in all schools in the severe concern category during this spring’s test.

“Important decisions will be made from this data that are critical to the future of Georgia’s children,” said GOSA Deputy Director, Dr. Eric Wearne. “Overall, Georgia’s schools are performing well and continue to excel in student achievement.”

The CRCT is a standardized assessment given to students in grades 1-8 in Georgia. The test is designed to measure how well students at each grade level have learned the state’s curriculum. CRCT results are used to determine whether schools have made Adequate Yearly Progress (AYP) as required by No Child Left Behind (NCLB).

GOSA plans future analyses of standardized test scores, possibly including End of Course Tests (EOCT) and Georgia High School Graduation Tests (GHSGT) and will also examine graduation and dropout rates and other factors that determine student achievement. The full 2009 CRCT erasure analysis report is included below.

The Governor’s Office of Student Achievement serves as the reporting and accountability agency for education in Georgia. As such, GOSA is charged by law with inspecting academic records of schools to ensure that education institutions are faithful to performance accountability requirements. Through an academic audit, GOSA reviews student assessment data and other school records reported to the State to confirm accuracy and explore the effectiveness of local school initiatives in improving achievement.

Contact Information: 
Related Files: