The Governor’s Office of Student Achievement (GOSA) today released the results of the spring 2017 Milestones End-of-Grade Test (EOG) and End-of-Course Test (EOC) assessment audit that included an answer change analysis and an unusual response analysis. The answer change analysis, formerly known as the erasure analysis, examined the prevalence of wrong-to-right answer changes. The unusual response analysis identified schools with testing groups that had unusual test gains as well as unusual student response patterns. 

GOSA partnered with Data Recognition Corporation (DRC), the state’s testing vendor in charge of developing and scoring Milestones exams, to conduct both analyses. The answer change analysis included a comprehensive examination of all 795,191 answer forms for the EOG and all 789,756 answer forms for the EOC in all subject areas. The unusual response analysis examined 14,704 testing groups in grades 4-8 on the EOG Mathematics and ELA assessments.[i]

For the EOG answer change analysis, 37 of 1,814 elementary and middle schools had at least one classroom flagged for wrong to right (w-t-r) erasures that required further inquiry after the State’s initial desktop audit using the data from DRC’s analyses, a decrease from 54 schools in 2016.  For the EOC answer change analysis, 9 schools out of 891 schools had at least one classroom requiring further inquiry, a decrease from 30 schools in 2016.  The number of schools flagged in both analyses decreased largely because 2017 was the second year of the inclusion of online assessments, allowing for a better understanding of answer change patterns when reviewing the data to determine a root cause for a flag without needing further inquiry.

For the EOG unusual response pattern analysis, 43 of 1,655 schools had at least one testing group identified for further inquiry.  The analysis looked at unusual score gains and unusual response patterns for English and Mathematics test-takers in grades 4-8. Unexpected patterns include (1) unlikely blocks of consecutive, identical answers, (2) highly correlated answers across tests, (3) correlation of responses across test items, and (4) cases where students miss easy items but answer difficult questions correctly.

After a thorough review of the inquiry reports, GOSA closed the audit and requires no further additional information for 82 of the schools identified for further inquiry.  Due to testing irregularities observed by test monitors during 2018 Milestones testing, the audit of Cirrus Charter Academy will remain open.  GOSA, in coordination with GaDOE, will continue to work with Cirrus Charter Academy to ensure proper controls are put into place to eliminate future testing irregularities.  All schools audited will be monitored in future years and may be subject to further action if identified for further inquiry in a future audit.

For more information on the audit process, click Download this pdf file. here .

For more information on GOSA’s academic auditing program, click here.

For the 2017 Georgia Milestones Assessment Audit State Board Presentation, click Download this pdf file. here .

For the Spring 2017 Georgia Milestones Assessment Desktop Audit Results, click Download this pdf file. here .

For the 2017 Georgia Milestones Assessment Inquiry Results Report, click Download this pdf file. here .

[i] Each testing group is the total number of students by grade level and subject area (ELA or mathematics) who took a certain test form (A or B) regardless of classroom assignment. For example, all students in a school who took the 4th grade mathematics Georgia Milestones Form A assessment are a testing group.