The Governor’s Office of Student Achievement (GOSA) today released the results of the spring 2016 Milestones End-of-Grade Test (EOG) and End-of-Course Test (EOC) assessment audit that included an answer change analysis and an unusual response analysis. The answer change analysis, formerly known as the erasure analysis, examined the prevalence of wrong-to-right answer changes. For the first time, this analysis included tests administered online rather than only paper and pencil. In addition, a newly conducted unusual response analysis identified schools with testing groups that had unusual test gains as well as unusual student response patterns. 

For the EOG answer change analysis, 54 of 1,811 elementary and middle schools had at least one classroom flagged for wrong to right (w-t-r) erasures that requires further inquiry after the State’s initial desktop audit, an increase from 19 schools in 2015.  This increase is largely due to the inclusion of online test-takers to the analysis (433,597 in 2016).

For the EOC answer change analysis, 30 schools out of 866 schools had at least one classroom requiring further inquiry, an increase from 14 schools in 2015.  As with EOG, the inclusion of schools testing online (721,082 in 2016) was largely responsible for an increase in the number of schools flagged for further inquiry.

For the EOG unusual response pattern analysis, 35 of 1,648 schools had at least one testing group identified for further inquiry.  The analysis looked at unusual score gains and unusual response patterns for English and Mathematics test-takers in grades 4-8 for which a natural cohort of students could be tracked from one year to the next. Unexpected patterns include (1) unlikely blocks of consecutive, identical answers, (2) highly correlated answers across tests, (3) correlation of responses across test items, and (4) cases where students miss easy items but answer difficult answers correctly.

Having a school or classroom flagged for further inquiry is not considered in itself an indication of cheating but rather a signal that further review is warranted.  In most cases, the review simply helps the school or district strengthen and refine the testing procedures.

GOSA partnered with Data Recognition Corporation (DRC), the state’s testing vendor in charge of developing and scoring Milestones exams, to conduct both analyses. The answer change analysis included a comprehensive examination of all 784,646 answer forms for the EOG and all 795,721 answer forms for the EOC in all subject areas. The unusual response analysis examined 16,326 testing groups in grades 4-8 on the EOG Mathematics and ELA assessments.[i]

For more information on the audit process, click Download this pdf file. here .

For more information on GOSA’s academic auditing program, click here.

For the 2016 Georgia Milestones Assessment Audit State Board Presentation, click Download this pdf file. here

For the Spring 2016 Georgia Milestones Assessment Desktop Audit Results, click Download this pdf file. here

For the list of 2016 Georgia Milestones EOG/EOC Schools Requiring Further Inquiry, click Download this pdf file. here

 

[i] Each testing group is the total number of students by grade level and subject area (ELA or mathematics) who took a certain test form (A or B) regardless of classroom assignment. For example, all students in a school who took the 4th grade mathematics Georgia Milestones Form A assessment are a testing group.