Texas students taking state-mandated tests this week are being used as guinea pigs for a new artificial intelligence grading system that will replace most human graders in the district.
texas tribune The Texas Education Agency (TEA) is rolling out an “automated scoring engine” that leverages natural language processing, a technology that enables chatbots like OpenAI’s ChatGPT to understand and communicate with users to provide insights about the state, the report said. open-ended questions. Texas Assessment of Academic Readiness (STAAR) exam. The agency estimates the system will save $15-20 million annually by reducing the need for temporary human raters, and plans to hire 2,000 raters this year, compared with the 6,000 needed in 2023.
“We want to keep as many of the constructed open-ended responses as possible, but they take a lot of time to score.”
The STAAR exam, which tests students in third through eighth grade on their understanding of the core curriculum, was redesigned last year to reduce the number of multiple-choice questions. It now contains seven times more open-ended questions, and Jose Rios, TEA’s director of student assessment, said the agency “wants to retain as many structured open-ended responses as possible, but they take an incredible amount of time to grade.”
According to a slideshow on the TEA website, the new scoring system was trained using 3,000 exam answers, which had undergone two rounds of human scoring. Some safety nets have also been implemented – for example, a quarter of all computer-rated results will be re-scored by humans, and answers that confuse AI systems will also be re-scored by humans (including using slang or non-English answers).
While TEA is optimistic that AI will allow it to save significant amounts of cash, some educators are not so keen to see it implemented. Louisville Independent School District Superintendent Lori Rapp said the district saw a “dramatic increase” in constructed answers receiving zero points in December 2023, when the automated scoring system came into limited use. “At this time, we cannot determine whether there is a problem with the test question or a problem with the new automated scoring system,” Rapp said.
Artificial intelligence essay scoring engines are nothing new. 2019 report from motherboard They were found to be used with varying degrees of success in at least 21 states, although TEA seems determined to avoid the same reputation. The fine print on TEA’s slides also emphasizes that its new scoring engine is a closed system and is inherently different from artificial intelligence in that “AI is a computer that adapts using progressive learning algorithms, allowing data to be programmed, essentially teaching itself .”
It’s not surprising to try to draw a line between them—there’s no shortage of teachers online despairing at how generative AI services can be used to cheat on assignments and homework. Students graded under this new grading system may have a hard time accepting the way they think “the rules are for you, not for me” are being applied here.