Last year, the number of students who received a zero on their State of Texas Assessments of Academic Readiness (STAAR) writing questions raised an alarm among educators.
The cause for concern stemmed from the new changes to the test, such as online testing and new types of questions. The new STAAR 2.0 redesign also indicates a shift toward machine-driven evaluation and the use of artificial intelligence (AI) in the education sector.
This has led to debates among teachers about transparency, algorithm bias, and the impact of these changes on both teaching and learning. They also consider that automated scoring may lead to the widening of inequities in education, as such scoring methods are dependent on data.
To do well in these exams, teachers may also feel the need to resort to teaching methods that try to beat the system instead of nurturing a student’s creativity and skills.
How automated grading systems play a role
The Texas Education Agency (TEA) now grades 75% of essays using automated scoring engines that are powered by natural language processing and machine learning.
The shift was a response to STAAR’s redesign — with an increased number of writing assessments and essay questions for all grade levels.
According to educators, the TEA did not provide adequate communication or a pilot period for the new system. Moreover, the increase in the number of zero scores in essay questions contradicted TEA’s claims that an automated system of grading is similar to that of humans.
The Texas Education Agency is now offering more information about the test. District testing coordinators can inquire about their students’ extended constructed responses (ECR) if they contact TEA before March 13.
The new changes are a result of House Bill (HB) 3906 and House Bill 3261, which was passed by the 86th Texas Legislature in 2019 and implemented in the 2022–2023 school year when writings were still being graded by human beings. HB 3906 implemented changes to the exam to “better align with classroom instruction” and placed a cap on the number of multiple-choice questions on the exam. HB 3261 said the STAAR test has to be administered online.
The results of the STAAR exam:
During the STAAR exams in fall 2023, a spike in zeroes was noted, with almost 80% of written responses on the English II End-of-Course exam receiving a zero score. On the other hand, around 25% of these responses received a zero score in spring, which was graded by humans. TEA said the difference can be attributed to the “seasonal variations in student populations” as several students in the fall are “re-testers.”
A large number of students from grades 3-8 received a zero score on the “constructed response” writing questions in spring 2023. Before the STAAR 2.0 redesign was implemented on the test, 5% of students in Texas scored 0 on 8, but after the STAAR 2.0 redesign, the number rose to 42% of students in the third grade, 46% in the fourth grade, 25% in the fifth grade, 23% in the sixth grade, 20% in the seventh grade, 23% in the eighth grade, and 25% in high school.
| Grade | Percentage of students who received a 0 score |
|---|---|
| 3 | 42% |
| 4 | 46% |
| 5 | 25% |
| 6 | 23% |
| 7 | 20% |
| 8 | 23% |
Data source: Texas Education Agency Research Portal
- Among these students, the third-, fifth-, sixth- and eighth-graders were being tested in writing for the first time.
- Black or African American students in Texas showed the highest percentage of zero scores on these questions compared to Hispanic/Latino, white and Asian students.
In the third, fourth and fifth grades, the same percentage of Black or African American and Hispanic/Latino students received a 10-point score on constructed response questions —0%, 1%, and 2%, respectively. While the percentage increased slightly in the sixth, seventh and eighth grades, they are significantly lower than white and Asian students in these grades.
Before automated scoring was introduced, one in four high school students, 19% of first-time test takers, and half of retest takers scored a zero on writing compositions in spring 2023. After the changes were implemented, 70% of high schoolers scored a zero on English I composition and around 80% on English II with automated scoring.
The increase in the number of students receiving zeros in these questions, however, denotes the number of students who failed the STAAR assessment. In 2023, 78% of fourth graders passed the reading language arts (RLA) STAAR test, while 77% passed the year before.
- The chart above shows that the percentage of students in the fourth grade who received a zero score on writing composition questions rose significantly after the STAAR test was redesigned — from 2% in 2017, 4% in 2018, 6% in 2019, and 3% in 2021 to 46% in 2023. Moreover, the percentage of Black students who scored a zero was higher than the average [comprising all students] for that year. Before the changes, students could receive a score of zero or 2-8. After the changes, the scoring system changed scores from 1-10. TEA did not release data for the 2019-2020 and the 2021-2022 school years.
- This chart shows that the percentage of seventh-grade students who received a zero score on writing composition questions rose significantly after the STAAR test was redesigned — from 2% in 2017, 2% in 2018, 3% in 2019, and 5% in 2021 to 20% in 2023. Based on the scoring system for such questions before the redesign, each answer could receive either zero or a score from 2-8. After the changes, each answer could get a score of 1-10. There is no data available for the 2019-2020 and the 2021-2022 school years.
The scoring guide lays out the strict criteria for marking “0” on an answer relating to a fourth-grade student’s idea presentation and expression, organization of thoughts, provision of evidence, and command over the language, among other parameters. Meanwhile, scoring models vary according to the different types of questions from grades 3-8.
To “approach” or pass a grade level, students needed to get half the test questions correct prior to the changes to the test. The changes in the question format, which allows partial credit, students need to receive credit for a third of the questions to pass the level.
Let’s rewind for a second
Last year, TEA made several changes to the STAAR exam, called the STAAR 2.0 redesign, for the 2023-2024 school year to comply with the Texas Legislature’s new laws.
Under the new rules, multiple-choice questions will comprise 75% of these tests. The rest of the questions will be a mix of other formats, depending on the grade of the students, such as
- Equation Editor, where students are required to write the answers,
- Text Entry, which requires students to type in a brief answer,
- Graphing, in which students can create graphs,
- Number Line, where students can answer on a number line,
- Inline Choice with a drop-down menu,
- Hot Spot and Hot Text, in which students can select areas of a graphic and cite evidence, respectively,
- Fraction Model with a space to answer in fractions,
- Drag and Drop for answer options,
- Multipart questions and Multiselect with multiple correct answers,
- Match Table Grid, which asks students to match statements or objects to categories in a table grid,
- Short Constructed Response, which requires students to provide an explanation to prove their understanding of a context, and
- Extended Constructed Response with in-depth responses.
Thus, all students from grades 3-8 had to tackle a new style of questioning and write their answers through an online platform, with writing included in every grade level.
Why the test scores are important to school districts
The STAAR test scores are also one of the criteria used by TEA to calculate its academic accountability ratings for public school districts. These include state student achievement in standardized tests, graduation rates, college, career, and military readiness, school progress, closing the achievement gaps among students of different races and ethnicities, and overall scores and ratings.
Teachers and school leaders are concerned about the new writing assessments and their impact on students’ creativity, reported Houston Public Media. However, the TEA is confident in its approach.
Houston ISD superintendent Mike Miles had said the STAAR test results from last year reveal that students need attention as the students “didn’t do well.” “They confirm that we should be bold,” he had said. “They confirm that the future’s already here and our kids are not ready for that.”
He also said that the achievement gap between students based on their ethnicities was another issue. During a press conference, he presented that in 2022, white students read at 60% proficiency, while Black students read at 11% proficiency and Hispanic students at 14% proficiency.s
“The gap is too large, it’s 49% for African Americans, it’s 46% for Hispanic students in HISD,” he said. “I think it is an equity issue.”
The TEA takeover of HISD was also primarily due to poor academic performance at Phillis Wheatley High School, a school with a 99.6% minority enrollment and more than 50% Black students.
The board of manager, appointed by TEA, aim to “improve student outcomes,” which includes increasing the number of students who meet STAAR grade levels.
More than 100 school districts in Texas sued the TEA over the A-F scale of grading schools that posed a risk of taking “A” schools and turning them into “C” grade schools.
For Miles’ New Education System (NES) schools, which had struggled with these tests, he created and implemented a centrally-created curriculum.
