Mississippi Miracle or Mirage? [Updated]
2019 NAEP Reading Scores Prompt Questions, Not Answers
[PLEASE see the series of UPDATES added at my primary blog]
—
There is a disturbing contradiction in the predicted jubilant response to Mississippi’s outlier 4th-grade results from the 2019 NAEP reading test. That contradiction can be found in a new article by Emily Hanford, using Mississippi to recycle her brand, a call for the “science of reading.”
This is a great deal to ask of the average reader, but Hanford’s argument is grounded in a claim that most students in the U.S. are being taught reading through methods that are not supported by scientific research (code for narrow types of quantitative research that can identify causal relationships and thus can be generalized to all students).
However, the contradiction lies in Hanford’s own concession about the 2019 NAEP reading data from Mississippi:
The state’s performance in reading was especially notable. Mississippi was the only state in the nation to post significant gains on the fourth-grade reading test. Fourth graders in Mississippi are now on par with the national average, reading as well or better than pupils in California, Texas, Michigan and 18 other states.
What’s up in Mississippi? There’s no way to know for sure what causes increases in test scores [emphasis added], but Mississippi has been doing something notable: making sure all of its teachers understand the science of reading.
To be fair, there is a way to know, and that would be conducting scientific research that teases out the factors that can be identified as causing the test score changes in the state.
In her missionary zeal for the “science of reading,” Hanford contradicts herself by taking most of the article to imply without any scientific evidence, without any research, that Mississippi’s gains are by her fervent implication a result of the state’s embracing the “science of reading”: “In 2013, legislators in Mississippi provided funding to start training the state’s teachers in the science of reading.”
Let me stress here a couple points.
First, scientific research connecting classroom practices to NAEP test scores is rare, but in the 1990s, comparative data were released on 1992 scores in 1997. That research showed a possible link between whole language practices and higher NAEP scores — something that Hanford and her “science of reading” followers may find shocking since they routinely claim that whole language and balanced literacy are not scientifically supported.
Therefore, it is simply far too soon after the release of the 2019 NAEP scores to suggest any relationship between classroom practices (as if they are uniform across an entire state) and NAEP scores. Any implications about Mississippi are premature and irresponsible to make for journalists, politicians, or advocates for education.
Premature and irresponsible.
Second, data from Mississippi are more than 4th-grade 2019 reading — if we genuinely want to know something of value about teaching children to read.
Mississippi’s outlier 4th-grade reading scores are way more complicatedonce we frame them against longitudinal NAEP scores as well as 8th-grade reading scores. These, then, are more data we should using to ask questions about Mississippi instead of making rash and unscientific claims:
Here are some complicated takeaways from this larger picture:
- If the “science of reading” is the cause of recent gains in 4th-grade reading in MS, how do we explain that MS has seen a trend of increased scores since 1998 and pretty significant jumps between 2005 and 2009[1], well before the shift identified by Hanford in 2013?
- Why does MS still show about the same gaps between Black and white students as well as between socioeconomic classes of students since 1998 if how we teach reading is the key factor in achievement?
- And a really powerful question concerns 8th grade: Are any 4th-grade gains by MS (or any state) merely mirages since many states with 4th-grade gains see a drop by 8th grade and since longitudinal 8th-grade scores are mostly flat since 1998?
- UPDATE: Todd Collins has raised another important caveat to the 4th-grade reading gains in Mississippi because the state has the highest 3rd-grade retention percentages in the country:
But Mississippi has taken the concept further than others, with a retention rate higher than any other state. In 2018–19, according to state department of education reports, 8 percent of all Mississippi K–3 students were held back (up from 6.6 percent the prior year). This implies that over the four grades, as many as 32 percent of all Mississippi students are held back; a more reasonable estimate is closer to 20 to 25 percent, allowing for some to be held back twice. (Mississippi’s Department of Education does not report how many students are retained more than once.)
This last concern means that significant numbers of students in states with 3rd-grade retention based on reading achievement and test scores are biologically 5th-graders being held to 4th-grade proficiency levels. Grade retention is not only correlated with many negative outcomes (dropping out, for example), but also likely associated with “false positives” on testing; as well, most states seeing bumps in 4th-grade test scores also show that those gains disappear by middle and high school.
Ultimately, if anyone wants to argue that how we teach reading in the U.S. must be grounded only in a narrow view of “scientific” (and that is a terrible argument, by the way), then any claims we make about the effectiveness of those practices must also be supported by scientific research.
Despite efforts to make Mississippi a shining example of how all states should address reading policy, we should be using Mississippi (and the 29 states scoring higher) to examine all the factors contributing to why students achieve at the levels they do on NAEP reading.
Unless of course we have real political courage and are willing to admit that NAEP and any form of standardized testing are the wrong way to make these decisions.
Here’s something to think about in that regard: As long as we use this sort of testing, we will always have some states above the average, several at the average, and some below the average — resulting in the same nonsensical hand wringing we see today that is no different than any decade over the last 100 years.
I recommend instead of all the scientific research needed to make any fair claim, we stop the testing, make teaching and learning conditions better, make the lives of children and their families in the U.S. better, and do the complicated daily work it requires to serve the needs of all students.
NOTE
[1] Hanford contradicts herself again and open the door to another question:
For years, everyone assumed Mississippi was at the bottom in reading because it was the poorest state in the nation. Mississippi is still the poorest state, but fourth graders there now read at the national average. While every other state’s fourth graders made no significant progress in reading on this year’s test, or lost ground, Mississippi’s fourth-grade reading scores are up by 10 points since 2013, when the state began the effort to train its teachers in the science of reading. Correlation isn’t causation* [emphasis added], but Mississippi has made a huge investment in helping teachers learn the science behind reading.
There is an 8-point jump in 4th-grade reading in MS from 2002 to 2009 — well before the 2013 shift to the “science of reading” — thus how is that explained?
* For the record, causation is a key component of “scientific,” which Hanford espouses for reading, yet she stoops to correlation (not scientific) to make her argument.
