Blog

Please welcome our guest blogger, Kingsbury Center Summer Intern Clay Johnson!  Clay is a doctoral student in Educational Statistics and Research Methods at the University of Arkansas at Fayetteville.

This week the US Department of Education released a new report linking individual states’ cut scores for labeling a student “proficient” with the latest results from NAEP (the National Assessment of Educational Progress, or “The Nation’s Report Card”).  The resulting headlines focus on the change in some states’ difficulty level for scoring proficient since the last round of NAEP scores.  More interesting for me though are these four new sets of rankings comparing the difficulty of passing each state’s test, since a very similar set of state rankings was recently published online by the Kingsbury Center in an interactive “data gallery.”

I find it fascinating to compare the rankings based on NAEP side by side with the Kingsbury Center rankings based on MAP.  If you’d like to play along, start with page 10 of the USDE report and select the appropriate filters in the Kingsbury Center data gallery. 

For example, among the 37 states ranked in both studies, my home state of Arkansas ranks 19th and 25th in reading cut score difficulty according to NAEP.  Judging by MAP scores, Arkansas ranks 16th and 29th – not great for rigor, but not bad for consistency.  My eyes are drawn to states like New York, where MAP places their reading tests in 1st and 2nd place for difficulty, but NAEP ranks them 17th and 19th.  Conversely, NAEP puts New Jersey 2nd in fourth grade reading expectations, but MAP has them at 35th – near the very bottom.

Granted, there are significant differences between these two studies that should be considered for any purpose beyond my simple observations.  NAEP and MAP differ in content and testing season, they are administered to different groups of students for different purposes, and one is paper-based while the other is computer adaptive.  MAP scores are better at predicting cut scores for some states, and NAEP is more reliable in others.  But despite the inherent complications, the big picture seems clear: Despite the efforts at the federal level toward promoting rigorous expectations for students across the US, states continue to vary vastly in what is required in their schools.  I look forward to seeing what progress toward national consistency, if any, might come with the upcoming switch to common standards and consortia assessments.