Blog

The Kingsbury Center recently completed a study aligning the proficiency cut scores on the California STAR (the state achievement test used for elementary and middle grades) and CAHSEE (the high school test) exams to the NWEA scale.  One value of these studies is that we can see how the proficiency cut scores on a state’s test compare in difficulty across grades. 

The graph below shows the level of difficulty of the various cut scores on STAR math as they compare to the NWEA norms.

What one notices is the lack of consistency - or “calibration” - in the tests’ math cut scores across the various grades.  The cut scores at grades 2, 3, and 4 (which are below the 40th percentile of our norm group) are considerably easier than the cut scores for the middle grades (which range as high as the 60th percentile).  The cut score on the high school test (the CAHSEE) reverts back to about the same relative level as the early grades.

So what’s the problem?  First, the differences in cut scores can lead policy makers and the public into serious misunderstandings about the performance of schools.  For example, because the cut score at the elementary grades is lower, a larger proportion of students will pass the test in elementary schools than are likely to pass at middle schools.  When these results get reported in the paper, readers are likely to assume that elementary schools are doing a better job.  In truth, middle school students are simply taking a tougher test (relative to their abilities). 

At the other end, the high school test is also easier than the middle school assessment.  Why?  It’s easier because students must pass the CAHSEE to qualify for a high school diploma.  With stakes this high, setting a standard that 60% of our norming group would fail is not politically viable.

How did this happen?  The primary problem is that the proficiency standards system wasn’t designed to send a coherent message.  The CAHSEE was intentionally designed to be less rigorous than the STAR because all students were required to pass it.  As a result, parents are confused when kids who failed STAR pass CAHSEE two years later.  And because elementary standards were set lower than middle school standards, kids who pass the elementary test may, even if they make normal progress, fail the middle school assessment.  This makes it impossible for parents and policy makers to reach any reasonable conclusion about how their own kids or their schools are really performing. 

The lesson? Cut scores should represent a consistent standard of performance across grades.  A student who passes at grade 2 should, if he or she makes normal progress, pass at grade 5, grade 8, and grade 10.  This assures that parents know whether their children are truly on track, and allows the public to better assess the performance of their schools.

If you’re interested in reading the reports, you can find them here:

California Linking Study: STAR

California Linking Study: CAHSEE