A Closer Look at State Assessments

I recently had the occasion to look at Washington’s achievement data on two key performance indicators: 4^th grade reading, and 8^th grade math.

I chose these data points because they represent key transition points in each student’s educational career. Research tells us students who are not reading at grade level by the end of 3^rd grade have difficulty catching up thereafter. Indeed, 3^rd grade reading proficiency is a strong statistical predictor of high school graduation. Similarly, middle school math represents a key transition point– students not achieving math proficiency in middle school are less likely to be college and or career-ready by the end of high school, and qualify for the high-skill, living wage jobs (many of them STEM-related) Washington is having difficulty filling.

Which Story Do We Believe? Juxtaposing State vs. National Data on Math and Reading

In my review, one finding sticks out. On state assessments, the Washington Report Card has more students achieving proficiency reading in 4th grade (67.3%) than math in 8th grade (50.4%). However, when comparing scores on the National Assessment of Educational Progress (NAEP), the inverse is true – Washington actually has a greater percentage of students proficient in 8^th grade math than 4^th grade reading: 40% compared to 34%. WA also outpaces the nation by a wider margin in 8^th grade math as well (see chart below).

So, the state and national data seem to tell conflicting stories, both in terms of the relative ranking of 8th grade math versus 4th grade reading, and in terms of how many kids are actually achieving ‘proficiency’. And the disparity is not trivial. For example, national NAEP Reading data suggests about half as many kids are achieving “proficiency” (MSP: 67.3% vs. NAEP: 34%).

So what is going on here? What is our biggest ‘transition point’ problem – middle school math or early reading? How do the state and national definitions of ‘proficiency’ differ, and which is to be trusted?

As it turns out, there are a few thing going on. And Washington is not unique among states in this situation.

For one, Washington’s definition of ‘proficient’ is less rigorous than that utilized by NAEP, although our relative standing among the states is much different in 8^th grade math than 4^th grade reading.

Figure 3 below shows a ranking of states produced by the National Center for Educational Statistics (NCES), comparing WA’s standards to equivalent scale scores on NAEP. I have put stars next to Washington in each case, and you will see WA ranks 16^th in Reading, with a proficiency standard that is below NAEP ‘basic’, and far below NAEP ‘proficient’.

But in 8^th grade math, it’s a much different story (See Figure 3). Washington’s 8^th grade math standard is considerably more rigorous than NAEP ‘basic’ and comes closer to NAEP ‘proficient’. It’s also noteworthy that we ranked 2nd in the nation on this comparison.

From this, It would appear that our 8^th grade math standards are simply more rigorous than our 4^th grade reading standards. This would seem to explain why our relative math/reading proficiency rates flip-flop in the state-to national comparisons. So maybe our performance in Math, in objective terms, isn’t really all that far behind our performance in Reading. Indeed, maybe it isn’t behind at all.

CHART 1 – WASHINGTON COMPARED TO THE NATION ON 4^TH GRADE READING AND 8^TH GRADE MATH – NATIONAL ASSESSMENT OF EDUCATIONAL PROGRESS (NAEP)

Additionally, aside from WA’s ranking, it is striking how widely varying the definitions of ‘proficiency’ are state-to-state. A student could be deemed ‘proficient’ in Reading if they live in Portland, Oregon, but considerably below proficiency if he or she lived just across the border in Vancouver, WA (note that Oregon ranks next-to-last among the states on the Reading standard). That is, in essence, the strongest argument there is for Common Core.

CHART 2 – WASHINGTON COMPARED TO OTHER STATES – STATE STANDARDS EQUIVALENCE TO NAEP SCALE SCORE – 2 GRAPHS: 8^TH GRADE MATH AND 4^TH GRADE READING 2009 (SOURCE, NCES)

I’m left with a few lasting impressions. First, I have a renewed appreciation of the need for Common Core – such widely divergent definitions of ‘proficiency’ across the states present real equity issues for students.

CHART 3 – NAEP SCALE EQUIVALENTS OF STATE GRADE 8 MATHEMATICS STANDARDS FOR PROFICIENT PERFORMANCE, BY STATE: 2009

But I also have a real curiosity as to whether our first test results from the Common Core assessments reveal the same relative dearth of proficiency in 8^th math vs. 4^th reading. The low math scores, relative to reading, have garnered a lot of attention, and driven a lot of policy decisions in this state over the past decade. But could it be that the new test results, scheduled for 2014-15, turn the conventional wisdom about math performance in this state on its head?