Why I Remain Committed to Educational Measurement

Gene V. Glass's name appears dozens of times in my EndNotes library. I bought his latest book the day it came out and will slow my skim to a read when I see his name, or that of his colleague Audrey Amrein Beardsley in my RSS feed. Like many whose day job involves the messy world of assessment and curriculum, I respect his thinking and value his take on events. His recent decision to shift his title and focus out of educational measurement was likely not an easy one, but it was likely the inevitable next step in his journey.  He, more so than most, know the boondoogle that is trying to quantify learning and his voice against large-scale testing used for accountability purposes will hopefully be listened to by those who set policy.

His points, though, haven't influenced my commitment to educational measurement. When I begin working with a new group of teachers, I often share a part of my educational philosophy: I believe we can talk about, describe, evaluate, measure, and learn about student learning without using numbers. I believe assessment is at its best when it is indistinguishable from learning. 

This, I believe, is the next level of educational measurement and why I remain committed to the field. 

My EndNote entries that included Glass were from a time when I was studying classic test design, cut score setting, and tests designed to evaluate the system from 10,000 feet. When I got back to the 1-foot perspective and shifted to focus on assessments that are more learner-centered, dynamic, and useful to teachers and students, my citations switched to researchers like Linda Darling-Hammond, Grant Wiggins, Giselle Martin-Kniep, and others. It is my firm belief that through the efforts like New Hampshire's teacher designed accountability measures, NY's Performance Consortiuum, California's exit portfolios, we're learning from the authentic assessment experiences of the 90's and creating a new approach to measurement in which the walls between curriculum, assessment, and instruction are blurred or non-existent; that all that remains is the learning. Our job as curriculum writers, assessment designers, sages, guides, teachers, and/or facilitators to create the conditions for learning and then get out of the way. 

The video below was created to talk about measuring learning in moocs but I nearly stood up and applauded at the two minute mark when Gardner Campbell makes the important - albeit simple sounding point: Why is measuring learning so hard? It depends on what you mean by measuring.



Is Opting Out *REALLY* the only option?

If I can be so bold, may I beg your indulgence for committing the sin of telling, not showing as I begin this post? Unfortunately, I neglected to take a picture of the student’s work and will do my best to convey its awesomeness.

A few years ago, I was working with a group of middle-level teachers designing assessments to meet NYS’s teacher evaluation system. They had a fair amount of leeway around the assessments’ design but had a few non-negotiables:
·         There would be no new testing. Any tasks had to be embedded in the curriculum and be seen by students to be a writing task, not a writing test.
·         The task had to offer novelty to the students by asking them a compelling, interesting authentic question. This required an interdisciplinary approach.
·         The task had to align to the Common Core Learning Standards as well as the district goal of perseverance. So while we were focusing on the craft of writing, the task needed be an opportunity for students to work on a task until they were satisfied with it and see their growth.

The task itself was complex in its simplicity. Students were presented with the essential question, “Does setting influence character?” and two writing prompts. [During the initial discussion of the question, a Social Studies teacher pushed back, raising a concern that the question didn’t cross disciplines. That it was an ELA question and didn’t really fit into SS or into life outside of school. One teacher turned to him and with a lovely raised eyebrow asked, “Any concerns about your daughter going on Spring Break to Mexico with her friends?” The ensuing conversation really highlighted how powerful essential questions can be.]  In the beginning of the year, students were prompted to select any character from any text they wanted and place them in NYC on September 10, 2001 and then a second scene dated September 12, 2001. Students’ work was evaluated on their ability to construct a narrative (W.8.3) through dialogue (W.8.3b) and show a change in their character development based on the change in setting.

I’ll date this anecdote by sharing that in the first round of the task, we read a lot of scenes involving Edward, Bella, Jacob, and Ironman in NYC. Although the adults struggled with cognitive dissonance, the students created scenes that were complex, powerful, awkward, and lovely. They struggled with the idea of character beyond just the moveable widgets in their stories but as it was a key component of the curriculum, it was something that could be addressed during the year. After revising the student handouts, we worked to anchor the rubric, revised it as needed, and gave the students feedback using a developmental writing rubric. The teachers took the work and feedback back to their classrooms and used them as a part of regular instruction, giving the students the option of using it for a portfolio piece. [And by the way, we used the work to set SLO baseline scores and targets for teacher evaluation but that’s another post.]

Time passed and it was time to try out the post-assessment. By then, students had been studying American History for about six months. So, this time, they had to create two scenes, placing a character in Philadelphia on July 3, 1776 and July 5, 1776.  During the “try out” period in which the teacher-designers gave the task to one or two groups of students and brought their work back to the group for rubric anchoring, one of the participating teachers floated into the meeting room with a grin that nearly split her face.  “Wait. Until. You. See. These.” She laid out her student work and not unlike a group at a baby shower, we ooed and awed over the responses. One particular students’ work elicited the same response from every reader: Well… how about that. It captured the complexity of character, nuance around American independence, and that eagerly sought trait of voice. The student’s first scene began: I am not a patient man but I have no choice. The middle part of the scene involved the unnamed character doing a variety of chores and physical labor on a farm on the outside of the city and ended with the line: Only one thousand, nine hundred, and ninety-nine days until I can earn my freedom and be with my family. The second scene was virtually the same, there were a few small changes in details but the character’s routine was similar. The ending, though. Oh the ending. Only one thousand, nine hundred, and ninety-eight days until I earn my freedom and be with my family. What this student understood was that for the vast majority of people living in the country born between July 3 and July 5, 1776, little had changed. 

Our first task was to use that students’ scene to anchor the top level of the rubric (“The student’s narrative is unexpected, surprising, or otherwise reflects an approach to the prompt that does not follow a common structure.”) The teachers followed the same routine as before; the tasks and feedback went back to the students who revised them and considered them for their portfolios. In addition, this time, they compared their baseline writing to their newer pieces and reflected on their growth as a storyteller. About a month later, the students took the 2013 state assessments. It’s my understanding that the teachers are still doing these kinds of tasks and their students are taking the state assessments. Their “Opt Out” numbers didn’t make the news.

In response to a flurry of recent editorials about the Opt Out movement, Leonie Haimson and Jeanette Deutermann make the claim that “opting out is the only option.” I recognize that my own confirmation bias flared up when I read their first sentence using two of my trigger words: exam and failed. Setting that aside, I’m struggling with their claim. Should the students from the anecdote above have opted out? Did their parents do the wrong thing by not forcing them to opt out? I’m especially struggling with their last line: The Opt Outs will continue until real teaching and learning return to our classrooms. 

 There’s a part of me that takes umbrage on behalf of teachers who do incredible work to ensure that state tests are only a pause in that real learning. That don’t devote weeks to test prep and support students through challenging conversations. I also want to speak up on behalf of teachers who do devote weeks to test prep because the pressure they feel is so great, they feel they have no other option.

My counterclaim: Opting out is not the only option. What about the option of ensuring that the other 177 days of the year are full of amazing, incredible, powerful learning moments? What about the option of having open conversations about what test prep can be instead of buying yet another test prep book? What about using the science of our profession – as troubled as it may be – to push back against what are perceived as poor quality items? 

The thought that keeps rolling around in my head: Opt Out won. The mouse roared and the elephant blinked. Now what? 

Gee willikers, Mister! I never thought of that!

It's a common theme among those who are not fans of Common Core to use sarcasm and derision when talking about teachers who attribute a change in their practice to those same standards. (I weighed the pros and cons of linking to particular writers who practice this habit and decided no. It's about patterns, not personalities.) The refrain is usually:

Educator: As a result of Common Core, I started doing this.
Response: Scoff. You needed Common Core to do *that*? Why weren't you doing it before?

A recent article in the NYTimes about changes in the English classroom has received the same response. Only this time, the subject of the refrain is the reporter. Several teacher-bloggers took to their pages to share their disdain at the reporter's lack of understanding around English classrooms pre-CCSS and their opinion on her seemingly uninformed understanding of the travesty that the CCSS are.

It's a provocative writing device as it attempts to establish the author of the response as a voice of authority. A sort of a: *I* knew about this before so my opinion on the matter is more righter than yours. The drawback is the attempt to shame the educator or reporter who spoke about the change in practice. And shame is the really the best adjective as many of these posts imply that the only right thing that teacher can do is hide her head, apologize for not knowing about *that* and rescind all support for the CCSS.

When I read posts with that tone, I'm reminded of Ignaz Semmelweis. Well, not him personally, I had to Google his name, but of his work. Prior to Mr. Semmelweis, midwives and doctors would rotate between deliveries without washing their hands. Mothers were dying at high rates and it was seen as just one of the consequences of giving birth. So these medical professionals were doing the best they could with the information they had and along comes Ignaz and they discovered they needed to change something in their practice. They weren't bad before, they weren't uninformed. In fact, they likely thought they were doing everything in their power to keep the mothers alive.

There are two connections I see to the education profession. 

First, why the disdain? Why the condemnation of teachers who come forward to share how they've reflected and evolved? Going after individuals who share stories of how their practice has changed is a bit like claiming you've been washing your hands all along. It's possible some doctors were. It's possible some had Clean Room Level 4 birthing suites. At the same time, when the profession realized a change was needed, the doctors who didn't know about hand washing weren't bad doctors the day before the "wash your hands because ew..." staff meeting and then good doctors once they started washing their hands. One would think we'd want to elevate the voices of teachers who share their thinking, not shame them back into silence. More to point, it's possible some doctors never lost a mother or a child. To their thinking, they didn't need to wash their hands as their practices were just fine, thank you very much. To those teacher bloggers, good for you but why not dial up the pride in reflective practitioners and dial back the disdain?

The second, more important, issue is race and social justice in education. Mad props to #educolor for their work on the matter and for offering a place for educators to listen and learn. The conversations about race and social justice aren't always smooth. They're not easy and people are going to get it wrong. 80%-ish of educators are white women, many of whom were good students praised for being well-behaved and doing the "right" thing in school and speaking up takes an incredible amount of courage. To reveal a past mistake in pubic? Not easy for anyone - especially teachers that are in predominately white schools where students are most often introduced to Black Americans in the month of February or via the Social Studies curriculum and slavery. When teachers are getting scoffed and derided for sharing their thinking around pedagogy, what's the benefit to sharing their thinking about the really hard and important stuff?

Teachers are increasingly asking. "Here's where I'm struggling. Where am I stuck?" or sharing, "Here's what I'm trying now. It was hard to get here but I'm glad now that I'm here." It baffles me that anyone would respond in any way other than, "Thanks for sharing. That must have been hard. Can you tell me more about that?"This isn't about protecting delicate white lady feelings or otherwise suggesting teachers can't handle critiques. (I'll refrain from commenting on the patterns when some bloggers are offered critiques via Twitter.) It's about saying, "Dude. Don't be such a jackass." Or fine. Be a jackass but take a moment and consider what happens when that teacher Googles her name.