The value-add of value-added data

Value-added data isn't new to Tennessee, but it has only been recently added as a part of a teacher's evaluation. As promised, I'll continue taking big ed reform ideas and discussing how they affect my life as a teacher. 

Chris Low, a teacher from KIPP Philly, tells a story that illustrates why value-added data is so important. He writes about a challenge faced by all of us who teach children who enter our classrooms several years behind grade level (Thanks to Matt Rubinstein for sending this my way):

There is a delicate balance at play in many Philadelphia schools. At the heart of the issue is the idea of expectations. How can teachers keep the bar high without giving in to the constant disillusionment of failed endeavors and unrealized goals? This past summer, during my training as a teaching fellow, my eyes were opened to the limitless potential of students unhindered by the boundaries of low expectations. For many Philadelphia students, the biggest barrier to success lies in the belief that great accomplishments are beyond their grasp. The goal of the teacher is to be a figure that pushes students past the limits of what they think they can do. 

But it’s never that simple. 

(Emphasis mine)

When I was a reporter in Vicksburg, MS, I visited classrooms to get a sense of the day-to-day life of a teacher. The most common refrain from them was, "My kids come to me unprepared." I first heard this from a high school teacher, so I went to talk to a middle school teacher. She told me the same thing, so I talked to a 4th grade teacher.

I got all the way to a kindergarten teacher who said, "You wouldn't believe how far behind the kids are when the start school." I had never considered that a child could be behind before they even started school, but it's true. I thought about how my parents had taught me how to read to before I started formal schooling. My friends who teach elementary school now tell me that most of their kids come in not knowing how to read.

That needs to change. At the same time, we have to help the kids who are already in the system and below grade level. If kids are behind, they need schools and teachers that will help them catch up as much as humanly possible. 

The trick is, in this era of accountability in education, we also have to be fair to the teachers who teach those kids. 

This is why value-added scores are so important. Measuring the rate of progress a student makes in a year, no matter where they started from, is vital information. It's not the only way to measure achievement, to be sure, but it's an important piece of information. Kids should be recognized for making progress, not just hitting an absolute goal.  I remember asking a friend who does marathons how he keeps going when he hits mile 16, is tired and frustrated, and still has 10 more miles to go. "Every step forward is a positive one," he replied. Even if some of our students aren't as far along as others, we still must have a way of showing them that they're taking positive steps.

Low has similar thoughts:

When kids feel successful, they try harder. Like when I learned how to ride a bike. The bike I learned on happened to be a girl’s bike that was embarrassingly small. I felt ridiculous, but my dad celebrated my small achievement anyway. That feeling of success helped me find the courage to try on something a little more impressive, a bigger bike. My dad had lowered the handlebar, but as soon as I proved myself capable, he quietly slid that bar a little higher, and I was back to reaching again. I’m learning how to do that for my students.

Apart from being used to build a student's confidence, value-added data is critical information for teachers to have about their classes as a whole. We in Tennessee are lucky to have a value-added system (TVAAS) in place. It's doesn't offer a complete picture of a student, mainly because the tests themselves are blunt instruments, but it's miles ahead of what was available when I taught in Philadelphia's public schools. 

Among other things, I use value-added data to adjust my lessons to the needs of the class as a whole and target specific students' learning needs. It also saves me weeks of time in diagnosing students, as well as providing a baseline I can use to chart progress throughout the year.  

Still, though, when I attended the Tennessee Council of Teachers of English conference a few months ago, I heard skepticism about value-added scores being used in evaluations. My fellow English teachers wondered about the swings in data they saw from year to year. They were also concerned about over-emphasizing the results of a single test. 

I share some of their concerns and think that the system needs tweaking, but that doesn't mean it should be junked entirely, as I've heard advocated by critics.

In many education debates, we set up a false dichotomy between something being perfect or not having it at all. Conversations about education will be a lot more productive when we acknowledge that our systems are much like our students: not yet where we want them to be, but making progress towards getting there. 

The reality is that many students enter a given classroom below grade level. We can and should have many other discussions regarding how to change that. In the meantime, the idea is to accelerate a student's improvement. Value-added data helps us accurately measure that growth.

One reason Common Core is a good thing

A former student once told me that he'd lived in five states since kindergarten. While that's a high number, he wasn't the only student I've ever known who had done that.

I'll grant that while students who've lived in three or more states are unusual, students going back and forth between states isn't. At least, it's not so unusual that it wouldn't affect academic outcomes for a decent number of students.

It's hard to get data on this specific issue because data tracking students making intra-district moves in a school gets lumped into data tracking students moving state-to-state. Regardless, the general facts on student mobility are sobering. From a 1994 U.S. General Accounting Office study -- and it's a safe assumption that the mobility rate has only increased since then: (Hat tip Hoover Institution)

• About 17 percent of all third graders—more than a half million—have changed schools frequently.
• More than 24 percent of third graders have attended two schools since the first grade.
• Of third graders from low-income families (incomes below $10,000), 30 percent have changed schools frequently, compared with approximately 10 percent from families with incomes of $25,000–$49,000 and 8 percent of children in families with incomes of $50,000 or more
• About 25 percent of third graders in inner-city schools have changed schools frequently, compared with about 15 percent of third graders in rural or suburban schools.

And it damages achievement:

• Overall, third graders who have changed schools frequently are two and a half times as likely to repeat a grade as third graders who have never changed schools (20 versus 8 percent).
• For all income groups, children who have changed schools frequently are more likely to repeat a grade than children who have never changed schools.
• Children who changed schools four or more times by the eighth grade were at least four times more likely to drop out than those who remained in the same school; this is true even after taking into account the socioeconomic status of a child’s family.

Moving from school to school within a district is something that could be addressed by policy. Students moving from state to state is another challenge altogether. It seems to me that the best solution is to make the academic transition as smooth as possible. Hence the need for grade-level expectations to be as consistent as possible from district-to-district and state-to-state. 

Of course, not all agree. South Carolina Gov. Nikki Haley sees, in the words of Education Secretary Arne Duncan, "a conspiracy theory in search of a conspiracy."

From USA Today:

"Just as we should not relinquish control of education to the Federal government," [Haley] wrote in a letter to a state lawmaker, "neither should we cede it to the consensus of other states."

With an education system ranked dead last in achievement, God forbid that South Carolina learn anything from other states. 

And, of course, if Arne Duncan is in favor of something, then Diane Ravitch opposes it. From the same USA Today article:

Last month, New York University education historian Diane Ravitch, a vocal Duncan critic, blasted the standards, writing in The New York Review of Books that they've never been field-tested. "No one knows whether these standards are good or bad, whether they will improve academic achievement or widen the achievement gap," she said.

A question for Ravitch: How will decreasing the differences between academic expectations state-to-state widen the achievement gap? 

I could see a problem if Common Core standards were less rigorous than the typical set of state standards, but they don't seem to be. I'm spending the year moving from Tennessee English I standards to Common Core. The latter are tougher -- which is a good thing. I'm also familiar with English I standards in Pennsylvania, Mississippi, and Texas** -- Common Core seems to be a step up for all of them. 

That aside, I suspect that some of the real winners will be these students who have high mobility rates, particularly those who change districts.

After all, does anyone envision a future where families less frequently move from city to city?

**During the past few years, I've spent more time than I'd care to admit perusing different states' standardized tests. It's yet another one of my nerdy hobbies. 

Part 3: Ravitch vs. Rhee -- Who's the real idealist?

Here's the final installment of my critique of Diane Ravitch's CNN article on Michelle Rhee and ed reform in general. Part 1 is here. Part 2 is here. 

*****
Ravitch:

Rhee believes in merit pay but she is unaware that merit pay has been tried again and again for nearly a century. It has never worked. It failed recently in New York City, Chicago, and Nashville. In Nashville, teachers were offered a $15,000 bonus to raise test scores. It didn’t make a difference. 

Merit pay comes in a lot of shapes and sizes. In this particular interview, Rhee talked about differentiated merit pay: paying high-performing teachers what they're worth, much in the same what that the best players on pro sports teams command the highest salaries. She -- and I -- think it's absurd that we have a system that treats all teachers the same and rewards seniority rather than competence and willingness to teach the toughest classrooms and most difficult subjects. 

The merit pay studies Ravitch cites were "after the fact" -- like giving an NFL player a bonus for having an excellent season. The differentiated pay Rhee talks about are "before the fact" -- like when a proven star signs a high-dollar free agent contract. 

Merit pay fails, as does evaluation by test scores, because they both compel teachers to teach to the test and ignore whatever is not tested, like the arts and physical education. Such policies harm the quality of education. No elite school—not Andover or Exeter or Sidwell Friends—evaluates its teachers by the scores of their students on standardized tests. Nor do any of the high-performing nations.

Is Ravitch saying that because I teach English I, a tested subject, I'll somehow force my students to neglect art and P.E.? How on earth could I do that? Other teachers teach those subjects. 

Also, Andover, Exeter, and Sidwell Friends don't look at the standardized test scores of their students? Is she joking? What are the SAT and ACT but standardized tests? Those schools examine those scores with a magnifying glass (and promote their results). Their ability to produce graduates who score highly on standardized tests is what keeps those at the top of the heap. There's tremendous pressure on those schools and teachers to teach a rigorous curriculum that translates to high standardized test scores. Does Ravitch somehow believe a rich parent is going to drop $40 grand on tuition and not expect their child to score in the 99th percentile?  

Ravitch:

Merit pay fails because teachers are doing the best they can with or without a bonus. Merit pay destroys teamwork and collaboration in the school. Teachers work together; they are not in an individual sport, trying to be first.

It's absolutely true that teaching is a team sport. It's also true that when another teacher does well, I benefit. I'm not sure why Ravitch believes that because the algebra teacher down the hall gets high test scores, that would somehow destroy my ability to collaborate with her. On the contrary, when students are receiving excellent instruction in all of their other classes, it makes my job easier.  

Ravitch:

Rhee and the corporate reform movement rely on the outdated behaviorist theories of the early 20th century. Modern cognitive psychology recognizes that intrinsic rewards are far more powerful than extrinsic rewards. People do their best when motivated by idealism and by their freedom to exercise their professional judgment.

Again, I'm confused. Why is Ravitch treating intrinsic and extrinsic rewards as mutually exclusive? I teach because it's rewarding to work with kids. I also have to make a living. All things equal, I'd prefer to make more money than less.

Her embrace of idealism here is ironic because Ravitch is among the chief critics of "idealistic" Teach for America teachers like myself (Philly '05!). From her article, "How, and How Not, to Improve the the Schools," published in the New York Review of Books:

Each year,TFA selects several thousand idealistic young people, gives them five weeks of training, and sends them out to teach.

Ravitch seems to believe there are two types of idealism. Her idea of bad idealism seems to be when elite college students join the teaching profession instead of choosing easier and/or higher-paying jobs. Given what she wrote for CNN, I guess her definition of good idealism is when all teachers are treated the same, regardless of competence or track record.

Personally, I lose some idealism when people who've never worked as a classroom teacher -- like Ravitch -- claim to know how teachers work best together.

Back to Ravitch's CNN article: 

The best organizations today recognize the importance of building a strong culture in the workplace—not with carrots and sticks—but with respect and collaboration. Andrea Gabor, the Bloomberg Professor of Business Journalism at Baruch College in New York City, recently wrote on my blog: “As W. Edwards Deming, a leading management expert and critic of merit pay, once put it: ‘The only reason an organization has dead wood is that management either hired dead wood or it hired live wood and killed it. Merit pay, by dividing and demoralizing employees, is a good way to erode initiative and overall quality.’”

As I wrote above, I'm not sure how other teachers doing well enough to earn a bonus is going negatively affect my morale. This isn't Goldman Sachs; if a teacher earns a bonus because they've done a great job, good for them. 

Speaking of deadwood employees, here's what really divides and demoralizes a teaching staff -- seeing incompetence compensated as well as excellence. I remember my first year in the classroom. One of the worst teachers with whom I worked had a doctorate and earned twice as much as the best teacher I knew, who had just a bachelor's degree. Tolerating teachers who don't do their job is what kills a positive teaching culture, not people who legitimately earn bonuses.

Ravitch:  

Our teachers need our support. Let’s put an end to the war on teachers in general and on experienced teachers in particular. No profession can exist without experienced practitioners. Teachers need tenure so they have academic freedom to teach controversial issues.

I loathe this sort of overstatement. Did Michelle Rhee shoot a teacher? Has she tossed a grenade into a school? No? Then can we please put an end to attaching the phrase "war on..." to every political disagreement we have? 

Of course teachers need to be supported. For me, that means that I've got colleagues who are committed and work hard. It means a principal who makes sure that everyone is focused on the same goal. It means that we work together to make sure students are held to a consistent expectation every moment they are in the school building. It means schools are properly equipped so teachers can focus on delivering the best lessons we can. 

As for tenure protecting teachers' ability to address controversial issues...well, when I taught in a district school in Philly, academic freedom wasn't the worry; it was having the freedom to teach in a school that wasn't in utter chaos. Tenure isn't necessary to keeping me in the classroom; it's working in a functional school that makes me want to stay. The ironic thing is that the best schools in which I've worked had no tenured teachers. 

I'm not sure how strong of a correlation there is between not having tenure and achievement. (Nearly no public schools in my home state of Mississippi have tenure, yet they aren't exactly celebrated for excellence.) It is logical, though, to expect some people to not consistently give their best when they know it's practically impossible for them to lose their jobs. It reminds me of a euphemism for a teacher who runs a classroom that's out of control: a "consequence-free environment." 

Ravitch seems to think that's a good way to run a school, though.

Ravitch:

Parents must be involved in helping their kids succeed. Research is clear that what parents do matters even more than what teachers do.  Parents affect their children’s attitudes, behavior, and willingness to study and learn.

I totally agree with this -- the problem is that kids don't choose their parents and there's little teachers can do to help parents be better caregivers. (The parents who need the most help are also the ones least likely to ask for it.) 

What teachers can do is make sure that a child gets a great education. For me, that means involving parents whenever possible and making do when I can't. This is the reality of teaching children who live in poverty. Education, though, the only hope if the cycle of poverty is going to be broken. Therefore, schools who serve children in poverty must answer the challenge. We must be advocates for the children who have no others. 

Ravitch: 

Our students and families and communities need support too. If reformers really cared about children, they would build a health clinic in every school. That would do more to improve test scores than all the teacher evaluation schemes and merit pay plans that the reformers are now imposing, without a shred of evidence.

I'd love to have a health clinic in every school and wish we had universal health care in this country. However, I'm not a doctor or a nurse. Metro Nashville Public Schools has no say over the funding of health care in this country. All I can do is do the best job I can teaching the children in my classroom each day and working with my colleagues to run a safe, supportive, academically-focused school. 

Addressing a different point, Tennessee implemented a revised teacher evaluation system that included student achievement scores as a part of the overall evaluation. While no one thinks the system is perfect and doesn't need some fixes, student proficiency levels rose in 23 out of 24 categories. This would constitute "a shred of evidence" that having meaningful evaluations for teachers would, in fact, improve test scores.

Ravitch, in conclusion:

We can improve our schools. We can improve our society. We must work on both at the same time.

I agree. Rhee and I just think our schools will be the agent that changes our society. Given Ravitch's proposed solutions, she seems to believe the opposite. 

Ravitch criticizes people like Rhee and me as "idealistic", but I think our plan is more pragmatic than defending the status quo in schools while wishing the symptoms of poverty away. 

Part 2: Ravitch vs. Rhee -- To change the poverty rate, you're going to have to change education

Here's part 2 of breaking down Diane Ravitch's CNN article on Michelle Rhee. Part 1 is here. Part 3 will come tomorrow.

****

Ravitch:

Why are our international rankings low? Our test scores are dragged down by poverty. On the latest international test, called PISA, our schools with low poverty had scores higher than those of Japan, Finland, and other high-scoring nations. American schools in which as many as 25% of the students are poor had scores equivalent to the top-scoring nations.  As the poverty level in the school rises, the scores fall.

Rhee ignores the one statistic where the United States is number one. We have the highest child poverty rate of any advanced nation in the world. Nearly 25% of our children live in poverty.

This is a scandal. Family poverty is the most reliable predictor of low test scores. How can we compare ourselves to nations like Finland where less than 5% of the children live in poverty?

Rhee and her fellow reformers say that poverty is just an excuse, but it is not. 

Poverty is a harsh fact of life.

No doubt -- Poverty is a harsh fact of life. The poverty rate -- and the child poverty rate in particular -- in the U.S. is unconscionable and unacceptable.

OK, so what do we do about it?  

The problem is that Ravitch offers no solution to educating low-income children. Her ideas, stated further down in her article, deal with parents being better caregivers (outside of a teacher's sphere of influence) and opening a health clinic in each school (outside of a typical school budget's sphere of influence). Nothing about actually, you know, teaching them -- unless you believe that continuing to do what we've always done will result in change.

Decreasing poverty would help a lot; the problem is how best to do it. Declaring in an article that it would sure be great to have less poverty doesn't do much, though.  

Rhee (and I) believe public schools offer the most effective and sustainable solution to America's deplorable child poverty rate: closing the educational achievement gap. Rhee is quite specific: attract the most talented people in to education, pay the highest performers like they're game-changers, offer parents choice as to the best school for their children, and better use the resources we have to focus on results.  

Ravitch:

Children who are homeless, who have asthma, who have vision problems or hearing problems will have trouble concentrating on their studies. Children who have a toothache may not do well on testing day. Children who don’t see a doctor when they are sick will not be able to perform well on tests. Children who live in squalor will be distracted from their schoolwork.

Individually, these are true statements, but this misrepresents the reality of teaching low-income students. She makes it seem as if teachers like me are practicing our craft in a M*A*S*H unit. I have, indeed, taught students with health and/or home problems. I'm not minimizing their challenges or the unfairness of it all. 

However, most students who have health and/or home problems that prevent them from fully participating in class don't want their English teacher to make excuses for them -- they want me to teach! I never cease to be amazed at what kids are willing to do when they have a genuine learning opportunity.  

What I've learned is that most kids are quite resilient, especially since they know that an education is what's going to get them out of a troubled home. This is why educators simply must have the belief that all students can learn and achieve at high levels. If we don't believe that our students will be successful, who will?  

The teachers who have that belief and live it out can help students achieve significant academic gains. What Ravitch never explains is how the same student can score years below grade level for one year, then spend a year with an outstanding teacher and grow several years in skill level. The reason the student didn't score well one year isn't because he or she lived in squalor -- it's because he or she wasn't taught well.

I've spent seven years teaching low-income kids. The biggest influence on their ability to perform well on tests is the expectations placed on them. If they are treated and taught like they're capable students, they'll usually perform as such.

Ravitch: 

Of course, we should have great teachers in every classroom, but the negative rhetoric that now comes from Rhee and every media outlet and movies like “Waiting for ‘Superman’ ” are demoralizing teachers and causing many excellent teachers to leave the profession.

Ravitch has a misguided sense for when the phrase "negative rhetoric" is applicable. Rhee says over and over again in the CNN interview that excellent teachers should be paid much more and given the freedom to educate kids in the best way they know how. Among other things, Waiting for Superman celebrates many schools that are offering kids a great education. I'm just one public school teacher here, but those are things that make me want to stay in the profession.  

What Rhee and Waiting for Superman reserve their "negative rhetoric" for are public school systems that regularly deny opportunities for low-income kids to go to a great school. 

Ravitch, on the other hand, saves her "negative rhetoric" for those who are trying to do something to fundamentally change those failing systems. 

I'm curious as to what she would say to those students directly affected by the status quo in those same systems.

****

The final part will be up tomorrow.