A lesson from Hollywood on how to run schools (No, really)

What can educators learns from Hollywood? Reading this interview with director Steven Soderburgh (via Andrew Sullivan), I picked up on some themes:

One thing I do know from making art is that ideology is the enemy of problem-solving. Nobody sits on a film set and says, “No, you can’t use green-screen VFX to solve that because I’m Catholic.” There’s no place for that, and that’s why I’ve stopped being embarrassed about being in the entertainment industry, because I’m surrounded by intelligent people who solve problems quickly and efficiently, primarily because issues of ideology don’t enter into the conversation.

Soderburgh is comparing filmmaking with Congress, but the same point can be made about how we run schools. Compare the ideology and dogmatism infusing many policy discussions versus the ruthless pragmatism exercised by the best teachers. 

I'm of the camp that thinks gifted educators are, at heart, artists who use technical skills and collaboration to create something that is, at best, only partially defined by numbers. Filmmaking is in some ways similar, which is why it's fascinating to read Soderbergh's take on the real assets of competent moviemakers:

And these people excel at problem-solving—that’s 99 percent of the job. I look at Hurricane Katrina, and I think if four days before landfall you gave a movie studio autonomy and a 100th of the billions the government spent on that disaster, and told them, “Lock this place down and get everyone taken care of,” we wouldn’t be using that disaster as an example of what not to do. A big movie involves clothing, feeding, and moving thousands of people around the world on a tight schedule. Problems are solved creatively and efficiently within a budget, or your ass is out of work. 

(Emphasis mine)

Soderbergh exaggerates in regards to prepping for Hurricane Katrina, but his overall point holds up: execution trumps everything. In both making movies and running schools, there's a tight schedule with lots of stuff happening at once. There's a budget. There's a deadline. 

Results are both measured with numbers and also judged as something better described than quantified. Filmmakers value box office, but know it shouldn't be the sole judge of a great movie. (The Shawshank Redemption is one example of this.) Educators know test scores matter and we also want students who will become thoughtful, informed, and kind citizens (note the work schools like KIPP and YES are doing with character education). 

I've worked in district and charter schools and have been asked what's the big difference? From my perspective as teacher, there is a difference, but it has little to do with different types of school governance. Rather, the difference is how diligent the adults are about problem solving. Some school environments facilitate solutions, others build barriers. The best charter and district schools find ways to create the former. Poorly run schools of both stripes do the latter.

This is why some charter schools succeed, some are mediocre, and others are worse than district schools. The governance model simply allows for more experimentation in solving the puzzle of how best to teach children. It doesn't guarantee anything about the abilities of the adults running the school.

All schools -- district and charter -- can learn something from successful filmmakers. Just like a movie, each school tells a story. The process of telling that story will involve numerous problems to be solved. The story will either be told well or poorly. The governance model of the school won't decide that. The adults who are solving the problems will.

Tests and dumb tests

"I am not opposed to all wars. I'm opposed to dumb wars."
-- Barack Obama, 2002

Borrowing Obama's formulation, I don't think most teachers oppose testing. I think many oppose dumb testing. 

Some teachers in Seattle are rebelling against testing -- sort of. They're boycotting giving a second test to their students. (The students in question already take an state-mandated end of course exam in some classes. As best I can tell, this isn't controversial.) Jesse Hagopian, a teacher at Garfield High School (and a Teach for America alumnus**) pinpoints the chief issue:

Students don’t take the MAP seriously because they know their scores don’t factor into their grades or graduation status. They approach it less seriously each time they take it, so their scores decline. Our district uses MAP scores in teacher evaluations, even though the MAP company recommends against using it to evaluate teacher effectiveness and it’s not mandated in our union contract.

That is dumb testing.

The more I read, this story seems less about whether testing is worthwhile and more of a tale of how an inane bureaucracy requires teachers to give a useless test. 

A letter from teachers at a Seattle elementary schools demonstrates why the test in questions is useless:

 ...the current version of the MAP test is aligned with the old state standards and it is clearly an unsuitable vehicle for evaluating students currently being taught the new required Common Core Standards. So not only are the results of little instructional value, but this discrepancy between what is taught and what is measured will yield falsely low scores making the MAP test invalid for the purpose of measuring student growth/teacher effectiveness.

(Emphasis mine.)

The test that's the focus of the boycott is the MAP (Measures of Academic Progress). It's a national test, though I don't know if the testmakers adjust the questions to suit the standards of different states. 

As it happens, I can somewhat empathize with the Seattle teachers. My ninth graders have a similar testing regimen -- a state-mandated end of course test (EOC) and the MAP. The reason I'm OK with the kids also taking the MAP is because the data from it helps me do a better job.

The MAP is something we chose to do as a school. We use it to track student progress throughout the year and make sure my students are on track to reach their desired ACT score. The MAP is taken three times a year on a computer and takes about an hour for most students to take a subject-specific test. Unlike the state-mandated tests, it offers a sophisticated breakdown of student growth and skill level.

It's very useful in terms of knowing where a student is, how quickly they're making progress, and tweaking instruction to meet students. As we switch to Common Core standards -- which are more rigorous -- the data from the MAP will be even more relevant. 

It is, of course, worthless if teachers don't use the data. 

The information I get from it actually saves quite a bit of class and planning time because I don't need to design my own diagnostic test, nor do I have to crunch the raw data. It's helped me teach because we're more accurately diagnosing skill deficits. It's one thing to know a student isn't reading on grade level. It's much more useful to know that, say, a student struggles with differentiating between certain vowel sounds. With that sort of precision, we're able to conduct targeted reading interventions. When students take the MAP again, we track progress. 

In other words, this isn't testing for testing's sake. We use this stuff to help our kids become better readers. It's a simple goal, but not an easy one. Helping kids make strides is complicated. That's why so many kids fall behind in the first place.

The idea of education is also simple, but not easy: everything a school does should help students learn. All tests, whether it's a ten-question vocabulary quiz or a state-mandated exam, are supposed to serve that goal. If the data from a test isn't meaningful, then what's the point in having students take it? 

As with so many things in education, poor execution sabotages a policy's intent. This seems like district leadership kept on adding testing systems without thinking through how it would play out in classrooms. The Seattle debate isn't about testing; it's about dumb testing.

****

** Read Hagopian's entire article. There's a persistent idea that TFA alumni don't have many different opinions about what ed reform should be. There's another one that many TFA alumni don't stay in the classroom. Both are false. 

The Mike Munchak model of school leadership

I've got a radical idea for improving schools. On paper, it won't cost more money. A bunch of schools are already doing it. Also, pretty much every other industry does this some form or fashion. So, as I said, radical by the standards of education policy. Here it is: District superintendents should allow principals to choose their staffs.

We in Nashville are watching an example of how this could work right now. After a crappy season, Titans coach Mike Munchak is "reorganizing" his staff. That is, he's firing some people and hiring others. Several coaches were given no more explanation than "the team is going in a different direction." Among these is longtime special teams coach Alan Lowry, best known as the architect of the most famous play in team history, the Music City Miracle. It's not like Lowry forgot how to coach football this year. But his contract had expired and observers speculate that the special teams unit needed shaking up.

This never gets old.

Tennessean sports columnist David Climer criticized Munchak's moves, but then wrote the following:

While some of Munchak’s decisions may seem odd, let’s be clear: Considering his situation, he must be given complete freedom to adjust his coaching staff in whatever way he sees fit. His job is on the line, so he must be allowed to win or lose with the assistants of his choice. If he thinks he can upgrade his staff, so be it.

(Emphasis mine)

So why does a football coach get the freedom to choose a staff, but many schools refuse to let their principals do the same? 

Back in my Philly days, I asked that question to a colleague. She looked at me as if I'd grown a third eye. "What would happen if a principal came in here and just cleaned house?" she asked, expecting that the answer was self-evident. 

Except that the particular school where I worked had been failing by every possible measure. We had the educational equivalent of a 2-14 season. If ever there was a school that was fit for a Indianapolis Colts-style makeover, this was it.

Of course, school district management being what it is, the district fired the previous administration while staff stayed (unless they decided to leave of their own volition).

This worked wonders in undermining most of the authority any administration would've had. Teachers could do the same thing they'd always done (admittedly a good thing in a few cases, but a recipe for repeating failure in most others). Whether a teacher was competent or not, none of us had reason to treat the new administration as anything other than the next group through the revolving door. I think I talked the principal five times during the entire year.

Now I work in a charter school. If she wants, the principal can fire me. Is that likely to happen? No. There are lots of good reasons why principals, even ones who were new to a school, wouldn't want to fire the staff en masse. The point is that a school leader should get to choose of who works for them. There's quite a difference between saying that something would be a bad idea -- like firing an entire staff -- versus prohibiting someone from doing it. That difference is trust. And if a superintendent doesn't trust a principal, then why is that person allowed to lead a school?

(I can see the eye-rolling from veteran teachers regarding that last point. Bear with me -- I'm talking about how things should be.)

I know that there are some terrible principals out there. (In my stint there, I learned the hard way that the School District of Philadelphia was notorious for hiring them.) The solution for hiring poor principals, though, isn't to make sure that they're powerless to change much of anything. That sort of arrangement can (and does) repel a whole lot of competent candidates. 

As for what to do with bad principals? Hold them as accountable as, say, the typical NFL coach. If the Titans don't go the playoffs next year, everybody expects owner Bud Adams to fire Munchak. The same idea should be applied to principals. If they don't get results, replace them.

Staff turnover is commonplace in other industries, particularly those were leadership is so closely aligned to performance. Indeed, school superintendents are held accountable for results even when they don't influence student achievement nearly as much as principals and teachers.

We understand the principle of giving freedom to lead in exchange for accountability when it applies to sports. Why doesn't it seem to apply to so many of our schools?

The value-add of value-added data

Value-added data isn't new to Tennessee, but it has only been recently added as a part of a teacher's evaluation. As promised, I'll continue taking big ed reform ideas and discussing how they affect my life as a teacher. 

Chris Low, a teacher from KIPP Philly, tells a story that illustrates why value-added data is so important. He writes about a challenge faced by all of us who teach children who enter our classrooms several years behind grade level (Thanks to Matt Rubinstein for sending this my way):

There is a delicate balance at play in many Philadelphia schools. At the heart of the issue is the idea of expectations. How can teachers keep the bar high without giving in to the constant disillusionment of failed endeavors and unrealized goals? This past summer, during my training as a teaching fellow, my eyes were opened to the limitless potential of students unhindered by the boundaries of low expectations. For many Philadelphia students, the biggest barrier to success lies in the belief that great accomplishments are beyond their grasp. The goal of the teacher is to be a figure that pushes students past the limits of what they think they can do. 

But it’s never that simple. 

(Emphasis mine)

When I was a reporter in Vicksburg, MS, I visited classrooms to get a sense of the day-to-day life of a teacher. The most common refrain from them was, "My kids come to me unprepared." I first heard this from a high school teacher, so I went to talk to a middle school teacher. She told me the same thing, so I talked to a 4th grade teacher.

I got all the way to a kindergarten teacher who said, "You wouldn't believe how far behind the kids are when the start school." I had never considered that a child could be behind before they even started school, but it's true. I thought about how my parents had taught me how to read to before I started formal schooling. My friends who teach elementary school now tell me that most of their kids come in not knowing how to read.

That needs to change. At the same time, we have to help the kids who are already in the system and below grade level. If kids are behind, they need schools and teachers that will help them catch up as much as humanly possible. 

The trick is, in this era of accountability in education, we also have to be fair to the teachers who teach those kids. 

This is why value-added scores are so important. Measuring the rate of progress a student makes in a year, no matter where they started from, is vital information. It's not the only way to measure achievement, to be sure, but it's an important piece of information. Kids should be recognized for making progress, not just hitting an absolute goal.  I remember asking a friend who does marathons how he keeps going when he hits mile 16, is tired and frustrated, and still has 10 more miles to go. "Every step forward is a positive one," he replied. Even if some of our students aren't as far along as others, we still must have a way of showing them that they're taking positive steps.

Low has similar thoughts:

When kids feel successful, they try harder. Like when I learned how to ride a bike. The bike I learned on happened to be a girl’s bike that was embarrassingly small. I felt ridiculous, but my dad celebrated my small achievement anyway. That feeling of success helped me find the courage to try on something a little more impressive, a bigger bike. My dad had lowered the handlebar, but as soon as I proved myself capable, he quietly slid that bar a little higher, and I was back to reaching again. I’m learning how to do that for my students.

Apart from being used to build a student's confidence, value-added data is critical information for teachers to have about their classes as a whole. We in Tennessee are lucky to have a value-added system (TVAAS) in place. It's doesn't offer a complete picture of a student, mainly because the tests themselves are blunt instruments, but it's miles ahead of what was available when I taught in Philadelphia's public schools. 

Among other things, I use value-added data to adjust my lessons to the needs of the class as a whole and target specific students' learning needs. It also saves me weeks of time in diagnosing students, as well as providing a baseline I can use to chart progress throughout the year.  

Still, though, when I attended the Tennessee Council of Teachers of English conference a few months ago, I heard skepticism about value-added scores being used in evaluations. My fellow English teachers wondered about the swings in data they saw from year to year. They were also concerned about over-emphasizing the results of a single test. 

I share some of their concerns and think that the system needs tweaking, but that doesn't mean it should be junked entirely, as I've heard advocated by critics.

In many education debates, we set up a false dichotomy between something being perfect or not having it at all. Conversations about education will be a lot more productive when we acknowledge that our systems are much like our students: not yet where we want them to be, but making progress towards getting there. 

The reality is that many students enter a given classroom below grade level. We can and should have many other discussions regarding how to change that. In the meantime, the idea is to accelerate a student's improvement. Value-added data helps us accurately measure that growth.

Assigned reading

Your assigned reading:

Andy Rotherham on "Why Chicago Matters"

 In practice the city gave a lot of ground on key issues to get kids back in school. Sure, in 2014 thirty percent of teachers evaluations will be based on how much students learn. But that’s state law in Illinois! It’s illegal for the contract to do less. Evaluation results will not be especially consequential anyway. Mediocre teachers can keep their jobs year after year and the great teachers in Chicago will not be protected during layoffs, which will still be determined largely based on seniority rather than effectiveness. It’s unclear meanwhile how the city is going to afford the 17 percent raise it committed to – especially at the same time Chicago’s teacher pension fund is nearing insolvency.  The city won on some issues, too, by protecting principal autonomy and maintaining a sensible policy on guaranteed jobs when there are layoffs because of the downsizing everyone can see coming. But, overall it’s hard to see the agreement as anything but a substantial victory for Lewis and one that will resonate far beyond Chicago.

Robert Pondiscio on "Hobson's Choice" 

  It would be ironic to be in the business of education and have little faith in parents’ ability to make an informed choice—or to correct course if that choice proved untenable.  My personal bottom line, speaking only for myself, is that choice is an intrinsic good.  I like exercising school choice for my child and I want you to have the same options.  And let’s face it, education is fundamentally coercive: you have to educate your child.  Some latitude in how you go about it is to be encouraged. 

Dana Goldstein on David Colman as "The Schoolmaster"

But what has proved most controversial is Coleman’s unilateral vision for American students, of college as the goal and a college-prep curriculum as the means. In public education, a new reform is always coming down the pike. Longtime educators develop a healthy cynicism about which grand policy ideas will trickle down to classrooms and which will sputter during implementation or simply go out of fashion. But David Coleman’s ideas are not just another wonkish trend. They have been adopted by almost every state, and over the next few years, they will substantively change what goes on in many American classrooms. Soon, as Coleman steps into his new position as the head of the College Board, they may also affect who applies to college and how applicants are evaluated. David Coleman’s ideas, for better or worse, are transforming American education as we know it. 

Ed Kindall & blaming low-income students

One more thing from that Tennessean article about the state's average ACT scores and the achievement gaps it revealed:

“The gap is there [between black and white students], and there’s never going to be improvement unless we look at the reasons,” said Ed Kindall, outgoing Metro Nashville Public Schools board member . 

Kindall thinks one issue could be motivation. “How do we motivate children to look at education as a way to get out of poverty?” he asked.

(The brackets are mine.)

I don't agree with Kindall on much, but I grant the following:

• He knows what life in segregated Nashville was like. I'm sure he has vivid memories of a time when black children were explicitly told what they could -- and more often, what they couldn't -- do.

• I'm sure he knows all too well what systemic racism has done to the education of generations of minority students.

• Kindall has a deep knowledge and experience of racism in a way that I will never know and cannot viscerally understand. That I am deeply empathetic and have dedicated my professional life to helping take down the barriers erected by decades of discrimination isn't enough; there's no substitute for experience.

So when he says things like this about the achievement gap shown on ACT average scores, I'm not so much angry as confused. Surely Kindall knows the students I teach. He visits their schools, talks to them at church, and sees them at the grocery store. Surely he knows that low-income students lack a lot of things, but a desire to get out of poverty isn't one of them. Surely he doesn't think the 2,880 students who graduated from MNPS in 2011 with ACT scores below 21 did so because of lack of motivation.

I've helped hundreds of low-income students in Nashville prepare to take the ACT. One thing I can say is that they don't lack the desire to do well. What most of the students lacked is the skill level -- not the raw intelligence or potential, mind you, but skill level -- necessary to get the score they wanted. This is the result of a public education system that has systemically failed to develop those skills.

 Contra Kindall, most students know exactly what their ticket out of poverty is. When I taught in an MNPS district high school, I remember counseling my 11th graders when their ACT scores came back. I watched more than few kids cry. Saying that our kids don't want to do well on the ACT is simply false.

And, for the moment, let's put aside the typical "blame the parents" riposte I see so often in these sorts of debates. If you'd like hold kids accountable for a choice they didn't make -- who their parents would be -- then you miss the entire point of public education. Also, you're reading the wrong blog.

Instead, let's focus the significant impact we can make. Namely, the quality of a school (and school district) matters. One proof point: a RAND study showed that Chicago's 5-12 charter schools (which are non-selective) "produce substantial positive effects on ACT scores."

If you're one of those people who reflexively dismiss data showing success in charter schools, here's a district-level example: the Broad Prize-winning Gwinnett County Schools in Georgia (which is very racially diverse, has a 50 percent free/reduced price lunch student population, and is almost twice the size of MNPS) was able to narrow achievement gaps and average a 22.3 on the ACT. The racial breakdowns, while still showing gaps, show much more progress than what we've seen in Nashville: black students - 19.2, Latino students - 20.7, white students - 24.0. While Georgia doesn't require all students to take the ACT, Gwinnett has high participation rates. Also, state-specific tests given to Gwinnett students have shown similar results in increasing achievement and closing gaps.

The point is that the quality of a school and school district can make a difference on college-readiness. What we control as voters, public servants, and educators can make a significant, positive impact on the prospects for low-income and minority students.

Kindall wants to look at the reasons for why there's an achievement gap and why low-income and minority students aren't scoring well on the ACT. But if most of our students do, in fact, have the motivation to do well and other schools and districts have shown success in getting poor and minority students to increase achievement and close gaps, then we must look at other reasons. Of course, all of those other reasons put the responsibility squarely on the adults who set up and run the public education system we have.

Ed Kindall knows exactly how a system can, for decades, deny poor and minority students the opportunities to achieve their potential. He also served on the Nashville school board for more than 20 years. Few people have had more influence on how the modern MNPS educates poor and minority students.

And at the end of his time on the board, he's blaming the kids for not being college-ready?

ACT: What it does (and doesn't) show

Tennessee's ACT average was released a few days ago. From Lisa Fingeroot at The Tennessean: 

Just 16 percent of Tennessee’s 2012 graduating seniors were fully prepared for college, according to a report released today by ACT, the organization that sponsors the college admissions test of the same name.

Tennessee's average was an 19.7, which was improvement from last year's 19.5. (Metro Nashville Public Schools averaged an 18.4, up for last year's 18.1) We also beat Mississippi, thereby causing a passel of Tennessee public officials to once again say, "Thank God for Mississippi."

(Is this a good time to mention I'm a product of Mississippi public education, kindergarten to college?)

A few thoughts on this:

• ACT is the most useful indicator we have for determining if schools, districts, and states are doing their jobs. A 21 average is considered the standard for telling if a student is likely to not only go to college, but graduate. The ACT isn't perfect, (see my analysis of the test below) but it's the best data we have.

• Of all the data we have on the achievement gap, the statistics below best illustrate how stark the gap is, as well as the importance of closing it. From the same article:

The 2012 scores “highlight the necessity for Tennessee to increase college readiness among certain racial minorities,” Huffman said. “Only 3 percent of black students and 9 percent of Hispanic students met college benchmarks in all four core subjects, compared to 18 percent of white students and 31 percent of Asian students.”

•  The ACT is more useful in evaluating an entire district's level of instruction, as opposed to just a high school. It tests skills that started forming years before a students becomes an 11th grader and can't easily be made up in a single year, or even several years. Where this comes in to sharp relief are the timed reading sections -- most high schools aren't set up to provide the remedial reading instruction to truly build a student's ability to blaze through the passages and have a decent understand as to what they read. 

• I've heard this qualification trotted out as an excuse in some quarters:

Tennessee is one of a handful of states where 100 percent of seniors took the ACT last year. The others are Mississippi, Colorado, Kentucky, Illinois, Louisiana, Michigan, North Dakota and Wyoming.

Making hay out of the fact that in some states, the testing population consists of more self-selected college-bound students, misses the point. First, Tennessee finished second-to-last on the "100-percent" list, too. Second, until we're averaging a 21, we can't say that our system is where it should be. Comparative rankings are interesting, but this is a test where absolute achievement matters. A 21 average is a strong indicator that a school system is doing what it's supposed to do -- prepare the median student for success at the next level. That's why MNPS -- and most other districts -- set a 21 average as the goal. 

• Breaking down the scores, we see some more red flags:

In English, 59 percent of graduating seniors were considered by ACT to be ready for college-level work — up one percentage point from the previous year.
However, the majority of students were not ready for college in the other three subjects tested by the ACT, even though scores improved slightly in each area.
In reading, 43 percent were considered ready for college; in math, 29 percent; and in science, 21 percent.

The number of students who are college-ready in math, especially given that the section is "highly predictive" of collegiate success, is particularly troubling.

****

The ACT has its flaws. (As does the SAT, but for the purposes of this post, I'm focusing on the ACT.) My issue with the ACT is that it's a timed test. Not only that, but it's a tightly timed test. A test-taker must answer at a brisk pace to have a shot at giving each question its due.  As someone who did well on the ACT, I never got any sort of advantage from my ability to read quickly in my college and working life. I get the purpose from a test-maker's point of view -- it's testing for a certain kind of intelligence and preparation. I just think those particular types of preparation and intelligence have limited applications beyond college-admission exams. 

There's some scholarly agreement on this, as a National Bureau of Economic Research study found that the Science and Reading subsections "have 'little or no' ability to help colleges predict whether applicants will succeed." The English and Math subsections, however, "are 'highly predictive' of college success."

ACT argues that the test has several purposes beyond predicting college success:

Jon Erickson, interim president of ACT's Education Division, made several points via an e-mail. He noted that the ACT is used "for multiple goals and purposes beyond just admissions or predicting overall student success." For example, it is used in course placement, and he said that the ACT has been "quite accurate" in that function

Not all high-stakes tests have a timed requirement, either. None of Tennessee's high school End of Course exams are timed. (Oddly, the elementary and middle school TCAP tests are timed. You tell me if you understand the logic behind the decision to time kids in middle school, but let them take a full school day, if they want, in high school.)

The thing is, I spend a lot of time teaching my students to slow down, to be patient* with a text. Every day I preach about the virtues of reading an article, story or a poem several times. "How many of you remember the name and face every person you meet the first time you see them?" I ask my ninth graders. No one raises their hands. "Exactly," I say. 

Being patient, learning to take time to think deeply about the meaning of something, is a skill that's useful in both college and real life. It's also a skill that is just as difficult to teach as is teaching a kid to read quickly and identify key parts of a passage. I've never understood why college-admissions tests value the latter at the expense of the former. 

All that said, the ACT is still useful for what it is -- a well-written test taken by tens of thousands of students. It's also among the most researched tests in the world, so even with its limitations and caveats, we get lots of useful data. Finally, until Common Core is fully implemented, it's also the closest thing we have to national standards. It's a worthy measuring stick for students and educators, though, as with all things used for evaluation, it shouldn't be the only tool used. 

As I've told hundreds of nervous high school students over the years, "You don't have to like the game, but you need to learn how to play it."

* Convincing 14-year-olds to be patient and thorough also provides an opportunity for me to teach the kiddos about the meaning of the allusion sisyphean.