ACT: What it does (and doesn't) show

Tennessee's ACT average was released a few days ago. From Lisa Fingeroot at The Tennessean: 

Just 16 percent of Tennessee’s 2012 graduating seniors were fully prepared for college, according to a report released today by ACT, the organization that sponsors the college admissions test of the same name.

Tennessee's average was an 19.7, which was improvement from last year's 19.5. (Metro Nashville Public Schools averaged an 18.4, up for last year's 18.1) We also beat Mississippi, thereby causing a passel of Tennessee public officials to once again say, "Thank God for Mississippi."

(Is this a good time to mention I'm a product of Mississippi public education, kindergarten to college?)

A few thoughts on this:

• ACT is the most useful indicator we have for determining if schools, districts, and states are doing their jobs. A 21 average is considered the standard for telling if a student is likely to not only go to college, but graduate. The ACT isn't perfect, (see my analysis of the test below) but it's the best data we have.

• Of all the data we have on the achievement gap, the statistics below best illustrate how stark the gap is, as well as the importance of closing it. From the same article:

The 2012 scores “highlight the necessity for Tennessee to increase college readiness among certain racial minorities,” Huffman said. “Only 3 percent of black students and 9 percent of Hispanic students met college benchmarks in all four core subjects, compared to 18 percent of white students and 31 percent of Asian students.”

•  The ACT is more useful in evaluating an entire district's level of instruction, as opposed to just a high school. It tests skills that started forming years before a students becomes an 11th grader and can't easily be made up in a single year, or even several years. Where this comes in to sharp relief are the timed reading sections -- most high schools aren't set up to provide the remedial reading instruction to truly build a student's ability to blaze through the passages and have a decent understand as to what they read. 

• I've heard this qualification trotted out as an excuse in some quarters:

Tennessee is one of a handful of states where 100 percent of seniors took the ACT last year. The others are Mississippi, Colorado, Kentucky, Illinois, Louisiana, Michigan, North Dakota and Wyoming.

Making hay out of the fact that in some states, the testing population consists of more self-selected college-bound students, misses the point. First, Tennessee finished second-to-last on the "100-percent" list, too. Second, until we're averaging a 21, we can't say that our system is where it should be. Comparative rankings are interesting, but this is a test where absolute achievement matters. A 21 average is a strong indicator that a school system is doing what it's supposed to do -- prepare the median student for success at the next level. That's why MNPS -- and most other districts -- set a 21 average as the goal. 

• Breaking down the scores, we see some more red flags:

In English, 59 percent of graduating seniors were considered by ACT to be ready for college-level work — up one percentage point from the previous year.
However, the majority of students were not ready for college in the other three subjects tested by the ACT, even though scores improved slightly in each area.
In reading, 43 percent were considered ready for college; in math, 29 percent; and in science, 21 percent.

The number of students who are college-ready in math, especially given that the section is "highly predictive" of collegiate success, is particularly troubling.

****

The ACT has its flaws. (As does the SAT, but for the purposes of this post, I'm focusing on the ACT.) My issue with the ACT is that it's a timed test. Not only that, but it's a tightly timed test. A test-taker must answer at a brisk pace to have a shot at giving each question its due.  As someone who did well on the ACT, I never got any sort of advantage from my ability to read quickly in my college and working life. I get the purpose from a test-maker's point of view -- it's testing for a certain kind of intelligence and preparation. I just think those particular types of preparation and intelligence have limited applications beyond college-admission exams. 

There's some scholarly agreement on this, as a National Bureau of Economic Research study found that the Science and Reading subsections "have 'little or no' ability to help colleges predict whether applicants will succeed." The English and Math subsections, however, "are 'highly predictive' of college success."

ACT argues that the test has several purposes beyond predicting college success:

Jon Erickson, interim president of ACT's Education Division, made several points via an e-mail. He noted that the ACT is used "for multiple goals and purposes beyond just admissions or predicting overall student success." For example, it is used in course placement, and he said that the ACT has been "quite accurate" in that function

Not all high-stakes tests have a timed requirement, either. None of Tennessee's high school End of Course exams are timed. (Oddly, the elementary and middle school TCAP tests are timed. You tell me if you understand the logic behind the decision to time kids in middle school, but let them take a full school day, if they want, in high school.)

The thing is, I spend a lot of time teaching my students to slow down, to be patient* with a text. Every day I preach about the virtues of reading an article, story or a poem several times. "How many of you remember the name and face every person you meet the first time you see them?" I ask my ninth graders. No one raises their hands. "Exactly," I say. 

Being patient, learning to take time to think deeply about the meaning of something, is a skill that's useful in both college and real life. It's also a skill that is just as difficult to teach as is teaching a kid to read quickly and identify key parts of a passage. I've never understood why college-admissions tests value the latter at the expense of the former. 

All that said, the ACT is still useful for what it is -- a well-written test taken by tens of thousands of students. It's also among the most researched tests in the world, so even with its limitations and caveats, we get lots of useful data. Finally, until Common Core is fully implemented, it's also the closest thing we have to national standards. It's a worthy measuring stick for students and educators, though, as with all things used for evaluation, it shouldn't be the only tool used. 

As I've told hundreds of nervous high school students over the years, "You don't have to like the game, but you need to learn how to play it."

* Convincing 14-year-olds to be patient and thorough also provides an opportunity for me to teach the kiddos about the meaning of the allusion sisyphean.