By Michael Miner
False Negatives
You had to wonder, reading Linda Lenz’s column in the June Catalyst, why no one had pointed this out before. Some truths are self-evident, and Lenz’s should have been one of them.
Lenz, the editor and publisher of Catalyst, was mulling the good news that reading test scores in grades three through nine of Chicago’s public schools had risen for the third year in a row. The announcement in May had made front-page headlines. “Daley raves about school test scores,” said the Sun-Times. “Chicago schools take turn for better,” announced the Tribune, whose editorial page hailed the “heartening trend.”
Lenz, whose magazine provides, by virtually everyone’s account, the most savvy coverage of the public schools, was more subdued. “In its test score press release,” she wrote, “the Reform Board credited programs and people. However, there was a glaring omission: Student retention, which inflates scores.”
Of course it does. When the slowest sixth-graders are held back instead of being promoted to seventh grade, that seventh grade will test higher simply because the slowest kids aren’t in it.
Lenz noted that the previous year the school board had held back 15,000 students in four grades–third, sixth, eighth, and ninth. As a result one grade–ninth–was twice affected: the slowest eighth-graders didn’t join it, and the slowest ninth-graders stayed around to take the tests a second time. Was it chance that, as Lenz observed, ninth grade “posted the largest gains of all”? Was it chance that the 5th and 11th grades, which weren’t affected by retention, posted the smallest, with 11th-grade scores actually dropping?
Lenz put the question: “How much of the test score gains reflect increased student achievement”–which Lenz doesn’t question and for which the Paul Vallas regime has been abundantly praised–“and how much reflect testing technicalities?” She said there’s no way to tell. “Even so, the board should have leveled with the public and explained that this year’s scores, to some extent, ran ahead of actual achievement. Simple honesty requires it.”
Surely there’s enough perspicacity at the daily papers for them to have produced this caveat on their own. The Sun-Times’s Rosalind Rossi, for one, gets the math. She’s written warily of test scores in the past, and last August, in a long two-part report on the retention debate, she quoted a Georgia educator who called retention “a test score shell game.” But Rossi wasn’t available to write up the school board’s jubilant announcement, and the reporters who got the assignment kept their story short and simple.
Standardized testing is a crude tool–along the lines of the hammer that makes every problem look like a nail. The board puts grade schools on probation on the basis of a single number: the percentage of each school’s students falling short of the national average in the reading comprehension portion of the Iowa Tests of Basic Skills. Probation means embarrassment, intervention, even the possible ouster of the principal and local school council. And a school is put on it if 85 percent of its students get a below-average Iowa grade in reading.
No one calls 85 percent a meaningless benchmark. As a minimum below which no excuse will do it may even be courageous. Even so, it’s blind to the environmental differences in schools and to the progress students might be making, even though they have an enormous way to go. “We get children with so many problems,” a reading specialist at a school in the Robert Taylor Homes that’s on probation told reporter Elizabeth Duffrin of Catalyst. “We’re more than teachers. We’re social workers, truant officers, nurses. None of that seems to be taken into account.”
In March the Consortium on Chicago School Research completed a three-year study by releasing a 50-page report, “Academic Productivity of Chicago Public Elementary Schools.” The consortium called the standardized tests, as used in Chicago, “crude and sometimes seriously biased indicators for making judgments about the productivity of individual schools.” For example, “significant improvements in the learning of very low achieving students…can go undetected.” A school whose students don’t test at the national norm goes on probation even if they end the year much closer to the norm than they began it. Consider a school full of immigrant children who can’t speak English. It’s not likely to produce high test scores, yet as the students master English, their parents often get their bearings too and move on to better neighborhoods. Therefore, the consortium observed, “The average attainment level…is not likely to get very high because teachers are constantly working with new students.”
In the short run, the consortium called for a more sophisticated interpretation of the data now available to the school board, so that schools could be rated by the “value added” by them to their students’ education during the year. In the long run, it wants a new and better way to assess Chicago schools.
The coverage the dailies gave this critique was superficial. Rossi, who might have rolled up her sleeves, was on vacation when the report came out. Janita Poe and Michael Martinez, writing in the Tribune, ignored the report’s analysis in favor of an incoherent account of “a series of behind-the-scenes political maneuvers at Mayor Richard Daley’s school board” over a supposed scheme to dump the Iowas. “That was a controversial story,” Martinez concedes, “and we did receive some criticism.”
“The worst coverage that we got on that particular story was from the Trib,” Anthony Bryk, director of the University of Chicago’s Center for School Improvement and of the consortium, told me by E-mail. “As for the Sun-Times, unfortunately Roz Rossi was out of Chicago….She has, however, paid attention to the reporting recommendations and subsequently did a FOI request from the Board so that she could approximate a value-added analysis on the progress of the Charter Schools.” Her two-part study of charter schools ran last month.
“Having been one [at the Sun-Times], I’m very sympathetic with daily newspaper reporters,” Linda Lenz told me. “You barely have time to get breaking news out. You can fault the resources editors put into the education beat as much as you fault any particular reporter. In the past, test scores didn’t always get great coverage. Why they need better coverage now is because this administration has put all of its emphasis in practically all areas on test scores. That change demands a better informed and more conscientious effort to report on them.”
In June the professional journal Education Week examined the press’s changing attitude toward standardized testing. It seems a lot of papers are getting smarter about it. For instance, “This spring, the Detroit Free Press announced that it would no longer rank schools and districts based simply on scores from statewide tests.” The Free Press made its decision after conducting a six-month computer analysis of data from the Michigan Educational Assessment Program. “It found that poverty and other factors outside a school’s control were so strongly linked to test scores that it made straight-up comparisons ‘inevitably flawed’ and ‘mostly meaningless.'”
The article, headlined “A New Accountability Player: The Local Newspaper,” cited papers in Seattle, Philadelphia, Charlotte, Los Angeles, Baltimore, and Omaha–but none in Chicago. That’s why it was reprinted last month in Substance, the renegade paper edited by Bowen High teacher George Schmidt. Substance wrote its own headline to make sure nobody missed the point: “Urban newspapers doing more complete analysis of school tests, data,” it announced, and the kicker asserted, “No other major city is as simple-minded as Chicago.”
Schmidt told me, “Every other newspaper in America is covering education, and we’re playing People magazine to Vallas’s public relations personality cult.”
What about Catalyst? I asked him. “They’re the closest to something I can like,” Schmidt answered. “And they’re evolving to be something critical. When they did an analysis of the reading scores, that’s the essence of one of the two biggest scams going on in terms of numbers.”
The other one, he said, is the budget.
And what about Rossi? “Generally she knows the beat,” Schmidt said. “She can cover it if they give her a long leash, but they won’t. She’s about all that’s left who’s any good, except me.”
The Spirit and the Letter
The published views of the Sun-Times’s editor in chief on the subject of minority hiring have been challenged by the newsroom union as not just regrettable but contrary to the union contract.
Last week I reported that the Sun-Times refused to put up a penny to help underwrite the national convention of the Asian American Journalists Association in Chicago. And there was no Sun-Times booth. Asked to explain by Voices, the convention newspaper, Nigel Wade reportedly replied, “Personally, I don’t believe in job booths. I think journalists should have the initiative to come and see us. My name is in the phone book. If they want a job they can call me up.” He went on to say, “Besides, we rarely have vacancies.”
Dan Lehmann, who chairs the Sun-Times unit of the Chicago Newspaper Guild, sat down at his keyboard. He started his letter to Wade by informing him that his views, if reported accurately, “appear to be in direct conflict with the language and spirit of the labor agreement” negotiated last autumn between the guild and the paper.
Lehmann reminded Wade of the article in the contract that states: “It shall continue to be the policy of the employer to actively recruit women and members of minority groups for all positions covered by this contract and to make a strong effort to achieve through its affirmative action program a work force reflecting the composition of minorities in the metropolitan area.”
Lehmann questions whether “actively recruit” comports with a policy of waiting until someone knocks on the door looking for a job that probably doesn’t exist.
The contract goes on to authorize a minority affairs committee of management and guild representatives that will meet every four months to discuss minority employment, training programs, and career opportunities. Lehman asked Wade for a “prompt meeting” of this committee–which hasn’t met in months and would have to be re-created, since only one of its guild members is still at the paper.
Lehmann handed Wade the letter last week. “He was very gracious. He read the letter and said he had nothing further to add. We should take it up with labor relations.”
News Bites
From the front pages of the Sun-Times:
July 20: “Flockhart: TV’s pretend ‘feminism'” (with photo of Calista Flockhart)
July 24: “Judging ‘Ally’s’ Emmys” (with photo of Calista Flockhart)
August 5: “Fashion: Wardrobes a la ‘Ally'” (with photo of Calista Flockhart)
If you’ve ever gone to a Bulls game, you know there’s not a minute when you aren’t being amused. When the game pauses the high jinks resume–the mascots and contests and LuvaBulls. The crowd can’t be trusted with stillness.
Same way with the president’s speech Monday night on TV. When he finished, the talking heads took over, flattening out the moment with the usual second-guessing, prognosticating, and searching for meaning. Perhaps to everyone’s relief, raw reality quickly got buried under comment. But when Clinton said goodnight it might have been interesting if television had then gone back to other programming, leaving American families in their living rooms alone with their thoughts.
During rush hour, the morning after Clinton came clean, I counted four people in my el car reading newspapers.
Art accompanying story in printed newspaper (not available in this archive): George Schmidt photo uncredited.