Typical Crap

A Trend So New It's Old By Jack Shafer debunks the LOUISE STORY story:

Story says she conducted the survey from November 2004 to January 2005 while at Columbia. Earlier this week, the editors of Gelf Magazine posted a list of questions from the survey, and remarked how some of them were "loaded," that is, likely to produce a biased result. For instance, the first question—"When you have children, do you plan to stay at home with them or do you plan to continue working? Why?"—assumes that the respondents intend to have children.

Story says she revised the survey after its weaknesses were pointed out to her. But in her final tabulation, she retained the responses from about 30 women who answered the first survey because, she said, the responses didn't vary significantly from those given to the revised version. She concedes the survey wasn't conducted with social-science rigor but calls it "a very good journalistic questionnaire."

The problem with Story's e-mail survey is not that she asked a lot of students questions. Reporters are supposed to ask lots of people questions. But if a journalist wants readers to be impressed by numbers like "roughly 60 percent," they must 1) say who collected the numbers and 2) explain how the numbers were collected. The Times and Story failed the reader by not stating that these findings were about as anecdotal and impressionistic as, say, the findings of a columnist like David S. Broder based on 100 interviews he conducted in Iowa to take the state's political temperature. Broder would never write, "roughly 60 percent of 100 Iowans interviewed believe the president is doing a bad job," because a hard number indicates that such numerical findings have real significance (especially when their source is not divulged). Instead, Broder would typically place the results in their proper anecdotal context, by using a phrase like "most of the Iowans I met with. ..." That would dispel the misapprehension that he thought his interviews had the same footing as as a Gallup poll. Story, on the other hand, presented her results to sound like good sociology.

Story has strong and adamant defenders. Master's adviser Sylvia Nasar, a former Times reporter, accuses me of "shooting first and asking questions later"....

Story wrote for the Times business section over the summer. Her Times colleague, the much decorated David Cay Johnston, is a major fan of her work. "Among the young journalists I have mentored over the past three decades, Louise Story is in a league by herself," he writes in an e-mail. She quickly grasps "subtle issues at the level of theory and principle whose significance and context she writes about in plain English."

Read on for further debunking. The Times calls this journalism? (And I fell for it! Typical me!)