Saturday, November 14, 2009

Steven Pinker Reviews Malcolm Gladwell

One "all knowing" social scientist reviews another - partial truths all around. Pinke ris not kind his assessment - a smell a feud brewing.

WHAT THE DOG SAW
And Other Adventures
By Malcolm Gladwell
410 pp. Little, Brown & Company. $27.99

Malcolm Gladwell, Eclectic Detective

Todd Heisler/The New York Times -Malcolm Gladwell

Published: November 7, 2009

Have you ever wondered why there are so many kinds of mustard but only one kind of ketchup? Or what Cézanne did before painting his first significant works in his 50s? Have you hungered for the story behind the Veg-O-Matic, star of the frenetic late-night TV ads? Or wanted to know where Led Zeppelin got the riff in “Whole Lotta Love”?

Neither had I, until I began this collection by the indefatigably curious journalist Malcolm Gladwell. The familiar jacket design, with its tiny graphic on a spare background, reminds us that Gladwell has become a brand. He is the author of the mega-best sellers “The Tipping Point,” “Blink” and “Out­liers”; a popular speaker on the Dilbert circuit; and a prolific contributor to The New Yorker, where the 19 articles in “What the Dog Saw” were originally published. This volume includes prequels to those books and other examples of Gladwell’s stock in trade: counterintuitive findings from little-known experts.

A third of the essays are portraits of “minor geniuses” — impassioned oddballs loosely connected to cultural trends. We meet the feuding clan of speed-talking pitchmen who gave us the Pocket Fisherman, Hair in a Can, and other it-slices!-it-dices! contraptions. There is the woman who came up with the slogan “Does she or doesn’t she?” and made hair coloring (and, Gladwell suggests, self-invention) respectable to millions of American women. The investor Nassim Taleb explains how markets can be blindsided by improbable but consequential events. A gourmet ketchup entrepreneur provides Gladwell the opportunity to explain the psychology of taste and to recount the history of condiments.

Another third are on the hazards of statistical prediction, especially when it comes to spectacular failures like Enron, 9/11, the fatal flight of John F. Kennedy Jr., the explosion of the space shuttle Challenger, the persistence of homelessness and the unsuccessful targeting of Scud missile launchers during the Persian Gulf war of 1991. For each debacle, Gladwell tries to single out a fallacy of reasoning behind it, such as that more information is always better, that pictures offer certainty, that events are distributed in a bell curve around typical cases, that clues available in hindsight should have been obvious before the fact and that the risk of failure in a complex system can be reduced to zero.

The final third are also about augury, this time about individuals rather than events. Why, he asks, is it so hard to prognosticate the performance of artists, teachers, quarterbacks, executives, serial killers and breeds of dogs?

The themes of the collection are a good way to characterize Gladwell himself: a minor genius who unwittingly demonstrates the hazards of statistical reasoning and who occasionally blunders into spectacular failures.

Gladwell is a writer of many gifts. His nose for the untold back story will have readers repeatedly muttering, “Gee, that’s interesting!” He avoids shopworn topics, easy moralization and conventional wisdom, encouraging his readers to think again and think different. His prose is transparent, with lucid explanations and a sense that we are chatting with the experts ourselves. Some chapters are master­pieces in the art of the essay. I particularly liked “Something Borrowed,” a moving examination of the elusive line between artistic influence and plagiarism, and “Dangerous Minds,” a suspenseful tale of criminal profiling that shows how self-anointed experts can delude their clients and themselves with elastic predictions.

An eclectic essayist is necessarily a dilettante, which is not in itself a bad thing. But Gladwell frequently holds forth about statistics and psychology, and his lack of technical grounding in these subjects can be jarring. He provides misleading definitions of “homology,” “saggital plane” and “power law” and quotes an expert speaking about an “igon value” (that’s eigenvalue, a basic concept in linear algebra). In the spirit of Gladwell, who likes to give portentous names to his aperçus, I will call this the Igon Value Problem: when a writer’s education on a topic consists in interviewing an expert, he is apt to offer generalizations that are banal, obtuse or flat wrong.

The banalities come from a gimmick that can be called the Straw We. First Gladwell disarmingly includes himself and the reader in a dubious consensus — for example, that “we” believe that jailing an executive will end corporate malfeasance, or that geniuses are invariably self-made prodigies or that eliminating a risk can make a system 100 percent safe. He then knocks it down with an ambiguous observation, such as that “risks are not easily manageable, accidents are not easily preventable.” As a generic statement, this is true but trite: of course many things can go wrong in a complex system, and of course people sometimes trade off safety for cost and convenience (we don’t drive to work wearing crash helmets in Mack trucks at 10 miles per hour). But as a more substantive claim that accident investigations are meaningless “rituals of reassurance” with no effect on safety, or that people have a “fundamental tendency to compensate for lower risks in one area by taking greater risks in another,” it is demonstrably false.

The problem with Gladwell’s generalizations about prediction is that he never zeroes in on the essence of a statistical problem and instead overinterprets some of its trappings. For example, in many cases of uncertainty, a decision maker has to act on an observation that may be either a signal from a target or noise from a distractor (a blip on a screen may be a missile or static; a blob on an X-ray may be a tumor or a harmless thickening). Improving the ability of your detection technology to discriminate signals from noise is always a good thing, because it lowers the chance you’ll mistake a target for a distractor or vice versa. But given the technology you have, there is an optimal threshold for a decision, which depends on the relative costs of missing a target and issuing a false alarm. By failing to identify this trade-off, Gladwell bamboozles his readers with pseudoparadoxes about the limitations of pictures and the downside of precise information.

Another example of an inherent trade-off in decision-making is the one that pits the accuracy of predictive information against the cost and complexity of acquiring it. Gladwell notes that I.Q. scores, teaching certificates and performance in college athletics are imperfect predictors of professional success. This sets up a “we” who is “used to dealing with prediction problems by going back and looking for better predictors.” Instead, Gladwell argues, “teaching should be open to anyone with a pulse and a college degree — and teachers should be judged after they have started their jobs, not before.”

But this “solution” misses the whole point of assessment, which is not clairvoyance but cost-effectiveness. To hire teachers indiscriminately and judge them on the job is an example of “going back and looking for better predictors”: the first year of a career is being used to predict the remainder. It’s simply the predictor that’s most expensive (in dollars and poorly taught students) along the accuracy-­cost trade-off. Nor does the absurdity of this solution for professional athletics (should every college quarterback play in the N.F.L.?) give Gladwell doubts about his misleading analogy between hiring teachers (where the goal is to weed out the bottom 15 percent) and drafting quarterbacks (where the goal is to discover the sliver of a percentage point at the top).

The common thread in Gladwell’s writing is a kind of populism, which seeks to undermine the ideals of talent, intelligence and analytical prowess in favor of luck, opportunity, experience and intuition. For an apolitical writer like Gladwell, this has the advantage of appealing both to the Horatio Alger right and to the egalitarian left. Unfortunately he wildly overstates his empirical case. It is simply not true that a quarter­back’s rank in the draft is uncorrelated with his success in the pros, that cognitive skills don’t predict a teacher’s effectiveness, that intelligence scores are poorly related to job performance or (the major claim in “Outliers”) that above a minimum I.Q. of 120, higher intelligence does not bring greater intellectual achievements.

The reasoning in “Outliers,” which consists of cherry-picked anecdotes, post-hoc sophistry and false dichotomies, had me gnawing on my Kindle. Fortunately for “What the Dog Saw,” the essay format is a better showcase for Gladwell’s talents, because the constraints of length and editors yield a higher ratio of fact to fancy. Readers have much to learn from Gladwell the journalist and essayist. But when it comes to Gladwell the social scientist, they should watch out for those igon values.

1 comment:

m klein said...

Physics Professor Steve Hsu also lays into Gladwell for his claim about iq over 120 being meaningless:

"What Pinker refers to as the major claim of Outliers: IQ above 120 doesn't matter, is easily shown to be false. Randomly selected eminent scientists have IQs much higher than 120 and also much higher than the average science PhD (120-130); math ability within the top percentile measured in childhood is predictive of future success in science and engineering; advanced education and a challenging career do not enhance adult IQs relative to childhood IQ.

So, accomplished scientists tend to have high IQs, and their IQs were already high before they became scientists -- the causality is clear. 10,000 hours of practice may be necessary but is certainly not sufficient to become a world class expert.

I recently remarked to a friend that many aspects of psychometrics which were well established by the 1950s now seem to have been completely forgotten due to political correctness. This leads to the jarring observation that recent social science articles (the kind that Gladwell is likely to cover) are sometimes completely wrong headed (even, contradicted by existing data of which the authors are unaware) whereas many 50 year old articles are clearly reasoned and correct. The data I cite in the links above comes from the Roe study of eminent scientists and the Terman longitudinal study of gifted individuals, both of which were conducted long ago, and the SMPY longitudinal study of mathematically precocious youth, which is ongoing. I've interacted with many social scientists whose worldview is inconsistent with the established results of these studies, of which they are unaware."

http://www.technologyreview.com/blog/post.aspx?bid=354&bpid=24410