Saturday, October 07, 2006

James Surowiecki: The Wisdom of Crowds

I saw James Surowiecki give a talk at a prediction market conference before I read his book, The Wisdom of Crowds . The talk was interesting and he demonstrated a strong grasp of the market and an appreciation for its value. He engaged the audience in a discussion on how Prediction Markets could be made to work better. I was quite impressed, but I assumed it was mostly unrelated to the book. I was wrong.

Surowiecki has done a commendable job of presenting broad coverage of the many ways in which crowds can work together to produce results that exceed those of individuals or poorly coordinated groups. He also explains the strengths and limitations of the effect--when groups can be expected to outperform individuals, and when will they underperform.

The book starts with some simple examples: taking the average of many people's uninformed guesses can work very well when everyone has a reasonable idea of the plausible range of values under consideration. Crowds do quite well at guessing the weight of a person or the number of jelly beans in a bowl. They don't do as well at guessing the distance to the moon.

Surowiecki explores different institutions we use to make decisions and predictions looking for rules of thumb about what makes each work well and what problems each is suited for. His subjects range from markets and corporations to democracy, traffic and science.

It's hard to find this summed up in one place in the book, but Surowiecki's thesis is that there are institutions that can do a good job of getting good decisions and predictions from people even when none of the participants could do as well, and that the important factors are that the institution is appropriate for the task, and the crowd is diverse and independent. When institutions encourage or allow people to follow one another, the crowd's independence is undermined, and the outcome can be swayed by strong individuals. If the participants all come from the same background, or share a point of view (because of the way they were selected, or because the institutional setting channeled their views) then you don't get robust results.

Surowiecki comes across as a strong proponent of markets in the book, but he also describes situations that are more suited for other tools. He lauds open source software development, and also points out that decentralized coordination without prices can work well for managing traffic or leading scientists to expand the frontiers or our knowledge. An enjoyable read with much to recommend it.

No comments: