|
|
Wikipedia and Wisdom of CrowdsJuly 12th, 2006 |
There’s some debate these days about Wikipedia and the wisdom of crowds. Then again, there always seems to be debate about it. The thing that I keep noticing, though, is people using a generic definition of “wisdom of crowds” that is quite different from my reading of the empirically verifiable parts of Surowiecki’s book.
Technically, Surowiecki’s conception of “wisdom of crowds” is ONLY applicable to quantifiable, objective data. The very loosey-goosey way of using it to discuss any sort of collective discussion and opinion generation is a misrepresentation of the actual (and very interesting) phenomenon.
You can summarize the core phenomenon as “given a large enough and varied population offering up their best estimates of quantity or probability, the average of all responses will be more accurate than any given individual response.”
But this is of very narrow application — the examples are of things like guessing weight, market predictions, oddsmaking, and so on. The output of each individual must be in a form that can be averaged mathematically. What’s more, you cannot use it in cases where one person’s well-expressed opinion can sway another, as that introduces a subsequent bias into everything (which is why the wisdom of crowds doesn’t always work for identifying the best product on the market, or the best art, or the like).
Using it for subjective things, such as opinions on politics, is a mistake for sure. And using it as a shorthand to describe the continuous editing and revision that appears on Wikipedia is also a mistake.
Wikipedia does not operate by wisdom of crowds. It operates by compromise and consensus, which is a very old mechanism (whereas the wisdom of crowds phenomenon is of relatively recent vintage).
The basic example of Wisdom of Crowds is “get 100 people to guess how many beans are in a jar. Nobody will be right, but the average of their guesses will be right.” It is important that the 100 people not be experts, specialists, or all completely ignorant either — you want a diverse group on every axis you can manage.
So, my prescription: to apply the wisdom of crowds to Wikipedia, ask 10000 assorted people to give a percentage estimate of the accuracy of Wikipedia. Then average the results. What you get back will be close to the actual accuracy percentage of Wikipedia. Then you can go argue about that to your heart’s content.

You can follow any responses to this entry through the RSS 2.0 feed. Responses are currently closed, but you can trackback from your own site. [?]Type in a relevant tag, and click the button, and help organize this blog's information.
[More Help]














[...] Comments [...]
[...] Raph writes: [...]
Raph Koster discusses Wikipedia and the wisdom of crowds: Wikipedia does not operate by wisdom of crowds. It operates by compromise and consensus, which is a very old mechanism (whereas the wisdom of crowds phenomenon is of relatively recent vintage).
[...] Recently, there has been some discussion about The Wisdom of Crowds and Wikipedia. There’s this article which is commented by a blog, which in turn is commented by another blog, and so on. People have interesting opinions on this phenomenon, but instead of jumping into the fire, I thought I’d offer a slightly amusing anecdote. [...]
[...] Koster extends this key point in a subsequent blog post: Technically, Surowiecki’s conception of “wisdom of crowds” is ONLY applicable to quantifiable, objective data. The very loosey-goosey way of using it to discuss any sort of collective discussion and opinion generation is a misrepresentation of the actual (and very interesting) phenomenon. [...]
sono condivise e corrette in un ambiente comune, fino ad arrivare ad un accordo finale raggiunto da tutti. La Wisdom of crowds si può ottenere solo nelle situazioni in cui i diversi input di informazione siano indipendenti. Ma ciò, critica Raph Koster, può avvenire solo quando si tratta di mettere assieme informazioni quantificabili: è infatti impossibile tenere isolate persone che danno giudizi (poiché sono influenzate da altre persone) e quindi far valere la Wisdom of crowds nei casi di dati
[...] http://www.raphkoster.com/?p=131 http://www.raphkoster.com/2006/07/12/wikipedia-and-wisdom-of-crowds/ http://www.raphkoster.com/?p=186 [...]
one collective verdict. And the people in the crowd need to be independent, so that they pay attention mostly to their own information, and not worrying about what everyone around them thinks.” Cass Sunstein, author of the excellent Infotopia andRaph Kosterqualifies the usefulness of the Wisdom of Crowds further: “Technically, Surowiecki’s conception of “wisdom of crowds” is ONLY applicable to quantifiable, objective data. The very loosey-goosey way of using it to discuss any sort of collective
s experiments etc. It sort of sounds like practical application of statistics and probability, but I wonder also is it just probabilities? While I listened, I also googled “wisdom of crowds critique” and this was myfavorite critiqueof what I skimmed…. Conclusions: interesting concept, not enough specific application to education to really hold our attention. But I realized that when I’m in groups that start heading towards group think, it