There’s some debate these days about Wikipedia and the wisdom of crowds. Then again, there always seems to be debate about it. The thing that I keep noticing, though, is people using a generic definition of “wisdom of crowds” that is quite different from my reading of the empirically verifiable parts of Surowiecki’s book.
Technically, Surowiecki’s conception of “wisdom of crowds” is ONLY applicable to quantifiable, objective data. The very loosey-goosey way of using it to discuss any sort of collective discussion and opinion generation is a misrepresentation of the actual (and very interesting) phenomenon.
You can summarize the core phenomenon as “given a large enough and varied population offering up their best estimates of quantity or probability, the average of all responses will be more accurate than any given individual response.”
But this is of very narrow application — the examples are of things like guessing weight, market predictions, oddsmaking, and so on. The output of each individual must be in a form that can be averaged mathematically. What’s more, you cannot use it in cases where one person’s well-expressed opinion can sway another, as that introduces a subsequent bias into everything (which is why the wisdom of crowds doesn’t always work for identifying the best product on the market, or the best art, or the like).
Using it for subjective things, such as opinions on politics, is a mistake for sure. And using it as a shorthand to describe the continuous editing and revision that appears on Wikipedia is also a mistake.
Wikipedia does not operate by wisdom of crowds. It operates by compromise and consensus, which is a very old mechanism (whereas the wisdom of crowds phenomenon is of relatively recent vintage).
The basic example of Wisdom of Crowds is “get 100 people to guess how many beans are in a jar. Nobody will be right, but the average of their guesses will be right.” It is important that the 100 people not be experts, specialists, or all completely ignorant either — you want a diverse group on every axis you can manage.
So, my prescription: to apply the wisdom of crowds to Wikipedia, ask 10000 assorted people to give a percentage estimate of the accuracy of Wikipedia. Then average the results. What you get back will be close to the actual accuracy percentage of Wikipedia. Then you can go argue about that to your heart’s content. 🙂