Where does popularity come from, or the Wisdom of Crowds revisited
Maybe a better example would have been something like market success or how one can dispel things like “I hate this game and its basically broken and everyone who likes it is dumb”. Wisdom of the Crowds sort of says if millions of people like something, then maybe there’s something there to like.
— Darniaq, on his blog
I’ve been meaning to write broadly on the subject of “the future of content” for a while now. And a huge part of that topic is tied in with the question of “what is popularity, and what does it mean, anyway?” Darniaq’s throwaway comment, along with this post on the Long Tail blog gives me an excuse to dig a little bit at that.
Darniaq is referencing, in part, this earlier post of mine in which I discuss The Wisdom of Crowds
My immediate reaction is that the wisdom of crowds isn’t everything. Let’s look at examples…
Let us consider the case of Eleanor Hodgeman Porter. You probably don’t know who she was. She lived from 1868 to 1920, and wrote a lot of books. Books that cumulatively sold millions of copies, such as Miss Billy, her first bestseller. Her big hit, however, was Pollyanna. Published in a first edition in 1913, it had sold a million copies by 1919 or so, and it’s still in print today. It had its first stage production on Broadway in 1915, and it’s been filmed a few times, both as silent movies and as a Disney movie starring Hayley Mills.
As of right now, my book is outselling Pollyanna on Amazon, by quite a lot. In fact, my book actually has a higher average review. Now, this doesn’t mean much, to my mind. Is my book better than Pollyanna? Will it be read longer? Somehow, I doubt it. Pollyanna opened big, and AToF did not.
On the other hand, what has been the lasting contribution to culture from Pollyanna? Not to be too down on old Eleanor, but the major lasting effect of the book seems to have been the coinage of the word “pollyanna,” which is a derogatory term for people who do what the heroine of the book does: look optimistically at the world. What we have here is a classic whose central worldview is derided so much that it has entered the language meaning the opposite of the author’s intention!
What does the widsom of crowds tell us about Pollyanna? Is it possible that this enduring classic of children’s literature is in fact “broken” in Darniaq’s terms? Which aspect of the wisdom of crowds do we trust — the ongoing sales and impressive lifetime record the title has racked up, or the popular conception of the work that has such a firm hold on the collective cultural brain?
Of course, one might say that the teen years of last century were simply a sunnier time (at least, prior to the Great War), and we have grown more cynical since then. That the wisdom of crowds effectively changes over time. What does this mean for popularity? Consider the saga of poor Percy Marks’ book The Plastic Age, which isn’t even available on Amazon anymore. It was the #2 fiction bestseller of 1924 and was filmed in 1925.
Then, it basically vanished. In 1980 it reappears as part of a book series entitled “Lost American Fiction.” Here we have an author whose papers were considered important enough that they are held by Yale, who wrote 17 books and was published by Harper’s and The Saturday Evening Post. The Plastic Age was in fact the #5 most-purchased book of fiction by libraries in its year of publication.
But it was Britney Spears. The “college novel” was undergoing a boom, and Marks’ book was sexually explicit, for the day. None of the reviews cited literary quality. The book had no “legs.”
One of the interesting things about how popularity works is the “open big” phenomenon. It’s best illustrated by a curve that looks like this:
This curve is incredibly familiar these days: it’s the curve showing number of moviegoers over time to a film release, the number of games sold per day of release, the number of hits to a website after a major PR push, and so on and on and on. What this curve is showing is that in a hit-driven world, you get most of your folks checking you out on the first day, certainly in the first week. And after that, you slowly slide down the radar until you’re in niche-land. Your status as a “bestseller” essentially depends on how slowly the slide happens. What’s your “legs”?
If the curve looks familiar from other contexts, it’s probably because it looks like the Long Tail. And indeed, on the sidebar of that wonderful blog, we find an excerpt from this New York Times article saying,
In most cases, nearly half of a movie’s total audience turns out in the first week of release, which means there has been very little or no word of mouth motivating most of the audience. In other words, many people go to a movie without any real information about it – without even reading a review. Or, put most cynically: Most of the time, there is no relationship between how good a film is, and how many people turn out to see it.
What is the wisdom of crowds telling us then? That we’re suckers for a marketing campaign?
Ah, but it’s not that simple. If you look at books on word-of-mouth marketing or even books on network graph theory or books on the psychology of influence you’ll see just how vulnerable we are to what gets called everything from “preferential link attachment” to “being a lemming” — to wit, the fact that we tend to do what others do. Over and over again, market research has shown that marketing itself is really low on the list of reasons to buy something. Usually, the reason you buy something is because a friend tells you to because they are an active user of the product — and the reason they have for having bought it may be no better. This means that large products tend to get larger, as long as users don’t expire out quickly. Over time, this can lead to market stagnation, as one dominant product locks in users and acquires a monopoly. A first-mover advantage can be significant here.
The classic word-of-mouth adoption curve looks very different from the “open big” curve. A book that was the #1 bestseller for two solid years in the early 1970’s had this sort of curve: Jonathan Livingston Seagull. Richard Bach’s book is, in fact, outselling my book even to this day, even though it was rejected by over 20 publishers before seeing the light of day, was greeted by derisive reviews (Publisher’s Weekly commented that it was “a mite too icky poo”), and had basically no advertising budget.
We’re actually seeing a few games in the online space today that are following J-curves: Runescape, Second Life, Eve Online… games with legs, that didn’t open big, but are spreading virally.
What does all this add up to? Well, in my opinion, it adds up to the notion that just because millions of people like something doesn’t mean that it isn’t broken, or that people aren’t dumb for liking it. And conversely, there may be things that everyone agrees are great, but that don’t have popularity. In a hit-driven world, popularity is ephemeral and means very little. The real questions come over the long haul, and maybe it’s there that the wisdom of crowds can emerge. But on short spans of time, the wisdom of crowds is just as vulnerable to lemming-like behavior as anything else.
The trifecta is something that opens fairly well, builds via word of mouth, and retains interest over a long enough span of time that it “stays in print” and continues to draw an audience year after year after year. This aligns nicely with the ideal business growth of an MMO, I think.
This is running long enough, and I am way way late for dinner, so I’ll stop here. But at some point I want to cycle back and talk about whether the gradual demise of hit-driven culture means that the ephemerality of entertainment is going to get worse rather than better. In a world of niche-only entertainment, are cultural artifacts increasingly disposable? What does that mean for the content creators?
This post wouldn’t have been possible without this wonderful resource.
Hmm, graphs not showing. I’ll debug when I get home.
Raph,
It’s been ages, but could you come over and kick SWG’s team in the arse for messing up the game?
Peace
Inoga
Looks like IE (both Mac and PC) have trouble displaying the PNGs that I used for the graphs. They show up fine in Safari and Firefox.
So use Firefox. 🙂
if millions of people like something, then maybe there’s something there to like.
just because millions of people like something doesn’t mean that it isn’t broken, or that people aren’t dumb for liking it.
*Both* of these have been basic principles for me for a long time. Both are truly valid and important.
Someone in this field HAS the duty to accept reality and try to understand it. So, understanding why something “clicks” is essential. There are elements that are visceral and strongly rooted in out culture that cannot be minimized or dismissed.
But then it’s also a duty to use that knowledge sometimes to change the reality, not just understand it and cope with it.
And the “disposable worlds” is a concept I feel as well… I’m curious to read about that from you.
My experience of mmo launches says something like:
– The lowest Quality denominator of all components is what guides the crowd.
As with Britney Spears book perhaps the quality of the writing in itself was somewhat lacking?
Some games seem to do rather well with only a few components as long as those deliver the expected experience. Quite many appear to try deliver more components than they can produce.
I guess this reasoning is somewhat stolen from Bill Ropers KGC talk ^^
I wonder whether these issues have more to do with The Tipping Point than Wisdom of Crowds: the viral spreading of recommendations that produce J-Curves after release and the film industry’s feting of connectors and mavens to build hype before a film’s release.
Does the difference come down to a content versus network comparison? If your product provides a short term blast of content, you hype it to connectors and mavens before the release, use the hype to make sure you get mind share at release and then use network effects to spread the product to as many people as possible before your content is consumed, the short term wow factor fades and your product is replaced by the next hit.
If your product relies on network effects you play the long game: release early, attract early adopters, mavens and connectors after release and then work hard to cross the chasm and enter the mainstream relying on the fact that the early adopters won’t consume the content and leave, but will get more value through network effects.
As James Surowiecki says, The Wisdom of Crowds works best when large scale crowds choose independently and is broken or distorted by the sequential linked decisions that are described by The Tipping Point.
So, once the opening dynamics of a product’s release are over, The Wisdom of Crowds may work well for valuing 2 best selling classics, but I think The Tipping Point is a better lens for looking at the early dynamics of opening big or small and the curves that follow.
Jim, I referenced all the network effects via “Linked” rather than “The Tipping Point” in the post above — have you read it?
If anything, I think the J-curve of gradual choices may be a better demonstration of the wisdom of crowds than the big opening Hollywood movie, which may be made of independent choices, but frequently does stupid things. At least, the J-curve is evidence of a wise choice. 🙂
The first showing of “Gigli” supposedly opened pretty well — it was cell phone calls from disgusted moviegoers that made the later showings on opening day terrible, thus showing the quality of the film. By contrast, “Office Space” didn’t set any box office records, and sort of vanished quietly, only to achieve cult status and enormous DVD sales.
Network effects explain both of these vey well, better than wisdom of crowds does. Hollywood Stock Exchange’s forecasting of both would be interesting to look up, for historical comparison.
The point being that while we can generally understand why something “clicks” we have a lot more trouble understanding why something “clicks and sticks” or why something doesn’t click, then sticks anyway.
Interestingly, SWG’s forum membership was aon a J-curve, but the actual game was on an open-big trajectory.
Raph, forecasting with stock exchanges is a good example of when to use the wisdom of crowds as you’re asking for a valuation in return for a potential profit, rather than asking “Do I like this”.
I think Darniaq’s original question of how can things be popular when they’re so clearly broken is better explained by the long tail: popular things tend to have some appeal to a wide variety of people who may all love different niche things more.
Thanks for the “Linked” recommendation, if I ever finish reading Ted’s book and the rest of the Linden reading list that I bought when Cory was in London I’ll check it out! 😉
Darniaq’s question was actually “can we stop saying things are bad or broken when they’re clearly popular?” at least how I read it. And my answer was “no.”
I should post with my full reading list for network effect stuff… it’s all the bibliography for the Small Worlds talk though. I tend to assume everyone is up on all of that already.
Raph, yes I agree. Things can be bad, broken and popular. I think the long tail offers one explaination: most things with mass appeal are just not totally broken in the eyes of a lot of people. They can still be pretty bad or broken though and their popularity is no defence.
One of the things that Chris Anderson says about stuff at the head of the tail is that it’s marginally satisfying to a lot of folks, as opposed to stuff down the tail, which is deeply satisfying to a few folks.
I wonder if that’s a truism for all users, or only for the majority of them. Perhaps a corollary to that is that stuff at the head is always broken.
I wasn’t trying to excuse something bad as being ok because it’s popular. Rather, I was attempting to draw a parallel between the Crowd’s acceptance of an experience versus the Outsider’s point of view that it’s broken.
There are good designs, bad designs, well-developed games, and poorly-developed/ tested ones. However, what makes something “good” or “bad” is specific to their point of view. “Good” and “Bad” development, for example, are far more qualifiable because many of the practices of development deal with known factors. Conversely “Good” and “Bad” designs are more easily measured in some industries than others (for example, a well-designed road is going to allow for measurable performance while a well designed game has little in the way of common metrics with other games).
And, of course, Crowds don’t answer all 🙂 To me, the most critical part of using the Wisdom of the Crowds is understanding the push/pull mechanism needed to interact with them to get the results you needed. That’s the more fascinating part of the book to me, the tools described therein to ensure the Crowds can be the Crowds they need to be.
I think the term “satisficing” may apply to the discussion here; my understanding of it is the same as the issues raised by Anderson regarding satisfaction with products close to the head of the curve. People go for the product that is a) known to them and b) good enough. Then they stop searching, usually.
The Outsider in your example, may therefore be someone who has a) knowledge of more alternatives b) different standards for quality.
Of course quality is subjective, but there’s an assortment of metrics that we can probably try using. Among the ones that occur to me are:
I could go on… several of these can be argued to be “good” or “bad” design choices, and not just subject to the whims of marketing.
(quick thought on the images. I see you posted PNG format images. If you convert them to JPEGs, you might find they are supported better in other browsers. I can get IE to open the PNG files alone, it just doesn’t render them embedded like Firefox)
So, now, on with the show!
‘Satisficing’ is a pretty interesting concept, and yea, even in the most layman’s view on life, it plays out in the real world as well. My father’s a GM man. That’s just the way it is, regardless of quality and crash test ratings and MPG.
In games though, particularly intrinsically social ones like MMOGs, I’d think it’d be pretty hard to be attrached, and *retained*, simply because it’s known to them and good enough. Nothing is good enough in perpetuity, particularly when an enjoyment cycle is not at the pace of a game’s cycle. Further, the inputs of alternatives are so pervasive through the social network, even those not actively looking are aware of the alternatives, and their stated benefits. I imagine more players actively choose to stick to a game than avoid considering alternatives because they stopped caring.
Design is tricky 🙂 Most times it fulfills a directed needstate. The goals are measurable, the process to get their measurable, and the results measurable. However, there is no perfect design because the needstate isn’t defined by “when design is done”. It’s defined by the very many factors involved in getting that concept to the audience. And particularly in complex gaming, design and development run in tamdem. So the constant push and pull between goals and ideals always results in muddied end results difficult to assess. Unless a design is based on a completely known quantity, the development process is an *actual* core competency (not a self-convinced one), the end goal realistic, and the ongoing goals monitored, it gets very hard to connect the end “Quality” with the incremental decisions that got you there.