|May 27th, 2008|
Mr Koster, I am wondering if you can help me with a bit of online Mythbusting? I have an interest in statistics with relation to gamers and beta testers. This quote: “Something like 90% of the people playing an MMO never post in the forums.” was recently made here and attributed to you and Rich Vogel here. (please read the thread to see why I am interested) So, I was wondering if you could confirm this? How did you collect this data? And if so, what other data can you share? Do you know of any other sources of this sort of data? And, yes. I would be very happy to see this email posted and commented on in your blog. Thank you for your time and effort.
Regards, Guy Russon
Well, as far as how that stat comes about (and it does vary game to game — don’t take 10% as gospel, becaus eyou are right it’s a “whisper stat” at this point), you simply measure your subscribers, measure your active forum posters, and derive a ratio. In the case of forums where they require a game registration in order to register for the forum, this is pretty easy.
Like Joe in that thread, I can’t share specific stats for specific games, but I will mention that community managers who have experience across different sorts of virtual worlds (gamey worlds, worldy games, kids’ worlds, social worlds and so on) tell me that there are very different levels of engagement on forums. Worldy games apparently have a much higher forum activity level than gamey worlds, for example. I am told that kids’ worlds have a lot of challenging drama, from a community management perspective, and so on.
There are many stats that get measured around stuff like this — and to address some that come up in the thread you pointed to:
- Most users who quit do in fact disappear silently. The poster who bothers to say goodbye is far less common than the person who simply reduces playtime until they just don’t show up, and then lets their account lapse. The #1 predictor of a given user churning out is decreasing playtime.
- Closed beta churn is a huge issue, and these days the pool of possible beta testers is full of “looky-loos,” people who just want to sample a game for free early. Many people underestimate the bugginess and general unplayability and lack of usability of an early beta or an alpha, and just bounce off.
- Very few people ever submit bug reports, as a percentage of the testing pool. (And fewer yet of those are reproducible, comprehensive, and comprehensible). In general, I don’t think that we tend to make it easy enough, either.
You can dig through some of the presentations on this site — I may have posted some other stats before. And there’s certainly been many postmortems and conference talks about MMOs with random stats mentioned. But I think you may find it hard to come by hard stats for different games out there, because by and large a lot of this stuff isn’t shared publicly. Also, the game industry is not nearly as obsessive about metrics and analytics as the web industry is, so sometimes some of this stuff is not instrumented as well as it should be prior to the game’s testing period beginning.
It’s generally considered best practice to take forum feedback as only one channel of input in terms of community sentiment. Forum posters tend to be passionate, highly informed about the game and its details, connected in the community, and are often thought-leaders who have strong influence over the overall game community. But this can be a drawback as much as a benefit, and needs to be balanced with input from the “silent majority” that plays but is less plugged in, is less devoted and more casual, and who doesn’t even read the website.
Other channels include exit surveys, user polling (SOE has had great results with log-in polls), gameplay metrics data, and gathering info from targeted focus groups that are intentionally made more representative of the playerbase as a whole.