(Tuesday) Mailbag: forum posters, churn, betas

Oh and in response to Kerri Knight:

I suppose the critical issue here is what kind of tool do you think a forum is?

…
That’ll probably tell you what kind of access and moderation policy to pursue.

This is a good point and I would suggest that this depends very much on what sort of game you are developing and what stage of development you are at?

Thank you for your interesting reply and thanks to all those who have taken the time to respond.

There are a lot of issues here – so many you probably could start another couple of separate topics!?

Am I frustrated by the response?
Yes and no.
Morgan Ramsey said:

Any market data as related to specific products is strongly protected by individual firms as trade secrets. The rationale is inherently and mostly competitive; however, the rationale can also be related to stocks and trader confidence.

Where stocks and shareholders are concerned there is a requirement for company accounts to be available. In that case it becomes impossible to hide real data on the number of subscribers – so I dispute that trader confidence is a motivation. Player confidence is another matter however?
But in any case, this only applies to games post retail release and does not explain the secrecy surrounding data related to Alpha, Beta and Open Testing. Shareholders also probably don’t care too much how many subscribers post on the forums unless there is an expense involved?
I do appreciate that some of this data is commercially sensitive but it actually suggests to me that the reluctance to release this data openly is more because it is “embarrassing” than commercial?
As Raph said:

Closed beta churn is a huge issue, and these days the pool of possible beta testers is full of “looky-loos,” people who just want to sample a game for free early. Many people underestimate the bugginess and general unplayability and lack of usability of an early beta or an alpha, and just bounce off.
Very few people ever submit bug reports, as a percentage of the testing pool. (And fewer yet of those are reproducible, comprehensive, and comprehensible). In general, I don’t think that we tend to make it easy enough, either.

This is where I believe the truth may lie. As I said in the original thread
http://www.burningsea.com/forums/showpost.php?p=361015&postcount=10

…more and more it looks to me that;
-companies are using Beta Testing to promote their games rather than test?
-Beta Testers do not test but regard the ‘Beta’ as a ‘free game’?

In fact, just last week the MMORPG.com newletter (#128, 19th May 2008) led with 2 of the 3 Feature Articles relating to Beta Signups. (1,000,000 for AoC and 100,000 for Stargate Worlds.)
http://mmorpg.com/gamelist.cfm?READ=10719&VIEW=news&GAME=191&bhcp=1
http://mmorpg.com/gamelist.cfm/game/242/view/news/read/10750
WAR actually had a visible counter and PotBS also made a big deal of offering 10,000 new beta positions too IIRC?

Why is this a concern?
My interest in this is as a player. I want to play good games.
I am a firm believer that the end consumer should be presented with a “polished” and “finished” product. Advertising your product is one thing but pre-release testing should be just that. I don’t care if you sign up 1,000,000 beta testers – I want to know that the “testers” you select are doing their best to see that I get that product and that the development companies are also addressing the issues those testers are reporting (Again, this is another issue – lately I have my doubts about this too.)

And so I am left with a conundrum. Development companies that are only too keen to release info on how many people have applied to become Beta Testers and release data on how many Beta Testing invites they are issuing – and yet when presented with the simple question of “what percentage actually ‘test’?” claim that the data is “sensitive”?

How would it be a “trade secret”? Unless, of course, the percentage is so low it shows that this sort of “Beta Testing” is ineffective except as a marketing tool?

That said, I have been an advocate of allowing members of the public to test games in this way. I do appreciate that games have become incredibly more complex with time and a modern MMORPG almost certainly requires a “Monte Carlo” style of testing at some point.

The issue appears to be that if the testing goes on for too long then the players will have played the game for free and move on when the game goes retail and paying subscription starts?
Or maybe Beta Testers need to be policed and ‘culled’ if they are not contributing reports on a regular basis?

The discussion on forums is a whole new issue which I also have an interest in for a number of reasons. But, way too much to discuss in one go (sadly).
@Dana “Gophur” Baldwin
Yes, I would be very interested in any data you care to release. Of course, WWIIoL is unusual in a few ways (I am an active subscriber BTW).
Which brings me to a few interesting points raised by other posters:

if your forum account is tied to your game account there’s an inherent decrease in moderation issues simply because people don’t want to be banned from the game;

“Any company that voids a service agreement with a customer (e.g., online game service subscription) due to the customer’s misuse of an extrinsic benefit of the agreement (e.g., online community access) needs to revise their policies.

Banning a Forum account is one thing – banning an entire account for forum conduct is entirely another. The two things should be kept separate – I would not be surprised to see legal challenges to this in the future.
Besides which, players should never feel they cannot express their views (even the negative ones) freely (if you want good games that is!). People really only get passionate when they actually care. Also, the Moderation has to be very ‘professional’ and even handed for this to work.
WWIIoL is actually a very good example of this. Players are free to even argue with the Devs provided it is done in a mature way. Players can be cruel at times, but most mature people on both sides realize that it is not personal.

And

I have to come down on the side of one Richard “Lowtax” Kyanka and say that a forum that’s pay-to-play is, from a business perspective, superior to one that’s completely free. Not because it generates revenue (though that’s certainly a good thing in business) but because the barrier to entry will make the forum a better place for those that do use it.

From a business and legal point of view taking money also elevates a forum from an extrinsic benefit to a primary service with all the added liabilities that implies.

In terms of Community Forums there is a lot more that could be discussed but I am interested to note that many of you appear to imply that the forums are completely separate from the game and result from the game.
I challenge both counts.
Forum Communities are often well established even before the game gets to retail and I contest that the atmosphere of the official forum community can and will ultimately reflect the attitude in game and vice versa.
There is a whole new discussion right there.

That will do for now – it’s late and I have Pirates to sink and Aircraft to shoot down…

Eolirin wrote:

but I don’t think it’s even remotely fair of you to simplify the reason down to simply being “people aren’t willing to work.”

I don’t care if I’m being “fair.” That’s the way the world works. Humans are naturally inclined to avoid complexity and thus avoid work. That’s why we have tools: to simplify complex tasks, sometimes to the point of automation. Avoiding work is a strong motivation, particularly in poorly managed organizations where everyone hates their jobs.

I’m not entirely sure that the tangible benefits are actually worth those costs depending on the particular game and the particular situation of the company making it.

You’re saying, “Marketing isn’t worth customer service costs.” That’s the same dumb attitude that accounting-driven organizations have towards marketing. “We need to cut costs. Let’s cut our capacity to generate revenue first!”

At least in terms of exclusive to subscriber forums, the official community forums are part of the service being provided, not something separate from it.

We just had a huge debate about this subject. Access to official community forums is an extrinsic benefit of the provider’s agreement to provide customers with an online entertainment service. The debate we had was about the importance of communicating the extrinsic value of such benefits to consumers to influence their decision making.

but more with the fact that it’s a completely thankless job that involves the people who interact with these groups being constantly misrepresented, insulted, and raged about.

If you need to be thanked for doing your job, join the military. You’ll even get yellow-ribbon bumper stickers.

Heavy application of the ban hammer does not tend to resolve forum flare ups, so increasing the amount of staff doesn’t necessarily allow things to be dealt with easier.

Again, you’re assuming that traditional resolutions are the only tools at the disposal of community management. There are many, many different ways to handle community problems. Sometimes the community software needs to be customized. Other times the policies and procedures within the community team need to adapt to ever-evolving people.

The longer a game remains in operation, and/or the larger it’s population becomes, the more hostile the members of it’s forums will tend to be.

Your assumptions still lead you to baseless conclusions. Guess what: as a product matures, so do the resources, which are increasingly allocated to other projects. As resources allocated to a product dwindle, so does the performance of the product’s managing team. I just love when people try to blame consumers for the failures of management. “Our customers are getting hostile because they’re assholes!” Whenever something goes awry in a community, management screwed up. Whenever management screws up, someone should step up to fix the resulting problems.

I’m reminded of what Guy Kawasaki said about marketing new products:

At the start of a revolution, at the start of your company, you may see people, who are not your intended customers, use your product in unintended ways. This makes many people freak out, “My God, the wrong people are buying our products in large quantities!” You know, “we need to hire new agency, we need to reposition our product, because they are not the customer!” For one thing, if you are an entrepreneur, the first lesson is: when this happens, take the money. Take the money.

Bad managers blame the wrong people. They should be blaming themselves.

So if it’s not a goal for there to be direct communication of some sort, well… I think that company really “needs to revise their policies.”

One of the primary responsibilities of community management is to serve as a liasion between the consumers and the company. A liasion is a middleman. When you go through a middleman to provide feedback to, say, developers, you’re not providing feedback directly to developers. You’re providing feedback to a filter who relays all, part, or none of your feedback to the right people. Moreover, anyone who’s not the CEO, the PR representative to the consumers, the community relations team, and the technical support team really shouldn’t even be posting in community forums until they’ve had public and media relations training.

And the last bit that you’re mentioning conflates Official Website with Official Forums.

Official forums are part of official websites. For the purpose of market research, when you ask consumers about official websites, their opinion is based on their experience with all or some elements of official websites, including official forums.

So there’s a greater tendency for the forums to tend toward hostile to begin with.

That’s just not true. When a community is managed properly and effectively, you don’t create aggregious problems. You don’t create hostile members. In the same way, when you run a board of directors properly and effectively, you don’t have dissent, ill will, and drops in productivity. Communities are not necessarily highly negative and hostile organizations, but if you approach them that way, you’ll make them that way. Genesis 1:27 as a management principle applies here.

Before I say anything else, I want to make it very very clear that I’m not against integration of community and marketing. I think it’s a wonderful idea. And I don’t think there are unsolvable problems stopping it either, but I don’t think it’s even remotely fair of you to simplify the reason down to simply being “people aren’t willing to work.” It’s not easy to do. And there’re a lot of issues that require a lot of resources to be dedicated to them before they can be resolved. Either in staff or in research and development. I’m not entirely sure that the tangible benefits are actually worth those costs depending on the particular game and the particular situation of the company making it. It’s great if you can do it, but not doing it doesn’t make you lazy.

Any company that voids a service agreement with a customer (e.g., online game service subscription) due to the customer’s misuse of an extrinsic benefit of the agreement (e.g., online community access) needs to revise their policies.

I would agree that being banned for an extrinsic activity would be a bad way to do business. But in the case of community forums it’s not extrinsic. At least in terms of exclusive to subscriber forums, the official community forums are part of the service being provided, not something separate from it. They’re there as a way of broadening the experience being offered by the game world and really shouldn’t be viewed as being a separate entity. As an analogy, a country club can have golf courses and a restaurant. If you act out at the restaurant and get kicked out, you shouldn’t expect to be able to use the golf courses, even if golf was what you were there for.

You’re assuming that the customer service function is poorly managed. FYI: CS is Customer Service. CM is Community Management. CSR/TSR ≠ community relations.

You’re assuming that human resources are fixed.

Management and staff size has nothing to do with it. I’m assuming, and I’ve got far too much evidence behind me to not, that any official forum is going to eventually become laden with a huge amount of negativity, and that the mods are going to have to constantly be dealing with it. This has little to do with the number of people involved running the positions, or with how those positions are managed, but more with the fact that it’s a completely thankless job that involves the people who interact with these groups being constantly misrepresented, insulted, and raged about. Even increasing the numbers of mods does not resolve this problem, as there’s an extreme amount of diminishing returns there. Heavy application of the ban hammer does not tend to resolve forum flare ups, so increasing the amount of staff doesn’t necessarily allow things to be dealt with easier. And this also applies to devs who post and community managers who post even if they’re not involved in moderation, because the tenor of the boards directly impacts their ability to function on them. Remember, problems tend to increase exponentially with size, and not linearly.

You’re assuming that “increasingly hostile” is a condition of being for communities.

I’m not assuming anything with this one, though I’m not generalizing out to “communities” even remotely. When applied to specifically official game forums, increasingly hostile *is* a state of being, though. The longer a game remains in operation, and/or the larger it’s population becomes, the more hostile the members of it’s forums will tend to be. This tends to be progressive too. Hostility rarely decreases long term on the forums. Walk around in any official forum for a game that’s been around for a while and this becomes extremely obvious.

You’re assuming that there’s only a single channel for customer feedback.

You’re assuming that direct communication with developers is actually a goal.

I never said anything about forums being the ONLY form of feedback or communication, but it is *a* channel of feedback, and invariably it’s one of the largest. And it’s usually the only two-way one. Players of MMOGs tend to like to be able to hear back from the devs, and they tend to like to be able to at least have the illusion that there’s an ongoing conversation. Dialogue is important. So if it’s not a goal for there to be direct communication of some sort, well… I think that company really “needs to revise their policies.” Forums are a big big part of having that communication framework; increasing noise makes it much harder to manage.

And the last bit that you’re mentioning conflates Official Website with Official Forums. Regardless of the import of the official website as an information source (I’m not questioning that, I know it’s true), the official forums don’t necessarily have the same pull anyway, even if they are open and accessible. One is an information source, and the other is a community structure. Communities aren’t necessarily viral, they can be insular too. And game forums tend to be just as likely to be hostile (if not more) as they do to be helpful to players looking for genuine information.

I’m willing to bet that there wouldn’t be that much in the way of people swimming up stream from non-subscriber to subscriber just because of official community forums, at least without completely redefining what an official forum looks like. Again, forums do not exist as information sources. They’re feedback and community devices. Communities are great, but without some sort of “in” they’re not very helpful a device for someone who isn’t already part of them. This can be dealt with, yes, but it’s not so simple. Because remember, the bulk of people who are on your community forums are there because they’re they’ve got something to complain about or are having an issue with something. Most of the people who are absolutely happy and content with the game tend to spend more time actually *playing*. So there’s a greater tendency for the forums to tend toward hostile to begin with. And a hostile forum population doesn’t really help when it comes to adding members to the community.

Eolirin the Contrarian wrote:

At a point it becomes a cost/benefit study, and the costs balloon very quickly due to the difficulty in managing increasingly large populations.

Avoiding work and controlling costs are different activities. In this and similar cases, professionals have simply decided to avoid working!

if your forum account is tied to your game account there’s an inherent decrease in moderation issues simply because people don’t want to be banned from the game;

Any company that voids a service agreement with a customer (e.g., online game service subscription) due to the customer’s misuse of an extrinsic benefit of the agreement (e.g., online community access) needs to revise their policies.

Also, considering how horribly stressful the general CSR’s job is, I’m not sure I’d want to go trying to make it that much harder for them to begin with.

You’re assuming that the customer service function is poorly managed. FYI: CS is Customer Service. CM is Community Management. CSR/TSR ≠ community relations.

a lot less direct communication with the dev team as the CSRs would increasingly have to fall back to pure moderation instead of answering questions

You’re assuming that human resources are fixed.

and in an increasingly hostile environment they’d be vastly less able to maintain a healthy relationship with the playerbase.

You’re assuming that “increasingly hostile” is a condition of being for communities.

Noise increases too, making it that much harder to for the devs to get accurate feedback.

You’re assuming that there’s only a single channel for customer feedback.

Forums are there as a method of communication with the devs too, not just for forming a player community.

You’re assuming that direct communication with developers is actually a goal.

So I don’t think it’s as simple as them not being willing to use those venues as a method of acquiring new users as much as that doing so would be a rather large trade off, and not just in cost for paying CSRs, but also in long term retention.

I’m a member of the Community Managers Group. I’ve read and discussed community managers’ thoughts on integrating community relations with marketing, and in fact, I was scheduled to write a chapter on the subject for a book.

Most community professionals recognize the advantages of integration, they’re enthusiastic about integration, they want integration, but regardless, integration is difficult to make happen whether due to budgetary and cultural constraints or simply because some people want to avoid doing more work.

‘Sides, the net isn’t so monolithic that the official forums need to worry about that directly, acquisition via forums and other internet communities can occur via non-official venues.

According to the “Improving Game Marketing” study conducted by Frank N. Magid & Associates for the MI6 Game Marketing Conference,

– official websites are vital to providing information that consumers need to make purchase decisions;

– consumer opinion leaders are more likely to use official websites as sources of information;

– official websites are the most important source of information for self-buyers (as opposed to gifters);

– the official website tends to be more significant to PC game buyers;

– and a 1/4 of consumers don’t even notice advertising/marketing campaigns.

Both official and fan communities are of great importance to the success of interactive entertainment products and services.

I think one of the hurdles you’re facing in understanding why restricting access to an entire product community to only buyers is such an awful tactic is that you’re approaching the issue as though there aren’t real solutions as well as solutions to the problems doing so attempts to avoid.

For example, there don’t have to be extraneous techical support and quality assurance costs due to, say, consumers with pirated copies of games [which was a big issue for the now-defunct Ritual Entertainment] if access to technical support was limited to buyers/subscribers.

Nonsubscribers should be able to participate in discussions about the product — their feedback is just as important as feedback from subscribers! — and they should be able to participate in general discussions so that they can be pulled into the community and therefore influenced to subscribe. There are more reasons, but I think that’s enough vertical scrolling for now.

Morgan, perhaps… but do remember that the increase in moderation difficulties is not linear with population increases, it’s closer to exponential. At a point it becomes a cost/benefit study, and the costs balloon very quickly due to the difficulty in managing increasingly large populations. The benefits are not so tangible.

Tying forum access to gameplay accounts have other benefits too: if your forum account is tied to your game account there’s an inherent decrease in moderation issues simply because people don’t want to be banned from the game; there’s an actual consequence to their actions other than losing the 10 minutes it takes to roll up a new forum account. Also, considering how horribly stressful the general CSR’s job is, I’m not sure I’d want to go trying to make it that much harder for them to begin with. You’d result in a lot less direct communication with the dev team as the CSRs would increasingly have to fall back to pure moderation instead of answering questions, and in an increasingly hostile environment they’d be vastly less able to maintain a healthy relationship with the playerbase. Noise increases too, making it that much harder to for the devs to get accurate feedback. Forums are there as a method of communication with the devs too, not just for forming a player community. So I don’t think it’s as simple as them not being willing to use those venues as a method of acquiring new users as much as that doing so would be a rather large trade off, and not just in cost for paying CSRs, but also in long term retention.

‘Sides, the net isn’t so monolithic that the official forums need to worry about that directly, acquisition via forums and other internet communities can occur via non-official venues. Fansite listings are usually provided via official sites too, so it’s not like it’s hard to find a forum to ask questions on. Of course, if the devs wanted to have a direct hand in this, they could easily have an open sub-forum for non-account people. The question is whether or not that would really provide much more of a benefit than a fansite’s forum would.

I’ve been tracking this about once a year now for seven years on the MMO I produce (WWIIOL) and 10% is about the right number for posters with more than just a handful of posts. Those were generally support issues or one-offs that come from official discussion threads related to a production note or announcement.

10% is best applied to paying subscribers who post regularly. That comes across as the players who are actively engaging in the forum community outside the game. In our particular case some of these players are primarily tied to the forum community and in fact rarely play the game. They are particularly prevelant in the OT forums. I would think (any one have input) that anyone with a very long running community (15% of our subscribers are from year 1) would see something similar.

@morgan

We also bar non subscribers from posting on several of our forums (community discussions), restrict to read only on others (dev and announce) and give full access on a few (support). I don;t think this approach is unique to either subscription based or game related forums especially when those communities are hosted by the developer.

There are probably marketing reasons why this is “stupid” but there are several why it is important. Value add propositions come readily to mind and in particular for us the number of customers to our small company who pay to chat in OT. Many of these players are the same long term friends of the company who have been here for years and support us primarily because they like us and like what we do and how we do it. Hundreds of these players have joined our builder program by giving us up to a thousand dollars for a lifetime account. I believe we were the first to do that sort of program. It was a natural given our devoted following.

It is also a good idea to restrict passersby. Meaning that if I ban you from the forums and you can just continue to create accounts I have to increase my moderating budget. I am also then exposing potential customers to the rantings of people who have no interest in paying for my service. If I restrict where new accounts have access I can concentrate my message and moderation resources more on those places. Flying Labs, being free to play may alter these dynamics, but I doubt it.

Sorry for the hijack Raph. If Guy thinks that our data is useful I’lll be happy to discuss it with the team and see if we can easily put some numbers together for him.

(Tuesday) Mailbag: forum posters, churn, betas

Meta

Recent Comments

Categories

Tags

Recent Trackbacks

Archives

Twitter @raphkoster

Academics

Comics

Game devs

MMO devs

News/Forums

Other

Players

(Tuesday) Mailbag: forum posters, churn, betas

Share this post:

Meta

Recent Comments

Categories

Tags

Recent Trackbacks

Archives

Twitter @raphkoster

Academics

Comics

Game devs

MMO devs

News/Forums

Other

Players