On Trust, Part II

 Posted by (Visited 20395 times)  Game talk
Feb 062006

In part I I gave the basic grounding for my take on the issues of trust, reputation, and policing. Now I want to dig into some of the deeper issues there.

The relay problem
Thus far, I have referenced two different broad models of trust: the communitarian model and the appeal to authority. Looking broadly at the two models, what we see is this:

  • The communitarian model relies on mutual enforcement, on peer pressure. The expectation of future interactions causes individuals to make an essentially selfish gesture — that of trading goodwill now for future goodwill from others. You can graph this is a tight group as a highly connected network.
  • The appeal to authority model is based on mediated trust — parties both have to place their trust and thus their well-being, in the hands of an allegedly disinterested third party. Again, the motives for this are essentially self-serving; by having a fair system, in the long run the entire community benefits. You can graph this as a triad: two participants in a transaction, A and B, with actor C acting as a mediator and guarantor of trust.

There’s a couple of interesting points to make about the authority model, however. Consider an extended diagram which illustrates the nature of hierarchical authority. What we see here is that most authority structures are not flat. Above your closest guarantor is another guarantor, and so on up the chain. What’s more, your greatest influence in a peer-pressure sense is limited to the close guarantor. Since trust is defined by an expectation of repeated interaction, You can only develop true trust with authorities who are “local” to you. Above that, what you really get is what I would term faith.

This can be best illustrated by the ways in which governments tend to work. The government you can actually influence is your local one. You can vote for your alderman, city council member, school board member, and so on. You can more easily make use of your social network to influence opinions of this person. In a well-organized or small enough community, you may even (gasp!) know them. The same holds true for the individuals used by the community to police behaviors: policemen, judges, district attorneys, and so on. Your “vote” is commensurately larger as a proportion of that individual’s total reputation (which in a democratic system is formalized at elections, just as on the Internet it is formalized when you do a Google search and get back a PageRanked result).

However, any legal appeals process may wander through the courts up through the chain. Your “share” of control over your district judge is minimal in a peer pressure sense. Your “say” in any given election is lessened in a strict mathematical sense, given the fact that every vote counts once, and there’s simply more people involved. Influencing the choices of the federal government in the U.S. is a very difficult proposition for a typical individual.

Worse, in the real world, we don’t even have that kind of flat voting system. In practice, because of how society is structured, you start getting the notion of unequal influence; subnetworks that can more effectively get out the vote, or subnetworks that can supply more donations for the campaign fund, and so on. The phrase “influence peddling” carries a very literal meaning; what is being traded is effectively the weight of reputation, a bias in arbitration. And we must keep in mind that every act of funding by a politician is effectively an act of arbitration.

In the end, the authority model is still reliant on communitarian principles; a bad supplier loses whole markets, not individual customers. A politician that snubs a constituent is damaging his relationship with that constituent’s network. A game that snubs one GLBT guildmember is damaging their relationship with all GLBT guilds and their members. And so on.

The perspective on a given case of arbitration differs dramatically depending on whether you are within the network of one of the actors or not. This is what is at the root of the phenomenon of classism in politics; the idea that arbitrators will tend to decide in favor of the party who is like them, such as the one they see at the country club. Consider the diagram where B and C are both in the same peer group, whereas A is not. Clearly, B will have more influence over the actions of C than A will, because B and C can expect to have future interactions, whereas further interactions between A and C are less likely. The question is whether C can be impartial in this case; for this sort of situation we have the concept of recusal.

Alas, recusal generally requires that C have the integrity to step back themselves; verification of the propriety of recusal lies above C, with someone we can call C’. And again, it’s far more likely that all the C‘s will be hanging out together in the future, so they’re less likely to accuse each other of wrongdoing…

There are many variant graphs that can be constructed. Consider the “scalar issue” — there may be a relationship between A and the eventual arbitrator of a dispute, but it may be a closer relationship on B‘s part, diluting the influence that A has on arbitration.

Even more interesting is the question that arises when C truly is independent, but has completely separate agendas informing their decision. For example, credit card companies act as guarantors of identity when we buy things. They are also involved in selling identity. There’s a conflict right there. The many cases of interlocking corporate boards serve as another example.

The issues with authority
Here is where we must turn to the notion of betrayal. It doesn’t mean “he done me wrong.” Specifically, betrayal can be defined as a high trust relationship that is broken.. We tend to feel it keenly in the case of interpersonal relationships — the more reinforced the relationship has been, the worse we will feel the betrayal.

In the communitarian model, betrayal is easily resolved; we can exile the betraying member (in fact, almost by definition, the act of betrayal is the act of departing the relationship web). In the authority model, however, the trust relationship shifts to the relations between A and C. When B betrays A it isn’t just A who feels the sting; since C is actuing as guarantor, A loses confidence in them as well. In effect, the loss of confidence attacks the foundation of the government itself.

This is the subtle erosion seen when you notice that a game operator isn’t cleaning up the supposedly impermissible racism or homophobia on global chat. It’s the more blatant erosion seen when an arbitration decision is decided wrongly, or when a nerf is made that favors one class over another.

Given our instinctive understanding of human tribal hierarchies, we tend to ascribe many of these actions to bias, when it’s likely many of them are simply ignorance or ineptitude. A thresholding phenomenon probably contributes here; there’s a distance (probably in the more distant suburbs of the monkeysphere) beyond which all entities are “them” and it’s very easy for us to say that THEY are the betrayers. At that point, we start to claim vast conspiracies when in fact incompetence is a far more likely explanation under Occam’s Razor. This is an extremely common pattern observable in both real-life politics (some of the more fanciful allegations made by Michael Moore come to mind) and in virtual world administration (they hate all <<insert group names here>>).

Now, the nature of the Internet is fundamentally communitarian at its basic technical level. Much of the Internet’s basic architecture was designed with the specific intent of not having a centralized authority (although of course there are some that have coalesced, such as the guarantors of identity that verify the validity of domain names). Resorting to an appeal to authority tends to fail when there is no authority.

This places a particular burden on all the regulatory initiatives that seek to tame the Net. You can only regulate a communitarian model by destroying it and converting it to an authority model. And the more distant the authority is, the higher its burden of proof, because humans distrust facelessness.

Humans seem torn between their desires for both models. We do crave the communitarian model; everything from novels like Cory Doctorow’s Eastern Standard Tribe to the multiplicity of social networking services out there serve as notice that we do in fact have an emotional hunger for the trustworthy peer group, a hunger that is perhaps not serviced well by an impersonal world. The drive to create guilds even in game that offered no explicit support for it (as happened in the early days of Ultima Online is a good signal that there are currents here that run deep.

In today’s world, the frictions of locality and status and culture have been lubricated, and these new networks are somewhat different in character than they used to be. Whereas once your peer group could be relied on to be homogenous in certain axes, and heterogeneous in others, based on the factors shaping it, today it is apt to be homogenous on those axes that are most likely to be disruptive or controversial. It is no accident that historical events like the Enlightenment, the American Revolution, Protestantism, the rise of robber barons, the Red Scare, large-scale terrorism, and the appearance of EFF can be linked with the development of and widespread acceptance of media. In previous times, the people who had an affinity with these philosophies and worldviews were widely separated; with the advent of better communications and travel, they were able to coalesce into new networks ever more tightly, and thus start to achieve a sort of “bloc voting” power on the world’s affairs.

How very different from the world in which say, a single sympathizer of Enlightenment ideals, might live, were there no means of communicating with others like them! They would likely be attacked as going against social norms, and would be unsupported by a peer group.

This sort of homogenous network has long been a major driver of human progress; consider the early forms of banks, as they existed in the Jewish and Muslim worlds, with networks of merchants serving as guarantors of wealth. Letters of introduction served a truly enormous purpose in permitting large-scale commerce. The network of trust that existed between distant merchants permitted ventures that were not otherwise possible, as a rug merchant in Turkey allowed withdrawals of assets that were originally deposited in Egypt.

Then again, the weakness of communitarian models is also in their homogeny, which has many vulnerabilities. A glance at the medical sciences will each us that homogenous populations are at great risk from outside attackers, for example. A homogenous community is less likely to effectively assess transgressions against outside parties. It is more resistant to ideas in many ways.

  • Can we architect communitarian models that resist homogeny?
  • Should we see the move towards this new sort of tribalism as a net positive or negative?
  • Should we encourage instead the current real-world model of authority?
  • How do we solve the disenfranchisement issues if we pursue the authority model?

This last one becomes more urgent as disenfranchisement rises as the authority chain grows longer. The bigger the infrastructure of a hierarchy, the more likely it is that poor little A at the bottom will not have his needs served by C””’ way at the top.

We might also ask ourselves whether the communitarian model is in fact a polite fiction we sell ourselves; Tim Burke touched on this in his comments on Part I. Is there such as thing as true bottom-up authority? After all, communitarian ideals are driven by consensus, but in practice consensus, particularly in small groups, is driven by strongmen, by natural leaders, by persuasive techniques. This is precisely why the development of things like secret balloting was such a core driver of human socil development.

This line of thought leads us to ask whether trust itself is in fact fundamentally authoritarian. The 3rd party mediator structre is to a degree a reaction to the inherently authoritarian nature of humanity, particularly of small groups. It is an attempt to create arbitration via the “rule of law” as opposed to the rule of the mob (which is in practice usually the rule of the mob’s leader).

And yet… the hardwiring in our brains is to risk the lack of trust. We’re hardwired to give an untrustworthy situation a shot. At that point, our understanding of mediation changes; it’s not about providing trust via proxy, it’s about healing the network after a failed transaction, and about chunking the good and bad transactions that are possible into classifiable groups.

Effectively, the trust chain looks like this:

Identity/reputation -> accreditation (moral authority) -> enforcement
In the end, the modern fluidity of information, and the modern ability to exit a group, greatly affect the viability of the authority model. Today we have the cute picture of authority structures asking, “what is our recourse against word of mouth?” This is a fascinating question, because it starts conjuring up entirely new models of trust. There is no valid graph for a large authority structure that is also highly trusted. At best it runs on faith.

Why this matters
Consider the insurance industry. Insurance is all about assuming no discontinuities in risk. They lay bets against average risk levels, and they plan on paying out slightly less than they take in. When the overall level of risk rises suddenly, the insurance industry suffers. This is why it’s the insurance companies that fund forward-thinking fields like bioscience (pandemics = risk), research against global warming, and so on; and also fund efforts that mitigate rapid changes to existing industries, such as DRM. Rapid shifts in cultural standards are a serious problem for the risk industry, and that means that fluidity of information flow may actually work against insurers, just as rumors spreading of a cash shortage causes a run on a bank.

This is also the movie industry’s problem with cell phones — when you can call your friends to warn them off of Gigli before the movie has even ended, you’re undermining the traditional patterns of risk. It’s possibly also why the Spanish government fell after the Madrid bombing — so much information, so much of it likely wrong, flew across the cell phone channels that the political landscape changed in Spain almost overnight.

What arises is the question of whether the (generally considered by all, including me, as benign) democratization of information can paradoxically result in undermining our freedom of thought. It takes time to sort through large quantities of information, and “bloc voting” in the reputation game can throw the wrong websites to the top of Google, or possibly throw misinformation about whether the U.S. government warned all the Jews in the Twin Towers in advance, or whether the Spanish government was to blame for the deaths of everyone on those trains.

The saving grace is that the bottom-up model has many channels in it, homogenous and stampeding though they may be. Network studies of Amazon bestsellers in the politics category show a distressing bipolarity to the books read by each “tribe” — there’s left-wing books and right-wing books, but few people cross the divide to read the other side’s arguments. But that’s still an improvement over the number of channels present in the authority model: just one.

Why is any of this a problem? Because of the culture clash that results in applying the solutions of one of the models — authority or communitarian — to the other problem domain. The inevitable area of conflict will be whenever dispute resolution or transactional costs force an interaction of the two worlds. An example of this today is the justification given by Blizzard for not permitting recruitment of GLBT guilds on their General Chat channel, a purely transactional decision based primarily on costs to the authority running afoul of communitarian notions of enforcement.

In the next segment, whenever I get around to it:

  • The realm of politics and trust
  • Finally, LambdaMOO.
  • Does altruism have a place?
  • Confidence, recourse, and autonomy of response: the ingredients of the models

  18 Responses to “On Trust, Part II”

  1. Blogroll Joel on Software Raph Koster Sunny Walker Thoughts for Now Sex, Lies and Advertising

  2. Bartle?s 5 most important folks in virtual worlds [IMG] Posted by Raph’s Website [HTML][XML][PERM] on Fri, 20 Jul 2007 03:12:38 +0000

  3. […] Still to come in part 2 (and beyond, if need be): […]

  4. One thing you don’t address is that homogenous networks never occur naturally (that I know of). Networks tend towards a power law function, where the number of links decreases logarithmically with respect to how many nodes there are with that number. (I hope I got that right…)

    So a “hub” like a charismatic leader might know a hundred people, but a loner might know only five. One of those five might be the leader, though. It’s interesting that the leader’s opinion may sway the entire 100, but the loner’s will only matter if the leader agrees with it. In other words, those 99 people are indifferent to the loner’s opinion because he’s not connected to them. But if he routes it through a trustworthy medium, the leader, then it takes on weight.

    A book you might look into is Linked.

  5. Once, again, I suspect I need to post my full social network reading list. Linked is on it already, as are Small Worlds by Watts and Nexus and many more. 🙂 Michael, you may want to look at the Small Worlds presentation I did (different than the book cited above) at GDC 2003 to see the full range of sources I am drawing from here.

    In any case, I was referring to homogeneity not in the sense of equality of standing among members, but rather of opinions among members. In that sense, homogeneity happens naturally all the time, and is happening increasingly with the greater ease of finding those of like opinion.

    The characteristics of power law distribution in social networks start to happen very early, but small groups do display less clustering if you treat links digitally (e.g., there’s a link or there’s not). Once you make the links analog (there’s a greater or lesser strength to the tie) you start seeing power law distributions even in very small groups. Hence the references to how even in what appear to be “flat” hierarchies, opinion leaders still manifest.

  6. What you have not yet touched on is where an individual fits into a large corporation. I doubt that many vets trust SOE, but a great deal of folk still trust Raph. Not that many folk trust UO, but again a lot trust Garriot. Does the individual become more potent than the large org, or does the large org become more impotent as they screw up more, makeing the individual the focus of trust?

    Again taking SWG as an example, SOE is hated to such an extent that Sony Products will not see light in old SWG players homes. But most folk I know wonder if Raph realises what they have done to your precious sandbox (that became ours, when you gave it to us) ?. Interesting topic though, food for some thought:)

  7. It seems that democracy is not possible under either the communitarian or authority models since, in the former model, it is not truly communitarian (if we’re talking about MMOs), and in the latter there is no democracy. In either case the “lobbyists” usually prevail: in the authority model they lead to “nerfs”, and in the communitarian model the “community” being served is probably a vocal minority.

    So trust is therefore only established between either the authority and the “special-interest group”, or between the leader(s) and some (very) small community.

    For the greater majority of silent individuals I feel there is no linkage under either model.

  8. I wonder, is it more important to have a trust in your sources of information, or trust in the authority/leaders of the community itself? Is there such a thing as objective reporting or do all information sources have their own agenda?


  9. There’s always an agenda, even if said agenda is to have completely impartial reporting of news. More common, however, is the agenda of increasing circulation of a news channel, or the agenda of editorializing on news. This blog, for example, has both of those as an agenda when I report news. 🙂

  10. For the greater majority of silent individuals I feel there is no linkage under either model.

    That’s because they’re not. To gain links with the authority, you have to join that vocal minority, even if you’re a dissenting opinion. In other words, it’s the old “If you don’t speak up, you deserve what you get.”

    In order for it to be a true democracy in a MUD (it’s a bit out of the question for an MMO), I think the entire population would have to be coders on the project. =P And opinion leaders would rise.

    Michael, you may want to look at the Small Worlds presentation I did

    This looks very familiar… I’ve definitely seen it before. I just forgot all about it; it was like a massive refresher course. =)

    What you have not yet touched on is where an individual fits into a large corporation.

    Perhaps it has to do with identificability (is that a word?) with a human being. SOE and UO are organizations. Full of people, yes, but players only see an abstract entity. It’s kinda like being fiercely anti-Semitic, and having a best friend who’s openly Jewish. “I hate them all, except him, because he’s cool.” I don’t know if that properly applies, though, since I don’t know how people regard Raph or Garriot.

    Sony Products will not see light in old SWG players homes

    You mean you play with the lights on? =P (joke…)

  11. […] Somewhat related to the Reputation in MMOGs article and commentary from last week, Raph Koster has a two part piece looking at webs of trust and how trust is relayed through communities. […]

  12. Again interesting, but lets take the converse, my Granddad fought in 2 world wars, he loathed Hitler with a vengance – possibly the only time I ever heard him swear, but actually he quite liked Germans, so much so that he married a German Jew, which was really going against the grain in the old days. Not sure if an Org displays an ethos or the individual is the ethos, and spreads it through the org. UO is a game people look back on with some fondness, and some shuddering, but Garriott is a guy who most look forward to hearing what he is up too. Raph may not like it as it causes some embarrassment (I guess) but he is still liked even by people who are not keen on the Org he works for. I for one, like reading what he has to say, and like hearing what the “next” game may be, but I think his concept of Galaxies vs what is on offer now are 2 very different things 🙂

  13. Right, and though I can’t speak directly to your grandfather’s specific case – perhaps he had some peculiar scenario – liking (or not-disliking) a group does not mean you like all of them. Perhaps your grandfather disliked Hitler, not because he was German, but because he was a nutcase. That would suggest that he hates all nutcases, which really has nothing to do with Germans. An American nutcase would receive that ire, too (unless mitigated by some other thing, like an intense love of all Americans).

    This is indicative, I think, of “distrust” being selectively transitive. If you distrust Hitler, you don’t distrust all Germans, males, white dudes, guys with mustaches, nutcases, charismatic leaders, disillusioned artists, etc. There’s some kind of linkage making people part of a group, and I would guess that Hitler was not, first and foremost, a German in your grandfather’s mind.

  14. […] I tried to make my third content chapter, but it’s a toughie. What’s worse, it covers much of the same ground that Raph is covering here and here, but from a different angle (and much more tersely).You see, there’s a couple of connected “bits” here. “Trust” is one thing. “Interaction/Friendship” are another thing. “Exploration” is another thing. But all three are intrinsically tied to social networking.[…]

  15. […] “This guild is made up of arrogant elitists.” This is the response of those who suddenly feel disenfranchised. The appeal to a higher authority has left these people feeling like they are not part of the country club; they are suddenly second-class citizens. This is exactly the effect that I described in Part II of the essay on trust. […]

  16. […] Read Part I Read Part II Read A side note […]

  17. […] Raph has been building a series on Trust. I find the meandering through all things real and virtual, personal and sociological, to be a great springboard for one of the potential themes throughout: the relationship between MMOG developers/publishers and the players. […]

  18. […] usability trust user_generated Hinzugefügt vor >30 Tagen von ckreutz, 1 Benutzer speichern Raph’s Website » On Trust, Part II trust raph koster Hinzugefügt vor >30 Tagen von igorschwarzm, 1 Benutzer speichern Raph’s […]

Sorry, the comment form is closed at this time.