Auto-puppeteering avatars patent
(Visited 6163 times) Game talk
Jun 092008
Massively has an article about a university in Australia patenting a way to extract emotional info from player actions and automatically puppeteer the avatar.
There’s a long history of this sort of thing out there, of course. This particular patent, for example, references pulling out emoticons from chat, as many worlds have done, but also pairing them up with voice analysis in order to better match up emotional markers provided by voice and the tone intended by a given emoticon.
17 Responses to “Auto-puppeteering avatars patent”
Sorry, the comment form is closed at this time.
I’m still holding out for an RP translator.
What they say: “@#$% you, you @#$% n00b! I’m gonna @#$@ kill your @#$% @#$%!”
What you hear: “Zounds, sir, I am sore wounded! Be you warned, I shall have my vengence!”
Of course, it could work in reverse as well:
What you say: “Huzzah! Good morrow to you, my lord, and well met!”
What they hear: “sup”
Hrm… I remember There having some of the best emotive response to cues in text and voice speech. It was darned impressive, especially considering the age.
More or less related and yes that’s also a shameless publicity, but I am really waiting for the emotivEpoc. ( http://emotiv.com/ )
I am not aware of any objective article analyzing how precise their technology is but it might just be great for animating avatar with ‘feeling behavior’.
Gad! That is almost a quote from the HumanML list discussions. I wonder if their patent conflicts with the patent the guys in the DC firm were applying for. I didn’t see the essential claims and that is what one needs to evaluate it against prior art.
The emotional state is pretty easy to derive. This is just a topical vector space although the situational semantics can be very tough if one uses a cultural history patterning algorithm. The tough part is the emote engine itself (controlling the avatar’s expressions). The U of Penn people did good work on this several years ago.
If one doesn’t care much about the patent politics, the approaches proposed seem to be validated by multiple research groups going down essentially the same path. It will be fun to see how this works out because in my opinion, this is where real AI for avatars begins. A fact database or assertion database doesn’t create emergent behavior that will pass a Turing test. Using a developing emotional structures (it is not a database; it is a set of engines routed together situationally) as the impetus for emerging selector structures will.
That may be how the story problem is solved in the sense of a never-ending story as a narrative of life.
Eve Online is attempting some neat emotive automated avatar behavior in their next expansion. The avatar is supposed to react to other avatars based on a bunch of different cues, including a player-controlled mood and in-game standings. The avatars are also supposed to have infinite morph points and numerous texture layers so you can really customize them like crazy. They even change appearance over time by getting fat and/or skinny.
I’m looking forward to seeing how far they go towards making my avatar sulk when I tell it I’m in a bad mood or perhaps walk with a bounce when I’m happy. CCP are really trying to go all-out according to recent interviews.
Hey, len, can you go into more detail on that last bit? How would that work exactly? I’m curious, but I lack the background to make complete sense out of what you said there about developing emotional structures.
This doesn’t really help the ingame / out of game context problem. Recording people’s reactions may just confuse what’s going on in their real lives (and short of a webcam there’s no sharing that) with what’s going on ingame.
There is a simple solution to all of this, of course. Players could always learn to type quickly. 😉
Pulling emoticons out of chat is really the only way to go… or just leave it to voip.
It takes a lot of space to get into it. I can’t do that here.
A basic approach is to routed values among objects that change a scalar attribute (eg, intensity) for emotional intensity and/or to pair emotional states as polarities. The key is understanding how fundamental emotions affect each other based on a set of internal and external stimuli and a history or learning set of events/objects encountered in an environment. This is sometimes referred to as a ‘situated’ system. Implementing it is harder given the need to identify objects in a promixate space and the change the intensity values as the objects are encountered based on prior experience and other factors such as cultural learning. A key observation is that emotions are generally not isolated experiences. The other is that behavior repetoires are learned even if agggregated with fundamental behaviors (smile, frown, wave) and these compound into complex behaviors such as greeting, dissing, and so on.
The problem is these are all just cognitive models, so fidelity is about as good as an actor achieves, but that can be quite good.
I see, so basically, you’re weighting emotional responses based on a set of interacting values that organically grow over time? Makes sense, since that’s typically how we develop as humans, or at least appear to.
Coupled with natural language processing, something set up like that would be very hard to distinguish from a real human, though, if I’m reading you right, there’d need to be some form of training time before the behaviors seemed to be more than child-like or, worse, alien.
Or the values can be preset to some theory of a personality type. I spent about a year researching this for the HumanML work and we found it was a fairly straightforward implementation in VRML/X3D because most of the components are easily modeled there. The catch is that there are many theories of personality types and the idea became this was a way to simulate those.
The project got dragged off the deep end and was pronounced a ‘monster’ by beltway bandits, but less than five years later we discovered applications for patents. Pro bono work on open lists always runs the dangers of the leeches taking the prize private. That is why I continue to support the Web3DC. It isn’t that the technology is the best, but that it is good enough for lots of applications and the IP policy ensures work done under their aegis can’t be heisted away from the public. It slows down growth because the venture capitalists hate that policy, but it preserves the values the web was built on originally. If we had stayed with the Web3DC instead of OASIS, I think we would have succeeded.
The basic ideas are not difficult. I was interested in watching emergent personality development to see if it accorded with cultural development looking for node points. It’s fascinating stuff because it pulls together many different complexity/chaos/network theories in a petri dish. That’s why I like Raph’s presentation. He is pulling together a lot of the same concepts and applying them to games/narratives which is where we started as well.
len, correct me if I’m wrong, but doesn’t having an example of prior art basically invalidate patents? I assume that the discussions and stuff are archived? I mean it’s a huge pain in the ass to actually fight that stuff, but if anyone wanted to there’d be an avenue for doing so no? Or am I being hopelessly naive in assuming that the system actually works as stated?
Still, really cool stuff.
The system works but it has to be executed, meaning, unless someone protests it goes through. The next problem is what are referred to as ‘essential claims’. In the US, a patent is based on a set of claims that are essential to the uniqueness or orginality of the invention. Often when we see these patents described in the press they are described in broad terms whereas the patent examiner has worked with the inventor to narrow the claims to the essential claims to narrow the coverage of the patent. The next problem is expertise and resources at the patent office with respect to the domain of the patent. A good friend of mine was the patent examiner for the EOLAS case and it mentioned two items to me 1) failure of software inventors to timestamp the documentation used to claim prior art (and also to recognize that others in their field had done similar work prior to their own, sometime willful) 2) the capacity of the patent office to work in detail on patent claims given the acceleration of such as the value of intellectual property began to soar in the 90s. Remember that prior to the win in the Intergraph chip case, patents in the software industry were rare.
The 3D graphics/virtual worlds/games market is a special case. It is relatively new and very hot. The big companies have been trying to capture the IP flags there for over a decade to dominate the market. It was a real source of frustration for the Web3D Consortium members because by dint of origin, they espoused open source and open IP, then the patent trolls from big and small companies began an assault. It was very hairy, but the formal participation agreement settled the terms. It is this agreement that keeps their numbers small and why some large companies will not support the Web3DC. It is why some of the recent attempts to form new consortia (actually trade organizations that act as standards authors) have stalled. The Web3DC acts as the holder of the contributed IP under the agreement and any standard written in partnership with ISO is subject to those agreements. It enables the consortium to act as the patent pool and ISO to oversee the standards process. It’s a good deal for the members but doesn’t often meet the ambitions of the big companies.
As I said, good enough technology and holding the line for start ups so they don’t get edged out. The biggest single beneficiary are the content companies who aren’t hostage to the browser vendors and the content owners who can rehost content when a browser vendor goes belly up or decides to abandon a product line. We learned the hard way in the VRML days that both happen far too often and given the high cost of 3D content, kept the market in a sub-optimum minima in the emergence slope.
IP is the bear in the woods of 3D on the web.
Ech. That really is a mess. Software patents always bothered me to a degree, especially considering how few of them were really that original. Patents were designed to encourage innovation, not to be a bludgeon for big business to use to stifle it.
Thanks for the overview though.
The standards/IP keiretsu with formal participation agreements seem to be the best solution. They pool patents and determine their values for working in the market. It really does come down to values. Like it or not, a very large percentage of the world act as senseless animals taking as much as they can regardless of the effects. Some do that brutally and others put a charming face on it with high sounding ideals but the truth is in the behaviors. My cynical sounding advice is to put together those who by dint of past behavior show they can be trusted, then watch each others backs. Sad but so.
Friends,
I’m thrilled to see this excitement over this technology, long over due. If you go to the US PTO website, http://www.uspto.gov and search on inventor Walt Froloff, you should find several patents that already do this and much more. In fact, emotive state vector was defined by me in that first 2000 patent. I have 4 patents after that, some have not been published as yet, don’t ask my why, they were filed 2004ish. I will also be releasing a book on the subject in a month or so available on Amazon.com, “Irrational Intelligence” This book has the models and methods for programmers to go to work in this long ignored field of feelings in devices. The book shows many applications of this technology. The back cover of that book will reads (sorry for the shameless plug here):
Irrational Intelligence teaches:
How to capture and transfer emotions through an interface
How to use feelings in to make incremental functions smarter
How to use feelings to bridge devices between languages
How to process a diversity of feelings in a computing device
How to use feelings in applications to enrich communication
How to manage serial or multiple concurrent feelings
How to use feelings to simplify complexity in computer game
How to use feelings in business and Internet
How to use feelings in business apps to maximize transactions
How to understand the emotive engine that lies within
Eolirin and Len,
The cognitive models presently are too primitive. And the inputing of emotive content is also currently very privative as you note. But with verbal commands, it could be as fast/faster than typing or selecting point/click. My book presents an emotive model which merges the subjective feelings of user with the objective definition of language/symbols. Then maps feelings to a pleasure-pain space and an energy model to charge and discharge feelings to proportional behavior. One can define an emotional make up for a creature, I call this Artificial Emotive Intelligence (AEI), which comprises a FSM architecture, which uses “external” events,eg x-y-z proximity, weapon type and range, threat (even verbal), etc, to drive creature emotive states and intensities, which then charge or discharge w/in proportional creature/character behavior. In this Emotive architecture, the characters become totally autonomous, relieving the programmer of the major hassel of defining a very confining behaviors and scenarios. Emotive states and intensities have build in functions and graphics in libraries which then synch up the events with verbal exchange with the prosidic interface and the visual/graphics. Its a very sweet and simple architecture, works just like us. That was my inspiration.
BTW, I am also an inventor, not a troll or ogre. I have a major interest in seeing all of our devices get smarter and especially games. It irks me to see my son waste so much of life with these mindless substitutes for games that are nothing more than military recruitment and training devices. I work to better myself. Sometimes you have to protect yourself from big entities that would step on you and not notice. Patenting does that somewhat.
Walt
Sure, I meant more that patenting often becomes a bludgeon when the big guys get there first and then stop any actual development from occuring because it doesn’t help their bottom line for it to.
The intent behind the patent system is a noble one, and for reasons you describe. The problem is that the reality of it has far too often become counter to that intent.
Anyway, I’ll be looking into that book, thanks for the heads up.