Affect and affordancies: my final (?) model Posted on August 6, 2017 by Matthew Tyler-Jones
(Its about how games trigger emotions, by the way. Oh and I’m not doing links, or proper references at this stage – you’ll have to make do with my EndNote metatext.)
Let’s start with some neurobiological research. Panksepp and Biven’s{, 2012 #96} thesis is that all mammals share seven instinctive emotions, even if different species’ higher brain functions can be very different. They label the seven core emotions: SEEKING; RAGE; FEAR; LUST; CARE; PANIC/GRIEF; and, PLAY
The core mammalian emotions defined by Panksepp and Biven
Everything else, they say, are sensory affects such as hunger, or socially constructed. It’s interesting to note that play is here identified as an emotion, not just a behavior. What evolutionary benefit does a play emotion give us, and other mammals? Panksepp and Biven suggest “to learn nonsocial physical skills like hunting, foraging and so on. It is also surely important for acquiring many social capacities, especially nascent aggressive, courting, sexual and in some species competitive and perhaps even parenting skills. It may be an essential force for the construction of the many higher functions of our social brains. Playful activities may help young animals learn to identify individuals with whom they can develop cooperative relationships and know who to avoid […] In short, the brain’s PLAY networks may help stitch individuals into the stratified social fabric that will be the staging ground for their lives.”
Nicole Lazarro{, 2009 #69} draws on the work of Paul Ekman in her study of emotions experienced by gamers. She explains how his research identified six emotions which appear to have universal facial expressions and a number of other emotions which while not quite meeting his criteria for being universal, come very close: emotions like Naches (pride in another’s achievements), Fiero. These two, along with schadenfreude and embarrassment add handy descriptors to the social emotions that Panksepp and Biven lump together and ignore in their study. Similarly, disgust puts a name to one of the sensory affects. I’ve also added her “ wonder” to that sensory category. And, while descriptors for the six core emotions sit well with her research, some of the emotions she lists in her study, for example Amusement, Curiosity, and Anger/Frustration, add nuance to some of Panksepp and Biven’s descriptors, so I’ve included them in the diagram below. All these emotions, she says, can frequently be recorded and recognised when watching players of video games.
Lazarro’s observed emotions in gamers
So, now we have a model of the core emotions, as described by Pankspepp and Bevin, overlaid with emotions attributed to gamers in the work of Lazarro. To that I have below added a third layer. The are affective triggers, drawn from Lazarro and Sylvester’s recommendations for game mechanics, or affordancies, and that trigger emotion in gamers, and Hamari{i, 2014 #148} et al’s motivation mechanics from their study of gamification. Given that each is a conflation of descriptions from up to three sources, they deserve some further explanation.
The final (?) model
Environment and spectacle –
While there is an enduring fashion for games that retain the “eight bit aesthetic”{Stuart, 2012 #165} the general trend of graphic development has been towards cinematic realism and “presence” {Riva, 2014 #4; Pinchbeck, 2005 #30; Pinchbeck, 2009 #33}. “Presence originates from the term ‘telepresence’, made famous by the computer scientist Marvin Minksy in a 1980 paper of the same name. From around 1991 (the date of the first issue of the MIT journal Presence: Teleoperators and Virtual Environments), presence has been typically defined as the capacity of the technology to make the user feel transported into a remote place and be able to efficiently interact with it.” {Pujol, 2012 #56}
Pujol and Champion attempt to unpick the definitions of presence. Starting with the idea that the ideal is a sense of being there, or blanking out the digital mediation of screen and controller, they touch upon immersion as a product of field of view and optical resolution. They also briefly summarize the idea that the human component of the system is likely to respond to the affordancies offered by the VR according to their interests, if the virtual component can in turn respond in a realistic way. They touch upon co-presence (sharing the VR with other users) before arguing that social presence (interacting with other users and virtual agents) is vitally important idea in the “potential [their emphasis] convergence between the presence and cultural heritage fields.”
“The conventional notion of presence as the sensation of ‘being there’ is a highly simplified way of expressing an internal perception of the environment and of ourselves in relation to it. A more comprehensive explanation would be that the sense of presence results from the interaction of various factors. These factors depend both on the system (immersivity, visual accuracy, real-time physical and social interactivity, invisibility of devices, consistency of the content) and on the participant (perception, attention, empathy, engagement, meaningfulness or relevance of the content, control, suspension of disbelief).” {Pujol, 2012 #56}
In one game, set in the tunnels of a post-apocalyptic Moscow underground, Sylvester describes how play moved to the ruined, frozen, surface and the powerful effect it had on him as a player:
“Though most would call Metro 2033 a shooter or RPG, I wouldn’t, because I don’t think its about shooting or roleplaying. I think its about discovering how a place like that makes you feel.”
Whether or not the gamers Lazarro studied felt they had been transported to the virtual world, she did see evidence of Wonder in their expressions, often from beautifully rendered or animated environmental effects. Sylvester separates out the beauty of “a sunset over the ocean” from the spectacle of “a slow-motion dive to dodge an incoming rocket.” He also offers a warning not to over use either.
Acquisition, points and leaderboards are reasonably self explanatory.
At the crudest level, they are sharable social evidence of success, skill or commitment. Their affective and motivational power comes from sharing, but acquisition also has an impact on the individual. As Sylvester acknowledges, gambling games are all about acquisition, and computer games often simulate the acquisition of wealth (or simply points). Of course gambling works in two ways, and the bitter emotions of loss shouldn’t be disregarded.
Insight –
Learning rewards and encourages curiosity and seeking. Not just any old learning though.
“If a lesson is obvious,” says Sylvester “there’s not much buzz in finally getting it because it was always fairly clear.”
Instead, he advocates a moment of insight, where everything that has come before “clicks into place and reveals the shape of the whole.”
Challenge is the emotional trigger that we most readily associate with video games,
testing the the player’s dexterity and pattern learning before rewarding him or her not just with a sense of accomplishment, but progression within the game. Sylvester argues that it is not as essential to games as it might appear, and cites Dear Esther as a game that can “create powerful emotions without players struggling.” However, Lazarro says that the opportunity for challenge and subsequent mastery can be an important motivation for some, if not all gamers.
She quotes the anonymous wife of one “hardcore” gamer
“I always know how my husband feels about a game.
If he screams ‘I hate it! I hate it! I hate it!’ then I know two things.
A) He’s going to finish it. B) He’s going to buy version two.
If he doesn’t say these things he will put it down after a couple of hours.”
Threat – Sylvester conflates revulsion (disgust) and fear,
but here I’m using Threat as an affordance of the core Fear affect. Some things (Sylvester cites spiders and snakes) scare enough people (whether instinctively or through cultural learning) that they are often used in games to provoke fear. Of course the fear response on a gamer sitting in an armchair in front of a monitor is going to be different from the fear of the same person lost in a dark forest, but some games work hard to create a close approximation. Sometimes it’s what you can’t see that scares you, and so game designers have learned to use cinematic effects such as jump-cuts and music to to increase tension and invoke a fear response.
Sex is a hard thing to get right in video games, though many have tried.
“Let’s be honest, sex in games is rarely super sexy sex.” Say the video game journalists Nielsen and Grey {Nielsen, 2016 #166} “… the key to effective video game sex: good writing, good characters.”
Which brings us to the narrative arc,
this is what film can do so well, engaging the audience’s empathy with one or more characters, as they face internal conflicts, grow and change. Change is a common factor for a lot of the affordances described above, and indeed, Sylvester declares that “the bedrock principle behind all emotional triggers is change”. I will address narrative in more detail in chapter 3. (XX)
Feedback, or rather the absence of it, is a trigger for panic/grief
which, as Panksepp and Bevin explain has a lot to do with isolation in mammals. When our character stops responding to your controls, is when a player feels a fleeting moment of grief. Feedback is offered to gamers through what happens on screen, haptic engines in controllers, and by sounds and music.
There’s a lot to say about music,
which I will address more fully in the next section. Before we move on, note how, in the model I’ve described, affordancies are placed in boxes radiating away from the core, social or sensory affects that they are most likely to trigger. In my text though I’ve already mentioned how some affordancies might contribute to the triggering of other affects: revulsion might help trigger fear for example.
Sound and music can be used to trigger, or contribute to triggering, all the affects. Imagine the soundtrack to Jaws or Psycho, and contrast that to the music of Love Story for example.
Music is playful, yes, so I’ve placed the label there, but its influence is more diffuse, an I’ve represented it in the diagram with a fuzzy halo around all the affects, to indicate how it can trigger our emotions without us even knowing:
“And music is wonderfully subtle – even more than most emotional triggers. Nobody ever gives it the credit it deserves because nobody consciously pays attention to it during play. But even though the conscious mind is oblivious, the unconscious is still processing the music into a continuous flow of feeling. You can tell because music is easily separable from the rest of the experience. Listen to a game soundtrack by itself, and you’ll feel much of what you felt during play. Play the game in silence, and you’ll be surprised at how hollow it feels.” {Sylvester, 2013 #89}
