Game Studies

The Semiotics of the Game Controller

Abstract

An important question for video game research is how to characterize the video game experience. One common view emphasizes that players are cognitively able to operate with the game world as the experiential point of reference leading to states of "immersion, "spatial presence", or "engagement". A different view is that the player is also embodied and situated in the actual world, entailing that the video game experience also involves physical preconditions like hardware for playing. One aspect that has not been clearly thematized and analyzed is the contribution of the video game controller. I propose that the controller is not just an input device that lets the player act in the video game, but to play a video game is to act both on the controller and in the video game through the controller. It has the the dual character of requiring the player to act in the actual world while simultaneously providing the player with the link to the video game. This duality is similar to signs, which makes it possible and fertile to analyze game-controller pairings according to general semiotic principles.

Keywords: game controller, video game experience, engagement, signs, semiotic resources

Introduction

Video games are complex systems where player, video game, and hardware are communicating with one another. To describe how this gives rise to the interactive character of video games is a fundemental question for video game research. What are the general features of the video game experience and what are the conditions for the communication? In discussing questions such as these, two diverging positions have been put forth. On the one hand, the game experience has been framed in terms of the player's cognitive ability to become engrossed in the video game and to thereby operate with the game world as their primary point of reference. This experiential immediacy is descrirbed with notions such as "immersion" (McMahan, 2004), "spatial presence" (Wirth et al., 2007), and "cognitive flow" (Baron, 2012). Given the focus on the player as already having a functionally competent ability to play, the question what makes this possible has been less explored. Rather, it has been taken for granted as something that happens more or less automatically. The immersionist viewpoint has been criticized from a phenomenological perspective (e.g. Kirkpatrick, 2009; Keogh, 2015). In playing a game, the player's experience has been characterized as a state of "split" or "proxied embodiment" involving both the actual and a virtual world (Keogh, 2015; Swink, 2009). By targeting the reciprocal and amalgamated relation between video game and player, it has been noticed that the video game experience is enabled, shaped, and formed by material preconditions like a specific hardware with a specific input device. In Keogh's words, the "player's embodied engagement with the material form of the videogame precedes (and perceives) any interpretative engagement with the videogame's virtual 'content'" (Keogh, 2015, p. 43).

The claim I want to put forth is that this dual character is constitutive for the video game experience and is crucially made possible by the video game controller. To use a controller is to act with a physical device in the actual world, but this action is immediately connected to an action in the video game. For instance, to press 'A' in Super Mario Bros. (1985) is to perform a jump. Thus, acting with a controller necessarily situates the player in the actual world, but provides a direct link to the game. Despite claims to the importance of the controller for the video game experience (e.g. Kirkpatrick, 2009; Crick, 2011; Keogh, 2015), the nature of its contribution still remains to be detailed.

I begin in Section 2 with a phenomenological description of some general features involved in using a controller and how it is connected to the video game experience. I move on in Section 3 to a semiotic analysis of the video game controller. Game-controller mappings are shown to share some distintive properties with signs (see also Bernal-Merino, 2016). Moreover, by analyzing specific game-controller pairings, I show that playing involves a gradual attunement to knowledge of semiotic principles. I conclude in Section 4 with some implications of a semiotic approach for understanding the game experience.

The game controller: Forgotten but insdispensable

With the exception of the attempts to develop technologies for virtual reality[1], video games are played with some kind of artifact registering player input, i.e. a controller of some kind. In discussing the video game experience, Crick notes the controller's basic and indispensable contribution for providing the player with features like agency: "the fundamental aspect that allows a video game player agency in a virtual world is, of course, the control device – affording the player the opportunity to act directly on and in that world" (Crick, 2011, p. 266). As Crick goes on to describe the controller "as an extension of the player's body," one is reminded of Merleau-Ponty's description of the blind man and his cane. From the point of view of a blind person, the cane is not an external object, but an integrated part of sensori-motor experience extending the reaches of the body (Merleau-Ponty, 1962[1946]). This situation can be compared with the experiential immediacy of being engrossed in a video game: the controller is an integrated part of the video game experience. Over time, there has been a gradual progression towards complete controller integration in the player's experience. Only the inexperienced player is painfully reminded of the controller's indispensable nature. In learning to play, one looks at the controller just as much as to the screen to identify the different buttons for carrying out the proper response. As one becomes more experienced, the controller becomes habitualized and recedes more and more to the background and the player can to a corresponding degree focus their attention to the game. Keogh (2015) describes this competence as a form of "embodied literacy." How the player comes to attain a mastery over the game controller is similar to the process of learning a language. The competent speaker uses language quite automatically without attending to how they produce it. In contrast, while in the process of learning a language, the main effort is on finding and producing the appropriate linguistic expressions. In a similar vein, a player must learn how to interact with the game in order to play it in a functionally competent way. In the act of playing, the controller is at the same time indispensable and marginalized. As Kirkpatrick (2009) notes, this dual character of the controller as both necessary and forgotten is reflected in how rarely the game controller is discussed:

The controller occupies a paradoxical position in computer game studies. Although it is central to gameplay experience – it marks physically the difference between play with a game and merely watching a screen – it goes largely unreflected on by gamers and in gaming literature. (Kirkpatrick, 2009 p. 130)

A famous example drawing on this paradoxical character is the boss fight against Psycho Mantis in Metal Gear Solid (Konami, 1998), which highlights the split between the material existence of the controller and what it represents. By virtue of having psychic abilities, Psycho Mantis anticipates the player's every move. To beat him, the player must come to the realization that the "psychic" link is broken by changing to a different controller port. What is effectively happening in Metal Gear Solid is that the forceful breaking of the fourth wall highlights the marginalized role of the controller and instead makes it thematic for the game experience. A succinct summary of how this tension is constantly involved in the video game experience is provided by Rush (2011):

[T]he player's awareness of the material/semiotic system… generates the tension between its two distinct perspectives: the physicality of the gesture on the mouse/controller and the system of its representation." (Rush, 2011: p. 248).

It is exactly in this tension that we find an important clue to the game controller's contribution to the video game experience. As Rush points out, a video game is a semiotic system, which crucially involves a pairing between controller and video game. Their relation to one another is comprised of two related but distinct actions, like pressing 'A' on the NES controller to jump in Super Mario Bros. (Nintendo, 1985) or swinging the Wii Remote in Zelda: Skyward Sword (2011) to perform a sword slash. It is through semiotic relations like these that the player becomes able to act in the game world while still remaining situated in the actual world. In the following section, I detail the semiotic properities of game controllers further and relate them to their constitutive character for the game experience.

The game controller: a semiotic account

In general terms, video game controllers are all devices that "[translate] muscle movements [...] into a language the computer understands" (Swink, 2009: p. 69). Behind this general definition, a huge variation in the design of video game controllers can be found (Saunders & Novak, 2007). For contemporary consoles, the design has converged on a somewhat standardized layout, like the close-to similar button layout and -placement on the contemporary generation of consoles (i.e. the Playstation Dualshock 4 controller, the Xbox One controller, and the Switch pro controller). Despite this increasing homogeneity, the design of controllers has developed from various sources of both gaming and non-gaming devices alike. Even though there has ben convergence in controller design, some traces of the design history are still present, for example that they have the directional pad of NES/Famicom as well as the joystick associated with Atari 2600 (Lu, 2003). In the following, I apply a semiotic analysis primarily to controllers like game pads. This delimitation serves as a starting point and, as we shall see in Section 3.2, to illustrate how indivual pairings between controller and video game always operate within a system of available game actions.

Viewed as devices that translate muscle movements, video game controllers are designed to have a specific function. This view of the controller is also implied by my earlier comparison to a blind man's cane. On such a view, the controller can be seen a type of tool. Such a categorization would be accurate in at least two respects. First, a tool extends from the body of its user and points to an activity beyond itself (Prieto, 1975). For instance, a hammer points to the activity of attaching something with a nail. Similarly, the controller can be seen as pointing to whatever activity is carried out in the video game. Second, just as the hammer is not thematic in the activity of hammering, the controller is not thematized as an object of a certain color, size and shape. Rather, it is integral to the activity of playing a video game to the extent that using the controller is the activity itself.

Despite these similarities, to consider the video game controller as just a tool is at risk of missing the point raised in the previous section: the controller-video game pairing as comprised of two intimately connected, but distinct actions. An action exerted on the controller in the actual world corresponds to an action within the gameworld. By virtue of embodying two distinct activities, an order of dependence is established such that the controller-action attains its significance and meaning in relation to the particular function it maps onto in the video game. From the perspective of the player, these two elements are differentiated so that one is not confused with the other. In other words, the buttons of game controllers do not only afford pushing them, but it is an activity that attains its specific meaning in relation to a particular game (or in relation to a mechanically and systemically distinct segment of a game).

The amalgamated relation of two actions is similar to signs like words and pictures. A sign connects two elements: an expression, say a string of sounds or letters which is inextricably fused with a content: that which the expression "stands for." For instance, the written string 'elephant' in English has a grey, big animal with a trunk as its content. An expression without content, say the sound 'goggabonk', is to a speaker of English just a sound. Content without expression would then be something like an idea without signification [2]. Importantly, a sign such as the word elephant can be used to signify its referent in the absence of any real elephants. That is, a sign stands for something other than itself. In a similar way, using a game controller allows the player to act in the gameworld, while still remaining located in the actual world.

At this juncture, it should be noted that video games realize a plurality of semiotic relations. A video game involves visual representations, for instance in-game buildings resembling actual buildings in some respects. Mechanical actions like jumping and shooting do not just connect video game and controller, but are also representations of these actions. These additional layers of representations are part of the broader semiotic context of video games, but not something I will elaborate on here. Instead, I focus the discussion to the relation between player, video game, and controller.

To date, semiotic theories and analyses have been quite uncommon in video game research. To my knowledge, a semiotic perspective has been explicitly endorsed to engage with issues like the difference between simulation and play (Myers, 1999), playing as a heterogenous activity (Kücklich, 2009), and games as both designed representations and as objects for player agency (Huber, 2013). A semiotic analysis of the communication between player and game is also provided by Bernal-Merino (2016). In the observations concerned specifically with pairings between game and controller, Bernal-Merino makes several observations similar to mine, but is not concerned with the controller specifically. To further specify some semiotic aspects of the game controller, I rely primarily on the Sonesson's (1989, 2003) analysis of signs and other type of meaning. Taking departure from this, I propose that a semiotic framework can be fruitfully applied to the video game experience, and hence provide a clue to Crick's characterization of the controller as a "fundamental aspect."

The coupling between game and controller

In principle, anything can be a sign for something. The surrounding world has a ubiquity of representations. To avoid underdetermining the notion of the sign, there are specific ways in which expression and content are connected to one another. Their relation to one another is in Peircean terminology the ground (Deely, 1982; Sonesson, 2003). For instance, the expression and content can be grounded in a resemblance-based relation, like an image representing a real-world object. The ground is the perspective from which the subject takes the former to stand for the latter.

Based on this semiotic framework, I want to suggest that the video game experience presupposes the player's familiarity with general semiotic principles. To play involves drawing from the implicit knowledge of these semiotic principles. It is by learning how they are realized that the player can forget the controller and become engrossed in the game. In my discussion, I will highlight the three different semiotic grounds for establishing the relation between game and controller: resemblance (iconicity), contiguity (indexicality) and convention (symbolicity) [3][4].

Resemblance

It can be been argued that interaction is at its most transparent when the controller-action resembles the action represented in the video game. One example is game- or genre-specific controllers designed to physically resemble their out-of-game referents, like the guitar in Guitar Hero (2005). This establishes visual, and to some degree, tactile resemblance between controller and its out-of-game referent. However, these similarities might not be sufficient to establish an iconic relation between game and controller: this coupling also requires a functional resemblance between the video game action, the controller, and the out-of-game referent. An example of such a resemblance-based relation is how pressure-sensitive shoulder buttons found on most contemporary console-controllers are often used. In first- and third-person shooter games, the right shoulder button typically maps to firing a weapon. Since the button is pressure-sensitive, it can mimic pressing the trigger of different types of firearms, and does in this specific regard resemble (part of) guns as a specific outside-of-game referent. When it maps onto the firing of a weapon, then the relation can be analyzed as iconic. When it is not used to fire a weapon, then the semiotic relation would be spelled out differently. In other words, it is the relation which is based on a certain principle. Just as the same sounds can form two different words in two different languages, the same button can be used for different actions in different games (or in different parts of the same game).

The type of resemblance-based pairings that arguably has received most attention is found on so-called "mimetic controllers" (a term used in Keogh, 2015): motion-sensitive controllers where a combination of techniques like gyroscope, gesture recognition, and optical sensors are used to detect and map the player's movements onto pre-defined actions in the game. Mimetic controllers are based on a motoric resemblance between the in-game action and a schematic template for which movements that count as "similar." For instance, a sweeping lateral move with the Wii Remote in Zelda: Skyward Sword (Nintendo EAD Group No 3, 2011) maps to an in-game slash with the sword; extending the arm in a thrusting move corresponds to Link lunging forward with the sword.

Mimetic controllers intend to provide a visuo-motor resemblance between the player and the video game. This form of similarity has been described in terms of "natural" as opposed to "traditional" controllers (e,g, Skalski, Tamborini, Shelton, Muncher & Lindmark, 2011; Shafer, Carbonara & Popova, 2014). However, to view such controllers as more natural is problematic for two related reasons. First, mimetic controllers also operate on coupling expression and content together, which is based on a semiotic relation between acting on the controller and the video game action. The relative ease of identifying iconic relations is not an argument that it entails having a more natural form of controller: it still presupposes that the player can connect controller and video game together in a sign relation. Second, while the connection between video game and controller might be iconically based, it is also the case that the experienced player has learned to alterate and distort their movements in a way that makes them less iconic vis-à-vis the actions represented by the game. Simply put, to play Skyward Sword does not make one a better swordsman[5]. In this regard, an iconic ground does not entail that players must actually move their bodies in a way resembling the on-screen avatar's movements. In other words, the degree of connection to the represented action might decay over time and the player movements need not be in the form of a mimetically simulating the represented action (Grodal (2002) raises this as a general effect of habituation on attempts at realistic representation in any media).

Contiguity

Signs can be based on spatial and temporal contiguity between expression and content (an indexical ground). Phrased in terms of classical rhetorical figures, an index establishes a metonymic relation between expression and content: the part stands for the whole. Various types of sign relations are based on contiguity. Traces made by an animal and smoke signaling fire exemplify what Grice (1957) called "natural meaning." The weather vane and the thermometer are both instruments based on indexical relations: how the weather vane is turned indicates the direction of the wind. But perhaps the most characteristic of indexical signs is pointing the index finger.

In video games, it is more difficult to come to find examples of indices than icons, but there are some examples. In many touch-screen based games, one can physically point on the screen to indicate what action should happen where. In Zelda: Skyward Sword (Nintendo, 2011) and Red Steel (Ubisoft Paris, 2006) for the Nintendo Wii, the Wii Remote is used to in some ways motorically-iconically represent a sword. At the same time, the controller itself is held as the hilt of a sword. The controller does not resemble the sword in its entirery, but by being held as a hilt, it indicates the whole sword. A similar use is found in the horseracing game Championship Jockey: G1 Jockey & Gallop Racer (Koei, 2010) where a whipping action with the PlayStation Move exhorts the player's ride.

The resemblance between shoulder button and triggers also highlights a part-whole relation. The shoulder buttons on many contemporary controllers resemble the trigger of a firearm but indicate the entire weapon. In this way, the trigger-shooting amalgam uses both iconic and indexical elements. A crucial facet of semiotic relations is thereby highlighted: signs are not restricted to one principle, but can – and often do – have more than one ground.

Conventionality

Many different kinds of signs have their ground in conventions, habits, regularities and sheer iteration. For such signs, the relation between expression and content is not granted by perceptual relations like similarity (icons) or contiguity (indices). A clear example is most linguistic signs which typically lack a transparent link between expression and content. An English word like elephant is not perceptually or otherwise similar to a large animal with a big trunk. In semiotic terms, such relations can be described as conventional or symbolic. While symbolic signs might lack a transparent connection between expression and content, they can still have systematic and motivated relations to other signs. This is something we return to in 3.2.

Individual mappings between controller and video game are often symbolical, and this is clearly the case with most uses of face buttons. In platform games such as Mega Man (Capcom, 1987), two types of actions can be performed with the two face buttons: jumping and shooting. The former action is carried out with the A-button, the latter with the B-button – the only face buttons on a NES/ Famicom-controller. None of the buttons are more apt for jumping than the other, unless one argued that the button closest to the thumb should be mapped to the most common action. This might have motivated the choice initially, but it has just been conventionalized that the button marked 'A' (or the button placed in similar position) is used for jumping in platforming games.

Even though face buttons typically have symbolic-conventional relations to in-game actions, they can be used iconically. In Track & Field II (Konami, 1988), the higher intensity the player sustain in prissing the A-button, the faster the virtual athlete runs. Thus, the measurable physical effort of the player corresponds to a comparable effort in the game.

Using the Semiotic Grounds: The Controller-Game As a System

The present analysis of the controller has been limited to individual pairings between in-game action and controller-action. Situated in an actual video game, there are several different actions that can be performed. When all of these are taken together, they participate in creating a holistic and hopefully coherent system for interacting with the mechanics and systems of the video game. An indicative example can be found in fighting games. With the use of single in-game actions carried out by stringing together several controller-actions (so-called combos), actions are not limited to individual buttons. Playing as the character Guile in Street Fighter II (Capcom, 1991), the three-button sequence "back-forward-punch" unleashes the single attack "Sonic Boom". By combining several button presses into single actions, the number of possible actions are dramatically increased.

The systematic and relational properties signs can affect their symbolic character, even to the extent that iconic relations can be established. This can be illustrated with linguistic examples. In many languages, there is a strong tendency for the plural form to be longer than the singular form (e.g. game-games), which corresponds to a quantitative increase of the referent as well. The correspondence between increase along the expression and content plane can be analyzed as forming an iconic relation emerging from the systematic relations between signs.

Such relations are examples of "diagrammatic iconicity" (Jakobson 1965), which can be applied to the coupling between controller and game. An example is the standard layout for most 3D-action or adventure games played with a mouse and a keyboard, e.g. Unreal Tournament (Epic Games/Digital Extremes, 1999). Here, the keys W, A, S and D are used for controlling the movements of the player character. Taken as an individual mapping, W is symbolically related to forward movement (any other key might just as well have been used without changing the type of relation between expression and content). When taken together with A, S, and D, the buttons form an inverted T-shape where W is in front of S, mapping to forward/backward movement, respectively. To to the left and right of these are A and D are which map to leftward/rightward movement. This means that a control system in its entirety may allow for motivated differentiation along the expression plane.

I would like to conclude this section by extrapolating on the relation between game as a medium and how it articulates meaning. Focusing specifically on the controller, there are instances where the configuration of the controller both constrains and enables how a game is played. To illustrate this, we can look at Goldeneye (Rare, 1997), a console first-person shooter that viably implemented the key functions of its mouse-and-keyboard controlled siblings. The unorthodox solution of Goldeneye was to split movement and aiming between both thumbs: lateral aiming and forward/backward movement with the left hand on a joystick and sagittal aiming and left/ right movement with the right hand using four face buttons. This example indicates that mechanisms of a game can adapt to the resources provided by the controller and its material configuration. Certain kinds of games do not function well, or at least we believe so, with certain controllers; for instance, they do not provide enough systematic means for differentiation between game-actions, like too few buttons to implement all the needed functions. In this regard, the semiotic-material condition of the controller might impact game design (Bogost, 2015)[6].

Conclusion

A semiotic account aims to clarify the ways in which a certain medium or resource conveys meaning. It should also specify the unique properties of a given resource. In this paper, I have proposed such an account for the video game controller, with a specific focus on game pads for contemporary consoles. This unveiled that two important features contribute to the video game experience. First, the video game experience is always mediated by a controller. The player is using the controller to get to the video game. In this way, the controller exhibits the dual character of existing in the actual world but at the same time allowing the player to act in the video game. By virtue of realizing two amalgamated actions, the relation between controller and game resembles that of signs. Second, the controller's mediation uses various and often fused forms of the three semiotic grounds (resemblance, contiguity and conventionality) for connecting expression to content. The cases discussed in Section 3 exemplify the different and simultaneously multiple grounds involved in mapping controller and game. This illustrates that the player's relation to the video game can be analysed from a semiotic point of view, and that such an analysis provides insights into how the controller allows players to act in and become immersed in the game. Even though they are just one part of the picture, the semiotic properties of video game controllers are nevertheless decisive features for providing the player with a link to the game. To corroborate these claims, future analyses should analyse additional input devices and controller types.

Since the controller establishes sign relations on the basis of players' semiotic knowledge, they function as an indispensable mediating layer for the game experience. It is for this reason that the tendency to strive for mimetic controllers, more transparent, and "natural" ways to interact with video games may be a misnomer (a point also raised by Keogh, 2015)[7]. From an experiential point of view, the player differentiates the game action from the controller action and binds them together according to the different semiotic grounds. Even though there is differentiation between the controller and the gameworld, the controller allows the player to get from one to the other.

As an input device, video game controllers arguably aim to be intuitive as well as easy to learn and use, but they also allow the player to engage with the gameworld. This is one of the important properties of the semiotics of the game experience. Even though the player need not feel immersed in the gameworld (cf. Kücklich, 2009), their actions must be vested with meaning and relevance within the game. In other words, the interactive practice is semiotically mediated to allow the player to participate in the gameworld (Rush, 2011). The proposed analysis considers the game experience as crucially involving game actions mapped to controller actions. Just as we use language in an automatic fashion, the controller's specific semiotic properties are often forgotten. The controller is the condition for playing video games, but once we have learned how to play the game the controller retracts to the margins of consciousness.

Endnotes

[1] In this regard, it is notable that contemporary solutions to virtual reality, such as the Oculus Rift, also require the player to use a hand-held controller.

[2] To view the sign as comprised of expression and content has been met with harsh criticism, for instance by Derrida (1976). In this paper, however, I do not aim to explore a general theory of the sign, but rather to explore some characterizations of the video game experience from the player's perspective. From such a point of view, the sign need not be problematic but perhaps even a necessary concept – as acknowledged by Derrida (1982).

[3] It should be noted that I refrain from looking at controller functions like pausing and entering the game menu. Even if such "deludic modes" are part of the video game experience (Kücklich, 2009), the player is not interacting with the gameworld and is therefore not involved of the specific relation I am concerned with here.

[4] Following Peirce, the terms for the three semiotic grounds have been iconicity, indexicality, and symbolicity. For sake of simplicity, I also use the more transparent terms resemblance, contiguity, and convention (Zlatev, 2018).

[5] I would like to thank an anonymous reviewer for making me aware of this important point.

[6] The relation between digital media and the hardware and computing systems that enable them has been the subject of platform studies (e.g. Montfort & Bogost, 2009). Albeit more limited in its scope, my claim that the material configuration of the controller can constrain and enable games is in some respects a related claim. A specific controller layout can influence design desicions to a considerable extent. One example is Starfox Zero (2016) which was explicitly designed with the goal to utilize the hardware and controller of the Wii U. Moreover, games like Goldeneye (1997) and Halo (2001), are clearly designed with a specific controller in mind, which can in turn influence the design of controller layouts more generally. I am grateful to an anonymous reviewer for making me aware of this connection to platform studies.

[7] A related question is whether some controller layouts are more easily learned or are experienced as more natural (cf. Williams 2013). By detailing different forms of interactive principles and practices, it is possible to study more precisely how players learn video games.

References

Baron, S. (2012). Cognitive flow: The psychology of great game design. Retrieved January 1st, 2018 from https://www.gamasutra.com/view/feature/166972/cognitive_flow_ the_psychology_of_.php.

Bernal-Merino, M. Á. (2016). Creating felicitous gaming experiences: Semiotics and pragmatics as tools for video game localisation. Signata, 7(16), 231–253.

Bogost, I. (2015). Video games are better without characters. Retrieved January 1st, 2018 from www.theatlantic.com/technology/archive/2015/03/video-games-are-better-without-characters/387556.

Ciccoricco, D. (2012). Narrative, cognition and the flow or Mirror's Edge. Games and Culture, 7(4), 263–280.

Crick, T. (2011). The game body: toward a phenomenology of contemporary video gaming. Games and Culture, 6(3), 259–269.

Deely, J. (1982). Introducing semiotics: Its history and doctrines. Indiana University.

Derrida, J. (1976). Of grammatology. Baltimore: The Johns Hopkins University Press

Derrida, J. (1982). Positions. Chicago: University of Chicago Press.

Grice, P. (1957). Meaning. The Philosophical Review, 66, 377–388.

Grodal, T. (2002). The experience of realism in audiovisual representation. In A. Jerslev (Ed.), Realism and 'reality' in film and media (pp. 67-92). Copenhagen: Museum Tusculanum Press.

Hjelmslev, L. (1953). Prolegomena to a theory of language. Baltimore: Indiana University Press.

Huber, W. H. (2013). The foundations of videogame authorship. PhD thesis: University of California, San Diego.

Jakobson, R. (1965). Quest for the essence of language. Diogenes, 13(51), 21–37.

Keogh, B. (2015). A play of bodies. A phenomenology of videogame experience. PhD thesis: RMIT University.

Kirkpatrick, G. (2009). Controller, hand, screen. Aesthethic form in the computer game. Games and Culture, 4(2): p. 127–143.

Kücklich, J. (2009). A techno-semiotic approach to cheating in computer games or how I learned to stop worrying and love the machine. Games and Culture, 4(2), 158–169.

Lu, W. (2003). Evolution of video game controllers. Retrieved September 19th, 2018 from http://web.stanford.edu/group/htgg/cgi-bin/drupal/sites/default/files2/wlu_2003_1.pdf

McGloin, R., Farrar, K. M. & Krmcar, M. (2011). The impact of controller naturalness on spatial presence, gamer enjoyment and perceived realism in a tennis simulation game. Presence-Teleoperators and Virtual Environments, 20(4), 309–324.

McMahan, A. (2003). Immersion, engagement and presence: A method for analyzing 3-D video games. In M.J.P. Wolff and B. Perron (Eds.), The Video Game Theory Reader (pp. 67–86). New York: Routledge.

Merleau-Ponty, M. (1962 [1946]). Phenomenology of perception. New York: Humanities Press.
Montfort, N. & Bogost, I. (2009). Racing the beam: The Atari video computer system. Cambridge: The MIT Press.

Myers, D. (1999). Simulation as play: a semiotic analysis. Simulation & Gaming, 30(2), 147–162.

Prieto, L. J. (1975). Essai de linguistique et sémiologie générales. Genéve: Droz.

Rush, J. (2011). Embodied metaphors: expressing control through first-person shooters. Games and Culture, 6(3), 245–258.

Saunders, K. & Novak, J. (2007). Game development essentials: Game interface design. Thomson Delmar Learning.

Shafer, D. M., Carbonara, C. P. & Popova L. (2014). Controller required? The impact of natural mapping on interactivity, realism, presence and enjoyment in motion-based video games. Presence-Teleoperators and Virtual Enviroments, (23)3, 267–286.

Shimomura, David. (2015). The semiotics of controller design. https://killscreen.com/articles/semiotics-controller-design/. Retrieved 180117.

Skalski, P. Tamborini, R., Shelton, A., Buncher, M. & Lindmark, P. (2011). Mapping the road to fun: Natural video game controllers, presence, and game enjoyment. New Media & Society, 13(2), 224–242.

Sonesson, G. (1989). Pictorial concepts. Inquiries into the semiotic heritage and its relevance for the analysis of the visual world. Lund: ARIS/Lund University Press.

Sonesson, G. (2003). The symbolic species revisited. Considerations on the semiotic turn in cognitive science and biology. SGBWP3. Lund: Lund University.

Williams, K.D. (2013). The effects of video game controls on hostility, identification and presence. Mass Communication and Society, 16(1), 26–48.

Wirth, W., Hartmann, T., Böckling, S., Vorderer, P., Klimmt, C., Schramm, H. Saari, T., Laarni, J. Ravaja, N., Ribeiro Gouvela, F., Biocca, F., Sacau, A., Jäncke, L., Baumgartner, T. and Jäncke, P. (2007). A process model of the formation of spatial presence experiences. Media Psychology, 9(3), 493­–525.

Zlatev, J. (2018). Meaning making from life to language: The Semiotic Hierarchy in the light of phenomenology. Cognitive Semiotics 11(1).

Ludography

Bungie. (2001). Halo: Combat Evolved [Xbox], USA: Microsoft Game Studies.

Capcom. (1987). Mega Man. [NES], Japan: Capcom.

Capcom. (1991). Street Fighter II. [Arcade], USA: Capcom.

Epic Games/Digital Extremes. (1999). Unreal Tournament. [PC], USA: GT Interactive.

Harmonix (2005). Guitar Hero. [PlayStation 2], USA: RedOctane.

KCEJ (1998). Metal Gear Solid. [PlayStation], Japan: Konami.

Koei. (2010). Championship Jockey: G1 Jockey & Gallop Racer. [PlayStation 3], Japan: Tecmo Koei.

Konami (1988). Track & Field II. [NES], Japan: Konami.

Nintendo EAD Group No 3. (2011). The Legend of Zelda: Skyward Sword. [Nintendo Wii], Japan: Nintendo.

Nintendo EPD/Platinum Games. (2016). Starfox Zero. [Wii U], Japan: Nintendo.

Nintendo R&D4. (1985). Super Mario Bros. [NES], Japan: Nintendo.

Rare. (1997) Goldeneye 007. [Nintedo 64], USA: Nintendo.

SimBin. (2005). GTR – Fia GT Racing Game. [PC], USA: Atari.

Ubisoft Paris. (2006). Red Steel. [Nintendo Wii], USA: Ubisoft.