Interactive Design and Game Development MFA thesis research

This past weekend I finished Where the Action Is by Paul Dourish. It’s the book I didn’t know I needed to read years ago. Back then I spent a bit of time in meetings drawing grids of small circles and thinking “What IS a button?”. In Action Dourish tours through sociology, psychology, phillosophy, and HCI (among other areas) to support at a set of interaction principles that echo my own intuitive thoughts, but fully resolved. For example, his use of phenomenology to describe how we can move from action on an object to action through an object puts the table under my undergrad ideas of  ”token chains” and “token switching” that I built from Costikyan’s token idea. Dourish offers arguments that could nullify nagging questions I’ve about interfaces and representation; for example, if the interface is roughly defined as the system with which we interact to effect change in a removed system, and we can move between interacting with a something and interacting through it simply by change our mental frame, then isn’t the interfaceness of something supremely subjective? If it is, does that mean we should design to facilitate the exposure of  a system rather than hiding it? Dourish not only argues that there is no distinction between representation and implementation, but explains why the distinction exists in the first place (age old phillisophical splitting of mind and body). It’s all still sinking in (I’m just now writing this after finishing the book over two days go) so I may not be doing a good job of articulating.

His idea of “embodied interaction” and its implications cover two thirds of my thesis. I’m now trying to figure out how to best incorporate and/or build on his work; I’m trying to understand what changes this will have on my research plans (am I going to have to dive into ontology and phenomenology???). There’s too many thoughts to deal with at once and I’ll likely come back to this book repeatedly, so I’ll simply end this post here.

Greg Costikyan, in attempting to define the basic elements of a game, describes the ‘token’ as an element in a game system the player has direct control over.

Agency in interactive systems is hardly restricted to one place however. (Press any key to toggle fullscreen).

NOTE: There is a bug in Google Chrome on Mac that prevents hiding the cursor! I HIGHLY recommend viewing it in another browser.

In The Human Interface, Jef Raskin hammers on one particular message; Modes result in mode errors, and must be eliminated wherever possible. Along with this he argues that any gesture must retain the same meaning (or effect) in as many places as possible, but preferably, in all places. This anti-modality view is built on the rather inarguable premise that any digital artifact must first be built with the cognitive limitations of people in mind. From this, as we have only one “locus of attention”, requiring a user to change their locus from the content to the context is both inefficient and prone to cause errors, particularly when the context is deep in the periphery of our attention.

Computer interfaces are still far from “humane” in many cases, so a focus on cognitive ergonomics or cognetics even today is hardly problematic. However, some things strike me as particularly… peculiar.

A multiplicity of layered, changing or even conflicting meanings in human gestures is the rule of our daily lives. It’s at the heart of a ‘passive agressive’ act for example. Us humans have a fantastic ability to move between models of structuring the world. We are also fantastic at misunderstanding the contexts within which actions are meant to be taken within. Computers are no different. It seems their biggest problem is an exaggerated inability to understand the context within which our actions are meant to take place.

The simple solution, of course, is to remove the potential for different contexts. A communicative flat land. This is Raskin’s ideal. No applications, no files, no modes. One context, no ambiguity.

While the elimination of preventable errors is nobel. I’m not convinced that a paradigm for designing interfaces that results in the removal of metaphors is either desirable or even the most efficient . Let alone forcing users into a land of a single context smacks of a bend-the-user-to-the-computer practice that Raskin is himself trying to eliminate in his book.

Perhaps some manipulations are universal, or close to it; Cut and Paste, Copy, Undo and Redo. These actions, or at least the pervasiveness of their use, is due to the reconfigurable qualities of computation as much as the structure of human thought. Outside of them there’s plenty of actions that might have very beneficial contextual interpretations. I don’t want auto indentation when typing a letter, but I do want it when writing code. A parenthesis is an individual character, but if I type it while a line of text is selected, I’d rather it bracket the content than replace it with “(“. We might want windows to be in the same place when waking a computer from sleep, but I doubt everyone wants their browser, on launching, to display the last web page they were on when sitting in a coffee shop.

The discrepancy between a computer’s understanding of context and our own is evident even in the removal of modes. This is highly evident in the social faux-pas on Facebook when new users unintentionally publish, publicly, information meant to be private. Raskin might argue that this stems from the modality created by applications, perhaps applications such as an email client. No matter how ideally ignorant a user should be of the computer’s mode, ignorance on the system’s part of the context of the user will always cause problems.

The issue seems not in designing for a universal context (though this can also help in avoiding bad design choices), but in understanding how people float between contexts and how this can be facilitated by a computer, not ignored.

The idea of computers responding to emotions is briefly mentioned. Raskin quickly sets it aside as a nice thought, as the ability for computers to successfully do this, or for designers to account for it at the time of writing is nill. Fair enough. However, he never addresses the emotions that users have — regardless of whether the computer reads them or not. Raskin talks of designing humane interfaces, polite and respectable interfaces, but his guidelines account for people as if they were machinery.

Jon Kolko offers a variety of insights and interviews on interaction design that far surpass those in my previous read, Designing for Interaction. Kolko manages to describe IxD process as both nebulous and useful. He provides engaging writing by he and other designers that colors business complexities and offers arguments for change wrapped in often lively writing.

In case your bitter black turtle necked heart forgot for a moment, remember —as you dig in your tight jeans’ pocket for enough changet to get your double-mocha-skinny-latte fix: we are designers. We are agents for change.

There are a number of useful footnotes and references that offer background to the views he presents. Overall I found Thoughts to present a lucid look at the use and rough edges of interaction design as a transfigured design discipline shaped from (what should always have been) a concern for people, and the increasing need in business for non-linear problem solving.

Looking deeper, I found some of his views — particularly concerning poetic interactions and design-as-langage — run unknowingly parallel to topics in game studies. Kolko’s take on the rhetoric of design is precious but feels idealic rather than descriptive. The  rhetoric of designed interactions (better described using the tearm procedure ala Ian Bogost), is treated far better by Bogost in Persuasive Games. Of course Bogost did have the luxory of an entire book.

The idea of “design is language” strikes me as practically sideswiping, blindly, Greg Costikyan’s description of the endogenous value systems found in games (see “I Have No Words and I Must Design“). Notably different from Kolko is Costikyan’s emphasis on the independence that game values have from their outside context. Kolko’s view is far more concerned with how the ‘designed language’ interacts, or makes use of, existing frames of reference. While I think Kolko uses language in a weak sense, that designing interaction requires the design of some structured set of actions-with-meaning would be hard to dispute. The implications of this idea are slighted in favor of Kolko’s interest in clearly situating emotional content as a valuable aspect of a designed interaction.

The interaction designer shapes culture directly though the creation of new visual form language. This semantic view of design—that objects are embedded with more than just functional signifigance—rejects the platitude of From Follows Function and instead recognizes the need for emotional and social connections in the human-made world…

While valid and important. I think the premis itself is more interesting than what it supports. From what I understand, language (strongly defined) has a particular set of qualities that differentiate it from the general category of communication. Notably, being composed of discreet recombinatorial parts and a natural inception. I doubt that any and all design, even design of reconfigurable systems, count as the design of language. What I think is interesting about language (among other things) is how a very abstract formal system with very superficial and arbitrary meaning in its parts can be used to create statements of very profound meaning. If designing interaction is the designing of language, it’s not so much the design of meaning so much as the design of systems that support the creation of meaning. He continues:

It is interesting to consider the implications of a design that allows regular people—people who don’t claim to be artists and may rarely get a chance to create much of anything at all—to be creative…

Exciting indeed. I believe this very idea motivates designers of toys and games. The idea of meaningful gameplay formed out of a base set of prescribed possibilities afforded by a system is typically discussed under the auspice of the term ‘emergent gameplay’ in such books as The Art of Game Design. Emergence is generally regarded as a positive quality as it supports the creation of sophisticated, meaning-rich, actions (or interactions).

If interaction design, or even design, “is language”, it seems there’s more to be said than that it supports poetic expression.

“Interaction Design’s Early Formal Education & Beyond”, a recent article by David Malouf, touches on an issue almost central to my thesis; the education of designers, specifically designers of interaction. While I focus on the issues screen designers face when studying interactivity, he address views about interaction design that hold “…the belief in the separation between form and interaction.” and that “This myth can no longer be maintained – definitely not in education.”

I couldn’t agree more. Though, I’ve never felt the need to make such a statement. I’m exceptionally interested in interaction, but early on I decided to restrict my studies to those with screens; an area with its own idiosyncrasies (like any design discipline), and an area I was familiar with. I always figured that any attempt to find and articulate fundamental interaction principles outside of the screen (with physical forms) would at minimum require a breadth of knowledge far removed and beyond from my expertise. I also felt that any ‘universal’ ideas of interaction without  some kind of embodiment in context would be relatively useless for students who require small specific problems in order to refine understanding, sensitivity, craft, and their ability for critical analysis.

Coincidentally, in the Arnheim I’ve been reading he argues at length about artificial separations between perception and thinking. In his work he explains the importance of the cognitive functions implicit in absorbing the world around us and how strong the ties are between perception and ‘higher level’ thinking; thinking which has traditionally been viewed as being separate from the forms that surround us, when in actuality it’s largely built out of it. Even while it’s valuable to abstract phenomena from context, it’s folly to disregard the existing or potential effects specific contexts have on those phenomena.

It is the relentless attachment to the world of the senses from which great ideas take flight.

-Rudolf Arnheim