I originally picked the book up a couple years ago, but it languished until recently. Staring at me one morning I thought I’d check it for a contemporary views on Lakoff and Johnson’s ideas of metaphor. Sure enough there was a chapter on it inside that reaffirmed the key points of what was described 20 years prior. As I revisited his ideas though I caught pieces of more contemporary thinking that elaborates on our metaphoric constructions.

Pinker’s discussion of spatial and temporal metaphoric structures are where I found this book most interesting. He describes how, regardless of ontologies be them physical, social, or capitalistic, we understand abstract relationships in terms of potentials for causality. The potential of influence (or may I say the potential for manipulation) is analogous to spatial proximity. Even more interesting is how we cognitively parse physical and continuous spatial relationships intro discreet states based on the functional or causal relationships of the entities involved. For example, being ‘under’ an umbrella is as much or more a statement about being able to stay dry as it is about the abstract positions of the elements involved.

[As an aside, and I may be out of my depth here, the conversion of a continuous variable into a discreet value of causal meaning seems to be some very basic form of 'thinking'. McCullough describes intelligent systems as those that respond to input in deliberate and varied ways. This is useful for categorization of existing artifacts, but less so for thinking through the creation of them. Perhaps it wold be more helpful to say that 'interactive' systems convert continuous input into discreet output, the variety of input and the the complexity of the conversion reflecting the general complexity of the system. This would give a starting point for those learning how to design such systems, a small reoccurring problem to understand. A mapping problem really.]

The verified importance of causality in our thinking, not to mention Pinker’s particular interest in verbs, makes my apparent fixation on designed manipulation less discouraging. (Perhaps it’s fitting that both Pinker and Miyamoto both are interested in verbs; Pinker finds in them the clues for how we interpret the world, Miyamato uses them to define how we play with it). Reading about the eternally present, if even implicit, agent in statements of action struck me in how similar to manipulation if painted verbs in general. The content is better than heartening though. Pinker presents research by Len Talmy, a linguist whose studies into causal perception have exposed a mental model of force dynamics. His studies look at our perception of action and effect in terms of a set of common mental configurations that concepts like prevention and helping map to. Finally, Pinker discusses our universally shared sensitivity to chains of action and intention exist even when end consequences are identical, and plays a major role in our understanding of ethical actions.

One last take away from How The Mind Works was a few related low level metaphoric structures. In my discussions about manipulation I always hit on the problem of metaphor definition. To manipulate something observable in screen space, it must first be described (and modeled). The limitless ways we would could look at something are exceptionally problematic though. What would be useful would be a limited but powerful set of categories to study. “Metaphors We Live By” showed how a great deal of thinking grows from ideas like Object, Container, and Surface. “How the Mind works” introduced me to some interesting elaborations on some of these constructs. Particularly interesting were the relationships between entity, multiples, and aggregates. Pinker shows how entities are defined by their spatial bounds, but can also be viewed, in a sort of figure ground shift, as an aggregate or media, whose shape is ephemeral and temporary. This shift can be used in language for effect in such statements as ‘There was cat all over the highway.’ (which dismisses the structural qualities that regularly define a the unfortunate creature). Also interesting is the distinction between aggregates and a large collections of similar but individual entities. While this distinction is universal, the placement of the dividing line is not. English treats hair and grass as aggregates but other languages describe them as multiples. i.e.. One might comb their hairs. I like these structures as they provide a cohesive set to select from when choosing a creating a model for something visual. A simple choice with implications for potential forms of manipulation.

If you’re familiar with contemporary ideas of language and mind you might not be interested in reading the entire book, but I’d recommend at least Chapter 4, “Cleaving The Air”. Chapter 5, “The Metaphor Metaphor” could also provide a brief but good alternate to “Metaphors we Live By”.

Last quarter I tried to pass on some more atomistic problems that often sit in between code and concept.  the first and one of the lowest level problems I’ve found is the mapping problem, the conversion of one value to another. The second is on the upper end and the first step for breaking apart a concept: splitting application logic into data and representation. (The model / view terms are of course yanked from the MVC pattern and are usually used to refer to something a little more complex that what I’m dealing with here. I use them in order to paint what is often a continuous set of code as smaller pieces with specific responsibilities).

Mapping
In any kind of interactive physical whatever one of the most common tasks, and place for designing the system is in ʻmappingʼ the range of input values to a range of output values. Ie. your sensor gives you 0–1023 but your rotating doll head goes from 0–2*PI radians (yuck). This is a mapping problem, and luckily, with a little math itʼs not that hard to solve.

(inputValue / maximumInputValue) * maximumOutputValue;

or, to use the example above:

(sensorValue/1023) * TWO_PI

What this code does is convert the sensor input into a percentage and then turn that percentage into an actual value in the target space. (sensorValue/1023) will turn our input into a number between 0 and 1 (0 if sensorValue is 0 and 1 if sensor value is 1023). Then we take the value of the target space and multiply by the variable. If our doll head turns to TWO_PI radians and we multiple that maximum value by a percentage, we get the actual number we want the head turned to.

Model / View
Interactive systems are kinda muddy. If weʼre going to design them we need to be able to describe them.

The first step (and easiest and most useful way to do this) is to describe what variables are in the system. A stripped down version of Pong with nothing but the ball needs variables for the x & y position of the ball and also the ballʼs x and y velocity.

Next we need the rules by which the system operates. Always start with the most basic version of a rule that you can actually program. We know in Pong that the ball bounces around, but even simpler than that is the fact the ball moves at all. Movement is a very common rule and is described in computer terms as such:

value = value + valueVelocity;

Or

xPosition = xPosition + xVelocity;

This code needs to be run continuously (every time the program updates). This is done by putting it in the the draw() function.

With a few variables and some math we now have a working model of our simple, non interactive, non winnable version of Pong. Problem is, we canʼt see it work.

The second step in describing an interactive piece is to describe how it represents, or visualizes, its model. This section of a program can be labeled the programʼs view. The view for our Pong game consists of drawing a rectangle to the screen based on the x & y values of the ball. ie:

rect(ballX,ballY, 10, 10);

This code also needs to be called every time the program loops. Just be careful not to mix it up with the model code. Theoretically you could put all your model processes in a function updateModel() and all your visualization code in a function updateView() and then your draw function would look like this:
void draw()
{
updateModel();
updateView();
}


(It would be more helpful if the draw function were just called mainLoop, or main, or loop, but oh well).

Now that we have this basic two part structure we can modify the artifacts gestalt by building more complex models or by building more sophisticated visualizations of the model as is.
What if you had two balls? What if the ballʼs x and y position is greater than the screen size? These are questions about the gameʼs model.

Looking at the speed and position of the ball, can we calculate and draw where it will be in a few moments? This is a view question.

As a designer you can work with either component, evaluate the gestalt, and then try something else. It also makes your program easier to understand if the responsibilities of each line of code are grouped together.

Lastly, thereʼs no rules that govern what the relative size (in terms of lines of code) and complexity (in terms of processing and math) of the model or the view is or should be. Your model could be one variable (meanness) and your view could do a lot of number crunching to draw a flower based on that one variable (like last project). Alternately your model could describe all the pieces of a flower, and the view just has to draw rectangles where the petals are.

“RPGs are many things, but they are almost never hard. As I realized in childhood, the vast majority of RPG challenges can be defeated simply by putting in time. RPGs reward patience, not skill. Almost never is the player required to work hard…Be aware of why you play the games you do the way you do. Be aware of how you use them. We humans are remarkably adept at finding ways to lie to ourselves, and ways to be self-destructive.”

“Games are just a minor symptom of a systemic disease:”

“Games fast becoming standard are the “followers” and “friends” games for example.”

“Why it’s more important than ever to get your mind right about what you’re doing with your life and why you’re doing it…. If you like these things because they entertain you and relax you, fine, more power to you. I have a 110″ inch screen in my media room that I play games on a couple hours a week, because I think they are fun. But don’t delude yourself: they are bullshit…Go for a walk.”

“What it really boils down to is that I traded one set of “achievement treadmills” (i.e. blogging and contributing to an Open Source project) for another more explicit set.”

Indirectly back to “life is a game”. Is there not something more meaningful beyond this or that activity? What is the ultimate meaningful activity? Is it all just a treadmill?

“Does it mean that these products [games like Modern Warfare 2 that soak up hundreds of hours of people's lives] aren’t making their users lives “better”? Well, it depends on how you define better.”

In the Invisible Computer by Donald Norman he presents an illustration that shows chairs created by several related but distinct disciplines. Each chair, sharing the same impetus for its creation, is indelibly marked by the frame from which its creators work. The manufacture’s chair is inexpensive in its construction, the engineer’s rigorous in its stability, and the designer’s chair is novel in its treatment of form. The basic artifactuality of the chair is consistent between each; an instrumental object for human sitting, and of course each design supports this. The conceptual metaphors which are brought to bare in elaborating on this description, however, are not.

Whether through the lens of beauty, communication, safety, or marketability, each discipline has its specialized metaphor(s) for describing artifacts and how they fit into one of a plurality of world views. An artifact is extra dimensional in this sense. It exists in a universe defined by the physicality of the human body and its biological sensorial abilities. It inhabits a universe where the existence of an item is predicated on a person’s desire and ability to purchase it. It also exist in a universe where continual existence often requires repeated sacrifice of resources. Making design decisions that effect an artifact’s qualities in any of these spaces requires a body of specialized knowledge about how those spaces function. This knowledge is generally consolidated into specific disciplines like marketing, engineering, manufacturing, or industrial design.[1]

•IxD is a discipline concerned with the creation of artifacts as CULTURAL OBJECTS.

Interaction design is one such emerging discipline; It’s universe is one of culture. Specifically, a culture defined by the activities and behaviors of its individuals and the artifacts that they interact with and through. Anthropology, sociology, and psychology are important areas of knowledge being quickly absorbed by this nascent discipline.[2] The metaphor that IxD uses is that of ARTIFACT as CULTURAL OBJECT. Furthermore, culture is viewed in terms of the activities and behaviors of the individuals that exist within it.[3]

This new field is predicated on Marshall McCluhan’s understanding that “We shape our tools, and then our tools shape us”.[4] As the nature of a culture’s artifacts and technology can have profound impacts on the rest of that culture, it behooves for us to understand these relationships and perhaps ethically required for us to do so. As Jon Kolko states, “Your work will change culture. Do work that is worth doing.”

•IxD is generally concerned with the design of “interactive” artifacts, particularly computational ones.

Of our technologies, few are having more profound an impact on society than those that include complex computational abilities. Any cultural artifact can influence behavior, but current computational technologies have capabilities of such magnitude that we are unable to ignore them. This importance is recognized outside of interaction design even, having engendered such other fields such as software studies and game studies.

While the cultural effects of these artifacts make them ripe subjects for interaction  design, an artifact’s particular interactive qualities are also important. Interactive artifacts are those defined as having the ability to sense contextual variables, process the information, and respond in some variable and deliberate fashion.[5] An interactive artifact’s peculiar ability to respond to changes in its context, or changes in its user’s behavior, makes it possible, if not necessary, to design for these changes.

Outside of interactive artifacts, interaction design may still exist, but limited ability to create artifacts that respond to behaviors might curtail the usefulness of its framing. Malcom McCullough, in using architectural principles to to argue for situated computing makes such a convincing argument that architecture itself might be viewed as an example of what pre-interactive interaction design might look like. Regardless, designing within a cultural framework is a necessary aspect of interaction design, complex interactivity is incidental.

•Behavior is not a medium. IxD is not a “Design” discipline.

An argument may be made that even if cultural framing is a primary requirement interactivity is still necessary in that it defines the medium interaction designers work in. The claim put forth by Robert Fabricant is that behavior — specifically the behavior of the user — is the ‘medium’ of interaction design.[6] I find this idea presumptuous at best. It implies that the behavior of a user is directly manipulable by the designer. Describing a person’s behavior as a material to be crafted is, by definition, to objectify said person. This argument applies equally to those, like Nathan Shedroff, who qualify interaction design in part as the “design of experiences”.[7] A stance refuted by both Shelly Evanson [8] and Jon Kolko [9].

While I seriously doubt anyone’s intention is to exert absolute control of a human being [10] I think its valuable to make the distinction between designing-to-influence a person’s behavior vs. designing a person’s behavior. Making this distinction allows us to see interaction design as a discipline closely related to study and practice of rhetoric, a potentially valuable insight, but a topic beyond the scope of this paper.

Another position acknowledges that the affection of a person’s behavior is an end separate from means, and states more specifically that the behavior of an artifact is the ‘medium’ an interaction designer manipulates. This position also holds that the tangible material with which this behavior manifests is irrelevant to the interaction designer, that interaction design is a trans-media discipline (or perhaps sans-media).

Contradicting this view are the many examples of “interaction design” that do not exhibit behavioral qualities.[11] Aside from practical arguments, the ‘behavior is medium’ view also compromises the stability of a theoretical definition of interaction design.

Design is, among other things, iterative. This practice requires the testing of potential solutions. It stems from the oftentimes correct assumption that the first solution thought up is generally not the best, that the gestalts of a solution are not known until it is put into context, that the solution can not be mentally modeled by a designer in its entirety, and that the exploration of potentials oftentimes results in new insights into the problem space. The result is a set of disciplines centered on the successive creation of prototypes leading up to a finished solution. In contrast, the standard deliverables of interaction design (wireframes, narratives, or visualizations) constitute only the first steps of an iterative process, and while they may lead to the creation of an artifact in the hands of someone else, these pieces do not constitute an iterative process. Furthermore, iterating from one project release to the next while useful and necessary, is not the type of iteration I discuss here. Any traditional designer who offered a rough solution to a client as final with the intent to “see how things go” and improve it “next time around” would hardly qualify for the title designer.

The conflict here is between the focus on creating artifacts alongside an assertion that interaction design is essentially immaterial. A definition that includes these conflicting stances states that IxD is responsible for the creation of an artifactual solution and simultaneously relieves it of the responsibility of actually creating a finished solution. All artifacts exist in a cultural space. All artifacts also inhabit a material space. An industrial designer who makes an artifact without the relevant training or skills in anthropology or sociology will make a cultural object, however mediocre it may be. An interaction designer, as defined so far, requires assistance in order to create an artifact at all.

Let me make a few things clear. The conversational gestalts of complex interactivity are unique to forms of interactivity beyond that of simple operation and are the result of design decisions. I think a focus on the social context is relavent and useful. I also do not think that capital “D” Design is concerned exclusivly with aesthetics or the design of materials. I do think that Interaction Design is a discipline. I do think that a Design discipline requires faculty within the material context. I parse words to clarify and understand.

Assuming the defining quality of an artifact is thing-ness, any cultural spaces it inhabits are beyond that of its intrinsic properties. The cultural space that artifacts inhabit, the context IxD is concerned with, constitutes a ‘next highest context’. As such the discipline of interaction design may be viewed as an advancement built on traditional design studies.

Alternately, If IxD is to be considered a Design discipline separate from traditional disciplines it will require a defining set of intrinsic artifactual properties separate from those of traditional design disciplines, and a working knowledge of how those qualities can be manipulated for different aesthetic results.

IxD is young if nothing else, so there’s nothing to say that continual investigation of interactivity won’t coalesce into a set of phenomena unaddressed by other disciplines. Until such a day it’s situation will remain nebulous, and risks irrelevance as experienced designers in traditional media, looking for more interesting, complex, and challenging contexts, begin solving problems explicitly defined by a cultural space interaction design is centered within.

In 1980 George Lakoff and Gregg Johnson published Metaphors We Live By. The book made the case that most of our thinking is largely metaphorical in nature; that we directly apply our understanding of the less abstract to the more abstract. Their description of metaphoric language as reflective of a thinking mind irrevocably situated in a physical context helped ignite an explosion of research in numerous fields including linguistics, psychology, philosophy, law, art, and politics. While there’s been much written since then, this work has been repeated referenced in other current texts that I highly value.

The edition I read contained an afterword written in 2003 that provided a great summation of the work, empirical support somewhat lacking in the original and some revisions and their thinking. If pressed for time I would recommend reading these 30 or so pages to the exclusion of the rest of the text.

Most importantly Metaphors We Live By validates my solution of using manipulation as a starting point for the design of interactivity and provides logical arguments against feedback that I  previously could only disagree with on intuitive grounds. In my review (which I have yet to talk about, but which I hope to do at some point in the next few weeks), a midst concerns about the scope of my research, I was questioned on the type of manipulation I was going to address “Manipulation is very broad; there’s all kinds of manipulation, semiotic, physical, interpersonal…” (paraphrased). In concern over the “unfocused” quality of my proposal itself and during an attempt to guide me to “just do a nice project” it was stated that if I was going to find some fundamental principles “… I would have found them by now”. On the first concern, I simply acknowledge the potential pitfalls and stated that I’d be focusing on the most basic form of manipulation, the equivalence between an intentional quantifiable value and that of a screen based entity (such as a box’s rotation to the position of a pointing device). On the second statement I simply bit my tongue. Having read Lakoff & Johnson I would now answer differently; There is not “many kinds of” manipulation. There is only one kind of manipulation which is conceptually mapped to many different experiential phenomena so as to highlight common qualities. These commonalities stem from our physical/spatial understanding of manipulation as a “base metaphor” concerning the movement of an object in space on behalf of an intent. My investigation of manipulation qua manipulation is still modest enough for the project, but more applicable to more complex systems than previously stated. Secondly, our understanding of what we experience and the metaphoric ontologies we use to understand abstract phenomena — such as complex interactivity —don’t get any more fundamental than those based on our body and the physical space it inhabits.

While Lakoff’s research is undoubtedly important, he’s been criticized as delivering his ideas more rhetorically than not and being remise in acknowledging others who have tread on similar, if not the same ground. Metaphors We Live By makes lengthy arguments against forms of objectivism and subjectivism that I’m not certain are necessarily reflective of philosophical thought at the time, and left me want of understanding the context within which the book was released into. The ideas of embodied thinking that seem to stem from their research into metaphor are highly reflective of thinking phenomenological thinking which was hardly underground at the time. At one point they do make reference to this area, but only as it exists, in their words, as “café phenomenology”. Their discussion limited to bullet points and bereft of topical consideration. One critical reviewer on Amazon.com points to Maurice Merleau-Ponty as providing much more nascent thinking on the issues that Lakoff and Johnson have waded into, going so far as to claim Lakoff and John’s most important ideas are ripped of from him. As Paul Dourish’s discussion of embodied thinking also references strongly Merleau-Ponty, it doesn’t seem impossible that there’s some truth there. Furthermore, I’ve found their most interesting ideas such as the reuniting of thinking and physical embodiment and how this manifests in a sort of layered understanding of the world echos those of Rudolf Arnheim whose book Visual Thinking I’ll be writing about in my next post.

Not enough time and attention are given to the designer’s first responsibility: to find and develop the visual solutions for living in our environment… Our goal is the training of a designer so familiar with the principles of abstraction that he automatically thinks of a visual problem in terms of organized relationships the feels free to study other aspects of the problem or to confer with specialists i related fields.

-Rowena Reed Kostello