Chris Dent

L505 Essay 9

2001-03-05

Mental Models and HCI

One obvious reason why scientific laws do not seem to have a place in natural language deduction is that many people do not have them. It is also clear that many do not know about scientific models. But this does not mean that ordinary people do not use models and laws to make deductions; it is just that the laws they use are often proto laws. Proto or folk laws enter all aspects of our deductive lives, ranging from folk physics, folk psychology, and even to laws in fictional worlds, like in Roadrunner cartoons (Aronson).

Student to his computer language instructor, sometime in the not too distant future: So, teacher, why is it that we get taught to speak like the computer instead of getting the computers to speak like us? I thought the computers could program themselves to do pretty much anything? They model the weather, they control the traffic in the tunnels in Boston, and they trade stocks for my dad; that’s all pretty complicated stuff, yeah? But still you’re standing there trying to teach us how to talk and think like a computer. I don’t get it.

Teacher: Do you really want to know or are you just wasting my time again?

Student: Uh. Hmm. Uh. I really want to know.

Teacher: Okay. The short answer is that computers, while complicated things are not as complicated as you and me. Computers are made by and run with formal sets of rules that can be chained in a linear and provable format.

Student: And we’re not.

Teacher: Right. You and I have brains that are somewhat like computers and while we are able to construct and understand the formal logic that computers use as far as anyone can tell our brains do not follow that formal logic themselves, at least not for all things. Our brains make leaps of faith and intuition, connecting seemingly unrelated concepts and processes to reach unexpected conclusions. That’s what makes people so incredible.

Student: Surely there must be some kind of formal something or other going on there that we could figure out if we were smart enough.

Teacher: Maybe, but thus far we haven’t figured it out. The problem appears to be in the way we conceptualize data and make conclusions. We can model an event in a computer with a very high degree of accuracy but there is always some dimension that we haven’t measured. You, a computer and I can all experience the same two different events and you and I will generally know the two events are different, even if they are quite similar, whereas the computer does not because we notice some ineffable quality that the computer does not have the capacity to sense.

Student: So why does this make us have to take this class?

Teacher: Oh yeah, right, your question. A few years ago human computer interaction researchers split into two main camps. One camp claimed that making interaction with the computer metaphoric would optimize the interaction. In other words making the interface to the computer similar to already familiar devices would ease use. Another camp claimed that while easing use was an important goal it should not supercede maximizing expressiveness and that metaphors actually got in the way of a verbose expressive language of interaction. This second camp believed that metaphors were good learning tools for beginners but built bad habits.

Student: So by metaphor you mean something like the desktop stuff we used in first grade?

Teacher: Yes. Now that you are familiar with the basic interaction with a computer this class is here to teach you to be more expressive. The basic theory is as follows: You, as a human, are a creature with a flexible and intuitive conception of reality. You have the fundaments of language-based expression in your head. It is a part of your nature. The computer is a construct. The only fundamentals it has are the ones we have given it and it happens that the ones we have given it can be captured in a very small number of extremely simple mathematical concepts. Remarkably those basic concepts can be used to model and manipulate all sorts of data…

Student: Except for us.

Teacher: …but they can’t make the conclusions we can make. In this class what we do is teach you a language that you and the computer can share. The computer has been taught this language and now, in here, we teach you the language. In some ways it is just like your Spanish or Chinese classes. In this class and those you learn the vocabulary and rules of grammar that define the language. As you gain experience and comfort with all three of the languages your ability to communicate complicated ideas, processes and instructions grows.

Student: But this computer language isn’t anything like Spanish or Chinese. It’s sort of like English but it is all rough and choppy. There’s no poetry in it.

Teacher: There are three reasons for that: You can’t hear any poetry in it because you have no experience with it. When you have a few years under your belt you’ll construct queries and make instructions that you and others will think are elegant. Also, it sounds like it does because that makes it easy for the computer to understand us without any ambiguity. We need to make it easy for the computer because the computer has no tolerance for ambiguity. A computer neither guesses nor hedges nor makes tentative interpretations, not in any real sense. It simply proceeds, in a linear fashion. Making it sound like English is an effort to make the language more accessible to speech interfaces. As we work with it more you’ll see that English provides a backbone but much of the grammar for aggregating data is simply made up. Does all that make sense?

Student: I think so.

Teacher: Tell me.

Student: Okay. We want to be able to work with computers in a precise fashion that maximizes our own ability to conclude as well as the computer’s ability to calculate. When we think we think in ways that cannot be modeled in any completely accurate fashion. We make conclusions that do not follow a demonstrable formal logic but are still accurate. Computers, on the other hand, do their thinking solely based on formal logic. It is possible to lay metaphors over the formal logic which attempt to interpret what we want and what the computer can do in a way that creates a common middle ground but these metaphors are stifling for both the user and the computer; expression is limited: compare a two year old and a forty year old describing a tree. Efforts to teach computers to communicate in a natural language in both directions have failed because we are unable to find an accurate model to represent how our brains work. On top of that computers do not deal well with ambiguity. So, because we are the more flexible and adroit members of the human-computer pair we have chosen to communicate in a way that the computer can understand. Instead of making the computer do more work to communicate with us we let the computer do what it is good at, which is calculate and process data, and we do what we are good at, which is communicate. It’s a fair trade. Happily the division of labor is productive for us: the computer presents us with additional perspectives to further our unfathomable conclusion making process.

Teacher: Ah, hmmm, nicely summed. You get the gold star for today…

REFERENCES

Aronson, Jerrold, "Mental Models and Deduction", American Behavioral Scientist, V.40, Issue 6, pp. 782-797.

Johnson-Laird, Philip N. (1991). Mental Models. In Posner, M. I. Foundations of cognitive science. Cambridge, MA: MIT Press. p. 469-499.