The Mind’s Eye

In 1967 Russian psychologist Alfred Yarbus published a ground-breaking text on visual perception. Yarbus had invented and adapted equipment to study the function of saccades as well as eye tracking during a wide variety of “looking” experiences. He noted that when examining complex objects, the eye tracks through and fixates on some but not all of the object or scene. He also noted that special attention is not particularly perceptual or feature-related. In other words, special fixation is not correlated with the amount of detail in a particular object, the brightness or contrast, or other elements such as outline  (Yarbus, 1967, p. 190).

When Yarbus studied eye fixation and tracking patterns, he found the observer’s attention is held by picture elements that provide information that helps the observer make meaning. Eye movements, he concluded, reflect the human thought process (Yarbus, 1967, p. 190). Elements that provide meaning, such as the eyes and mouth of a person will be scanned repeatedly, while other elements will receive little or no attention.

Taking this idea a step further, Yarbus decided to record the eye movements of subjects while looking at the same picture but under different instructions. Yarbus concluded: “Depending on the task in which a person is engaged, the distribution of points of fixation on an object will vary correspondingly” (Yarbus, 1967, p. 192). –Kind of like a “Whorf’s hypothesis for visual attention” isn’t it? See examples from The Unexpected Visitor, below:

As you can see, tracking patterns varied accodring to the purpose of the viewer. Over 40 years later, Fred Volkmar and Ami Klin at the Yale Child Study Center decided to take Yarbus’ ideas a step further. Using modern eye-tracking equipment, they have been studying the eye movements of autistic children and adults compared to controls to get a window into the social mind of people with this disability. One of the things that intrigues this group is the large discrepancy between many autistic individuals’ ability to learn specific social skills and the ability to apply those skills in context. In one study, the Team at Yale decided to track the eye movements of autistic individuals and controls while watching the highly emotionally charged film, Who’s Afraid of Virginia Woolf? As it turns out, individuals with autism focused twice as much on the mouth region of the actors as controls, and controls focused on the eyes. In another example, controls immediately tracked a pointing gesture to the object of attention, while autistic individuals missed the nonverbal cue and resorted to “fishing about” with scanning patterns in order to make some meaning of the exchange. In yet another highly charged scene, one subject with autism fixated on a light switch, giving the object the same level of attention that controls would give to a face or a social exchange.

In order to process social situations and interpret the multiple simultaneous events involved, we must first pay attention. Joint attention, in fact, is essential for checking the reaction(s) of our communicative partner against our own. Joint attention readies us for action (Brunik & Gardenfors, 2003; Locke, 2007), allowing us to mimic and thereby understand the purposive actions made by others (Fogassi & Ferrari, 2004). In other words, each partner has some awareness that the attention and action are shared. Peacocke (2004) calls this “mutual open-ended perceptual availability”.

Scientists have identified neurons that likely gave rise to the mimicking behaviors that are considered foundational for human communication. These “mirror neurons” are brain cells that are activated, triggering the motor cortex, when an animal (or person) performs an action that it observes performed by another animal (Locke, 2007; Kan et. al., 2006; Fogassi & Ferrari, 2004).  Interestingly, children with autism have significant difficulty with both joint attention and imitation. This manifests itself in difficulty not only understanding a communicative partner’s intent (ie “theory of mind”), but in significantly compromised communication systems in general. Courchesne (in Sacks, 1985) hypothesized that children with autism have difficulty tracking and attending to changes in their surroundings, and he noted that differences in the cerebellum, which regulates and coordinates the timing and sequencing of motor movements, could be implicated in the disorder.

Does coordinated motor movement underlie vocal communication? The idea that gestural communication preceded oral communication is well accepted ( Fogassi & Ferrari, 2004; Locke, 2007).  If we even think  about moving the hands or look at  pictures of tools, this activates the motor cortex and Broca’s area ( Lock, 2007). The inability of the Yale subjects to track quickly, follow gestures, and focus attention on the salient elements of  scene certainly seem to support this hypothesis.

There are many more ongoing studies at the Yale Center. For example, the Yale team is also using light-point technology to see whether people with autism personify point maps of people when they are in motion. This is the same technology used by movie studios to realistically animate characters played by actors by providing them with a new computer-generated “skin”.  They are even looking at computerized social skills training methods and robot training coaches. Below you will find a link to the Yale Child Study Center.

I think this study reflects Kuhn’s ideas, because the Yale team has applied a revolutionary idea from the 1960’s to a new construct. We don’t always think of quantitative research as “creative”.

Link to Article

Yarbus, A. (1967). Eye Movements and Vision, Plenum Press, New York.

Fogassi, L., & Ferrari, F.. (2004). Mirror neurons, gestures, and language evolution. Interaction Studies, 5(3), 345-363.

Hopkins, W. D., Russell, J. L., Cantalupo, C. (2007). Neuroanatomical correlates of handedness for tool use in chimpanzees (Pan Troglodytes). Psychological Science, 18(11), 971-977.

Locke, J. (2007). Bimodal signaling in infancy. Interactive Studies, 8(1), 159-175.

Peacocke, C. (2004). Joint attention: Its nature,  reflexivity, and relation to common knowledge. EILA, 298-324.

Sacks, O. (1985). The Man Who Mistook his Wife for a Hat, Touchstone, New York.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s