Abstract
This study explores the synthesis of gesture, speech, and a concept map as these are used for summarizing academic text by an adult non-native speaker of English who was an MATEFL (Master of Arts Teaching English as a Foreign Language) student at a university in northern Thailand. Specifically investigated is the spontaneous development of gestural signs that acquire symbolic meaning over short periods of time (i.e., microgenesis), and how these gestural signs are related to the concept map, speech, and cognition. Results demonstrate the effectiveness of using a developmental approach to the data and the linguistic categories of figure, ground, path, and manner to infer features of cognition. Findings have implications for using gesture for assessing language proficiency and isolating specific types of difficulties non-native speakers may have with the English language (e.g., lexical, grammatical). Data illustrate how gesture works with other mediational means (i.e., concept maps) to support communication.
Key Words: gesture; microgenesis; graphic organizers; Vygotsky; private speech; thinking for speaking
Introduction
Since the late seventies, graphic organizers, also known as key visuals in some of the research (Mohan, 1986), have been used to integrate language learning and content knowledge in mainstream, English-dominant curriculum. Following success in mainstream curriculum, graphic organizers became a part of ESL and EFL curriculum (English as a Second Language; English as a Foreign Language) (Peregoy & Boyle, 2001; Robinson, 1998). Common examples of graphic organizers include concept maps, Venn diagrams, matrices, tree diagrams, information flow-charts, and KWL charts (Merkley & Jeffries, 2000). For this study, a concept map was the most prominent graphic organizer (see Figure 1). A concept map prompts learners to write words and phrases in circles, ovals, or other shapes. These shapes are then connected with linking words and lines, thus reflecting a meaningful, visual/semantic arrangement of vocabulary and concepts (Novak & Gowin, 1984).
The focus of this paper is on the interrelated, supportive features of a concept map, gesture, and speech (in English) of Chou, a native speaker of Chinese and teacher of English from southern Yunnan, People’s Republic of China. Data were part of a larger study that took place in a Master of English as a Foreign Language (MATEFL) program at a university in northern Thailand. The most salient features of data were the semiotic contrasts provided by the gestures, oral English language, and the concept map.
Purposes/Objectives of the Study
By using a developmental approach to the data; that is, by focusing on the unfolding synthesis of three representational systems (i.e., gesture, speech, and the concept map) as these are displayed by video and audiotapes, the genesis of human activity with objects, language, and other interlocutors can be investigated “right before one’s eyes” (Vygotsky, 1978, p. 61; see also Werner, 1978). The data of Chou for this paper also illustrate how a developmental approach to language interactions can be used for identifying specific areas of learner tensions with the target language (i.e., tension between the learner and mediational means such as language and the use of visual displays) (Wertsch, 1998). Moreover, this study demonstrates how students and teachers of language can more clearly identify and understand how they and other interlocutors transform gestures, words, and other media from the immediate surroundings into semiotic resources (Van Leeuween, 2005)
Figure 1: A concept map created and used by Chou as the only reference for an oral summary of an academic text about human senses (see Appendix One for the full transcript). Chou held this concept map in one or both hands as she summarized the article.
See PDF for this
A variety of issues about gesture, language, and cognition addressed in prior research (e.g., Kita, 2000; McNeill, 1992; McNeill & Duncan, 2000; McCafferty, 2004; Stam 2002) were encountered during the present study. These issues include how gesture provides a window into cognition (McNeill, 1992), how gesture is used as a type of thinking for speaking (McNeill & Duncan, 2000), and how gesture can be used to identify specific moments of linguistic tensions and transformations in the speaker’s use of English during a presentation using a concept map.
From these broad issues, the following research questions guided the study:
Question One: How did gesture exemplify figure, ground, path, and manner (i.e., motion events), and using the motion event as a unit of analysis, was their any gestural evidence in a shift in thinking for speaking in the data?
Question Two: How did gesture and the concept map support and/or constrain Chou’s English language and overall content during this oral presentation?
Theoretical Framework
Many of the key points of Vygotsky’s theories are well known. Those most relevant for the present study include mediation, signification; a related term that takes a wider and more semiotic approach to the communicative context: semiotic resources (Van Leeuwan, 2005), and a number of terms related to understanding transformations in thinking for speaking (McNeill, 2000): inner speech, private speech, the psychological predicate, and the notion of Growth Points.
Mediation is generally understood as the use of concrete objects and abstract signs, including gesture, to monitor and/or regulate mental activity (Wertsch, 1998; Wells, 1999). From this perspective, signs and symbol-systems (e.g., mathematical formulas, video games, and language) are crucial to human learning and development (Vygotsky, 1978; 1986; see also Gee, 2003). Foundational to understanding mediation is the process of signification, which is also a central concept in the field of semiotics (Chandler, 2002).
Vygotsky (1978) presented signification as the process of assigning meaning to an object; thus, creating a sign, such as tying a string around one’s finger to remember something. The string is now transformed into a sign that connects the abstract mental world with the world of objects; that is, the string is now a semiotic resource for mediating mental activity, in this example, the activity of remembering.
Semiotic resources can be generally understood as those “actions” and “objects” that gain meaning during social communication (Van Leeuwen, 2005, p. 4). These actions and objects have a “theoretical semiotic potential” and an “actual semiotic potential” (ibid). Theoretical semiotic potential is “constituted by all their [the actions and objects] past uses and all their potential uses” (ibid). The actual semiotic potential is construed by users of the resource and “by such potential uses as might be uncovered by the users on the basis of their specific needs and interests” (ibid) This process of signification; that is, creating semiotic resources, occurs on two dynamically linked planes of development, the interpsychological plane (between people) and the intrapsychological (within a person) (Vygotsky, 1981; Wertsch, 1991, 1998).
Foregrounding Chou’s gesture as it co-occurred with the English language and the concept map provided an interesting glimpse into this signification process, specifically, moments of gesture as private speech; that is, speech for oneself (McCafferty, 1994; Wertsch, 1979), which became an important semiotic resource to complete goal-directed activity. Chou’s goal was to complete a specific literacy activity: the summarization of academic text.
Gesture, Private Speech, Inner Speech
This study drew heavily on gesture research from McNeill (1992), McNeill and Duncan (2000), and McCafferty (1998; 2000; 2004). The term gesture is used in a slightly different way for this study than by McNeill (1992) who cited Kendon’s (1988) parameters on gesture as a continuum. On this continuum, gesticulation occupies the left extreme and sign language occupies the right extreme. McNeill (1992) described gesticulations as "spontaneous and idiosyncratic gestures that occur while one speaks" (p. 36). Gesticulations are in contrast to emblems (e.g., making a culturally specific obscene gesture with the hand) or sign language, which is to the right of gesticulation and emblems on Kendon’s continuum (Kendon, 1988; McNeill, 1992).
Overall, the gestures presented in this paper are slightly different from the gesticulations presented by McNeill (1992), McNeill and Duncan (2000), and McCafferty (1998; 2000; 2004). Chou’s gestures are less spontaneous because they were generated in the context of a pre-planned oral presentation in which a concept map was positioned as a semiotic resource to support communication in the English language. In addition, the majority of the data from McNeill’s and McCafferty’s work were generated by narrative recall, pictures prompting narratives, or conversation. Most important for the present study, the concept map provided a unique influential contrast to oral language and gesture. This contrast highlighted moments of private speech and inner speech.
The term private speech (Wertsch, 1979) is used to distinguish Vygotsky’s (1986) ideas about egocentric speech from those of Piaget. Egocentric speech is the well-known phenomenon of children talking to themselves while they are playing or involved in other problem-solving activity. For Piaget, egocentric speech disappeared as children developed; for Vygotsky, egocentric speech developed into private speech, or speech for oneself, which is important for mediating cognition. In other words, talking to oneself during problem-solving activity becomes important in the context of goal-oriented activity. Closely related to private speech is inner speech.
The three main semantic characteristics of inner speech are "agglutination, the preponderance of sense over meaning, and the influx of sense" (Wertsch, 1979, p.79). A fourth syntactic feature of inner speech is known as the psychological predicate (Wertsch, 1985; 1991). A frequently cited example of the psychological predicate is the occasion of people waiting for a bus to approach. They will not bother to say an entire sentence such as "The bus for which we are waiting is coming" (Vygotsky, 1986, p. 236). They will say "coming" or something similar (Vygotsky, 1986, p. 236); this expresses the changing nature of context and thought (McNeill, 1992).
McNeill (1992) and others (e.g., McCafferty, 1998; McNeill and Duncan, 2000) have used gesture to infer private speech and the psychological predicate. In previous research on gesture, Vygotsky’s notion of inner speech has been linked to private speech and gesture in native and non-native speakers of English, including students of English as a Second Language (Lantolf, 2003; McCafferty 1998, 2002, 2004; McCafferty & Ahmed, 2000).
Relevant Literature
There are many books and journal articles investigating gesture and language from a variety of perspectives. The most relevant articles for this study are those that investigated language and gesture with theoretical frameworks related to the work of Vygotsky. This includes the notion of a growth point, and that gesture is a type of thinking for speaking (McNeill & Duncan, 2002). Though not directly related to Vygotsky’s work, Kita’s (2000) Image Packaging Hypothesis is also reviewed.
Growth Points, Motion Events, and the Relationships between L1s and Gestures
McNeill and Duncan (2000) identified a specific point in the utterance called a growth point (GP) (see also McNeill, 1992). Growth points are comparable to Vygotsky’s psychological predicate described earlier (Wertsch, 1985; 1991). They can be found by focusing on “speech-gesture synchrony and co-expressivity” (McNeill & Duncan, 2000, p. 144). Growth points, as these are expressed through gesture, mark “a significant departure in the immediate context” and “implies this context as a background” (p. 145). McNeill and Duncan (2000) used GPs to infer a type of thinking for speaking, which “refers to how speakers organize their thinking to meet the demands of linguistic encoding on-line during acts of speaking—what Saussure (1959) termed parole rather than langue” (p. 141).
In order to illustrate the relationship of a speaker’s L1 and gesture, McNeill and Duncan (2000) presented examples of the GP in Georgian, English, Spanish, and Chinese through a specific semantic domain, called the motion event (Talmy, 1985; 1991). The motion event is an important concept for the present study and other studies reviewed (e.g., Negueruela, Lantolf, Jordan, Gelabert, 2004; McCafferty, 2004).
Citing Talmy (1985; 1991), and using an example of a cartoon character (Tweety bird) stuffing a bowling ball down and inside a drain pipe in which another cartoon character (Sylvester) is climbing up (also inside), McNeil and Duncan (2000) describe the major components of a motion event: figure, ground, path, and manner. A moving object, called the ‘figure’ as in, “drops it down,” where the “it” indexes the bowling ball, the object in motion; A reference object, called the ‘ground’, as in “drops it down the drain-pipe,” where the downward trajectory occurs in relation to a non-moving object, the drainpipe; A trajectory, or ‘path’, as in “drops it down,” where the bowling ball moves on a downward trajectory; A ‘manner’, as it “it rolls down,” where the motion is performed in a certain way. (McNeil and Duncan, 2000, pp. 148-149).
Another important feature of classifying figure, ground, path, and manner is how different languages present motion events (McNeill & Duncan, 2000). For understanding this feature of language, McNeill and Duncan used Talmy’s notion of verb-framed languages and satellite-framed languages (Talmy, 1985; 1991, as cited in McNeil and Duncan, 2000).
Slobin (2003), provided the following examples in distinguishing French, a verb-framed language, with English, a satellite framed language:
a. The dog went into the house
b. Le chien est entré dans la maison en courant.
"The dog entered the house." (Slobin, 2003, p. 162)
Slobin (2003) argued that “English frames path by means of a satellite (in); French “frames” path by means of a verb (entrer) (emphasis in original, Slobin, 2003, p. 162). The other example that Slobin (2003) provides is how manner of motion is coded in English as compared to French:
a. The dog ran into the house
b. Le chien est entré dans la maison en courant.
“The dog entered the house by running." (Slobin, 2003, p. 162)
In comparing the codibility of path and manner (i.e., motion events) between English and French, Slobin (2003) described path as highly codable for both English and French. He also described manner as highly codable for English and other satellite-framed languages, such as Mandarin Chinese, which is relevant for the present study. However, for verb-framed languages, such as French and Japanese, Slobin (2003) described manner as “an adjunct—an optional addition to a clause that is already complete” (Slobin, 2003, p. 162).
McNeill and Duncan (2000) found gestures for the Chinese speaker to be different than gestures used by English and Spanish speakers. For Chinese, there was a noticeable delay between where the gesture falls and the motion verb to which it refers. McNeill and Duncan (2000) proposed a specific type of thinking for speaking for Chinese because speakers of Chinese frame topics, in contrast to English, which is “founded on subject-predicate relations” (McNeill and Duncan, 2000, p. 153). McNeill and Duncan (2000) borrowed two terms from Li and Thompson (1976, 1981, as cited in McNeill and Duncan, 2000): “topic prominent” for Chinese, and “subject prominent” for English (McNeill and Duncan, 2000, p. 153). “The hallmark for this Chinese pattern is a gesture that occurs earlier in the temporal sequence of speech than the timing facts of English and Spanish would lead us to expect” (ibid, p. 152).
Overall, for a Chinese speaker as contrasted with an English speaker, the following five differences can be expected (four of these differences were gleaned from McNeill and Duncan, 2000; the fifth difference is from McNeill, 1992): 1) gesture does not necessarily co-occur with the verb; 2) a delay between the verb and gesture, sometimes a several-word delay; 3) the gesture of those with English as the L1 coincide more with the grammatical predicate than those with Chinese as the L1; 4) because Chinese has a large number of manner verbs, like English, it is expected that Chinese will also emphasize manner; and, 5) metaphoric gestures known as container metaphors (McNeill, 1992) are uncommon for Chinese speakers. The container metaphor is described as a metaphoric gesture with an idea presented as if in a container, and this container is then given or offered to the listener (McNeill, 1992; McNeill & Duncan, 2000).
Another important finding for the present study from McNeill and Duncan’s (2000) research is the notion of Growth Points as “points of departure from the preceding discourse what Vygotsky called psychological predicates (as opposed to grammatical predicates)” (McNeill and Duncan, 2000, p. 145) (see also McNeill, 1992). Equally important was the way McNeill and Duncan (2000) demonstrated the effectiveness of using the categories figure, ground, path, and manner, to analyze thinking for speaking patterns (see also Negueruela, Lantolf, Jordan, & Gelabert, 2004).
Before proceeding to other studies investigating the GP and related features of speech and gesture, it is necessary to review categories of gesture from McNeill (1992). These categories were adopted for the present study and are relevant to other studies reviewed.
Categories of Gesture
The major categories for the gestures in the present study were iconic, metaphoric, beats, and deictic gestures. Categories were derived from semiotics, specifically the work of C.S. Peirce (Peirce, 1931, cited in McNeill, 1992), one of the founders of semiotics and American pragmatics (Dillon, O’Brien & Heilman, 2000). In addition, the broader category of illustrator (Argyle, 1998) was useful to describe the overall representational role of gestures (see also McCafferty, 2002).
Iconic gestures are very similar to the semantic content of speech. An example would be to describe a church steeple by bringing the hands and fingertips together to form a steeple while saying “steeple” (McNeill, 1992).
Metaphoric gestures create a kind of contained space, as if the speaker were holding or containing something. For example, a speaker might tell you about two sides of an argument for a multi-story parking garage. The speaker would spread her hands slightly above her waist. She would do this on the left side of her body for the pro side of the argument; then move her hands in this gesture space (between the chin and waist) to the right side of her body while she presented con sides of the argument. This is called, splitting the gesture space (McNeill, 1992; see also McCafferty & Ahmed, 2000).
Another type of gesture is beats: "Beats are so named because they look like beating musical time" (McNeill, 1992, p. 15). McNeill (1992) explained that beats are noticeable for their two movements of "in/out, up/down, etc” (ibid, p. 15). Beats have a distinctive pragmatic function such as when the discourse shifts and/or when the speaker wants to emphasize a particular word or phrase (McNeill, 1992). For example, a speaker might say "However, that sentence is convoluted for good reason". You might see the speaker make a short up and down movement with one or both of her hands on "however", “sentence”, and “convoluted”.
Deictic gestures point somewhere. McNeill (1992) provided the example of an abstract deictic gesture of a man sitting down and saying, "Where did you come from before?" (ibid, p. 114). The man points to a space between himself and the interlocutor. For the present study, a distinction between abstract and concrete deictic gestures is made. If the gesture refers to the immediate context or closely refers to an object, such as a speaker pointing to her nose when she says “I can’t smell”; this is a concrete deictic gesture. This is not to be confused with a gesture designating a specific area within metaphoric space (Kita, 2000; McCafferty, 2004), which is labeled an abstract deictic.
A final category for the present study was illustrator. “Illustrators add considerably to the amount of information conveyed by speech, especially about shapes, physical objects, and spatial relations” (Argyle, 1998, p. 195). McCafferty (2002) described an illustrator he called a “splash” gesture, which was “used by the speaker to provide an imagistic realization of the accompanying verbalization” (McCafferty, 2002, p. 197).
An Actional View of Gesture
Kita (2000) used some of these categories to describe and propose types of thinking that underlie representational gestures. Kita’s (2000) proposal is called the Information Packaging Hypothesis. To generate this theory Kita analyzed the way manner was coded by eleven Japanese Speakers and eleven English speakers when they recounted the interactions of cartoon figures. Again, this was Sylvester the cat trying to catch Tweety Bird.
Kita’s (2000) main argument is that there are two types of thinking underlying representational gestures: analytic thinking and spatio-motoric thinking (for Kita, 2000, representational gestures are iconic gestures and abstract deictic gestures). Kita (2000) describes analytic thinking as linguistic and non-linguistic, including scripts. Most important, analytic thinking does not necessarily involve any information from the senses. “Analytic thinking organizes information by hierarchically structuring decontextualized conceptual templates (henceforth analytic templates)” (Kita, 2000, p. 164).
Spatio-motoric thinking is a way of organizing “information with action schemas and their modulation according to the features of the environment” (Kita, 2000, p. 164). People move through the environment deciding direction, noticing and following objects with their eyes, picking up a tool perhaps and letting their hands run over the different features, opening and closing the tool, testing the pressure necessary for a controlling grip (ibid). Kita (2000) proposed that, “spatio-motoric thinking can be applied to the virtual environment that is internally created as imagery. Representational gestures are actions in the virtual environment” (ibid, p.165). In other words, humans have sensing bodies that move through the world, and this is foundational to the images humans create. Images involving spatial relationships and physical sensations are important for thinking.
This notion of language as embodied and the links between the physical environment and human cognitive systems is a part of the grand debate over the separation of the mind and body; that is, Cartesian dualism (Varela, Thompson, & Rosch, 1991; McCafferty, 2002, 2004). It is beyond the scope of this paper to cover all the facets of this argument; however, by recognizing language and gesture as tools for regulating human cognition (Wertsch, 1985; Kozulin, 1998), the perceived border between the external world of the body and the internal world of the mind collapses (Kozulin, 1998; John-Steiner & Meehan, 2000). The data with Chou in the present paper supports this notion of language as embodied, as well as some of the positions Kita (2000) takes with the Information Packaging Hypothesis. These findings will be presented and discussed later in the paper.
Gesture, the L1, and the L2
Two studies investigating the relationship of gesture and thinking for speaking in a first language (L1) and second language (L2) are McCafferty (2004) and Negueruela, Lantolf, Jordan, and Gelabert (2004). First, McCafferty’s (2004) study is presented. This study provides support for Kita’s (2000) notion of language and gesture as embodied, particularly in the way gesture reveals speakers physically positioning themselves in a virtual world as they speak and gesture. Following McCafferty (2004), Negueruela et al (2004) is reviewed. Although there are differences in these studies, both took similar Vygotskian approaches to the data and both focus on situations in which motion is important. Finally, Vygotskian studies that look at relationships between gesture and L2 learning are presented.
McCafferty (2004) investigated the relationship of gesture and language with a Taiwanese student and an American graduate assistant who was a native-speaker of English (the same participants as McCafferty, 2002). McCafferty’s (2004) focus was on the intrapersonal use of gesture, specifically, “the use of gesture and space as a self-organizing form of mediation for L2 learning” (McCafferty, 2004, p. 149). Clearly evident in McCafferty’s (2004) study is the way spatio-motoric thinking played a strong facilitative role for cognition and communication.
The data were collected while participants (J, the native English speaker, and B, the Taiwanese international student) discussed a variety of topics in front of a video camera in a classroom. Using examples of J and B establishing location in a metaphoric space, McCafferty’s (2004) data show how thinking unfolds through gesture.
While J was talking about speaking language from “different countries”, he “moves right hand out in front of him at chest level, palm down, hand sweeping laterally, designating countries with beats” (McCafferty, 2004, p. 157). In a similar way, B, represented Korea, Japan, and China in the gestural space around him when saying, “it’s from China transport to Japan and Korea” (“it” refers to Kanji, a written language that generally can be read and understood by Koreans, Chinese, and Japanese). McCafferty’s (2004) transcription of B’s gesture follows.
For the transcription, the gesture phrase (generally regarded as from one resting point to another across time) (McNeill, 1992; McCafferty, 2002) is transcribed below the oral speech that co-occurred with the stroke. B’s Oral speech that co-occurred with gesture is in brackets. A similar transcription protocol is used for Segment Two for the present study.
B1….
b) [it’s from]
left hand comes off the arm of the chair, up in front of him at chest level with two beats (one for each word)
c) [China]
left hand moves beyond his body, palm up in a beat
d) [transport to]
left hand retraces same path as in c, with palm down, fingers splayed
e) [Japan and Korea]
same palm down, fingers-splayed hand position, moves left hand right with beat for Japan, then back just a little with second beat for Korea (McCafferty, 2004, p. 158)
Several words later, B returns to the same gestural space in front of him where he previously indicated China, and he performs a beat on the word “China” (ibid, p. 158). He also refers to Korea and Japan while producing another beat and performs beats on Korea and China.
As can be seen in this data, B is using the gestural space around him to create a virtual map that illustrates the locations of Korea, Japan, and China, and then he uses an arc to signify the movement of writing, and later in this same segment he refers to culture. According to McCafferty (2004), this can be linked to Kita’s (2000) ideas on spatio-motoric thinking and analytic thinking:
[B]ecause of B’s difficulties expressing himself in English, he resorted to the spatio-motoric channel for thinking. That is, because he did not have control over the language, and thus, analytic thinking, he mapped out an organizing principle of the discourse—the historical relationship of two of the satellite countries of China—in virtual space, and referred back to this virtual map through gestures. (McCafferty, 2004, p. 161)
McCafferty (2004) concludes that his study “provides evidence that representational gestures carry communicative functions” (McCafferty, 2004, p. 162). In addition, McCafferty (2004) relates his findings back to the notion that language is embodied and representational gestures are actional and closely linked to cognition. Moreover, even though McCafferty (2004) did not explicitly relate his findings to McNeill and Duncan’s (2000) notion of thinking for speaking, McCafferty (2004) points out that “B was enacting patterns of the language and its use through gesture in an embodied experience of the L2 with regard to emphasis and syllable structure, although this is clearly an area in need of further research” (ibid, p. 162). In this way, McCafferty (2004) extends the findings of McCafferty (1998), which supported the notion of beats having “a self-regulatory function” (ibid, p.162). Most interesting is how spatio-motoric thinking, expressed through gesture, served a self-regulatory function.
Another study that investigated thinking for speaking in an L2 was Negueruela, Lantolf, Jordan, and Gelabert (2004). To investigate McNeill and Duncan’s (2000) notion of thinking for speaking, Negueruela et al. (2004) used path and manner as these were expressed in the motion verbs of English and Spanish.
Recall that English is a satellite-framed language and Spanish is a verb-framed language (Slobin, 2003). Also recall that thinking for speaking “refers to how speakers organize their thinking to meet the demands of linguistic decoding online during acts of speaking” (McNeill and Duncan, 2000, p. 141). In addition, satellite-framed languages and verb-framed languages code manner and path differently through gesture (ibid). Negueruela et al. (2004) wanted to know if advanced L2 speakers shifted to an L2 pattern of thinking for speaking (TFS). As in the McNeill and Duncan (2000) study, Negueruela et al (2004) focused on how gesture expresses manner and path during motion-event situations and how gesture contrasts with speech in these moments.
For their study, Negueruela et al. used twelve participants: three L2 Spanish speakers with English as their L1, three L2 English speakers with Spanish as their L1, and three monolingual English and three monolingual Spanish speakers as controls. Participants were video taped and asked to create narratives from transparencies presented on an overhead projector (only pictures, no text). They were arranged in a variety of language and interview configurations, including responding alone with an interviewer or co-constructing the narrative with another participant, but unable to see each other’s gestures.
The contrasting nature of participant arrangements regarding their L1s and L2s allowed several features of language and gesture co-occurrence to be explored, in particular, path-event constructions and manner-event constructions in English and in Spanish. These comparisons of manner- and path-event constructions were extensions of the following question: “Do advanced L2 speakers shift toward an L2 TFS pattern or do they rely on their L1 pattern as evidenced in the gesture/speech interface?” (Negueruela et al., 2004, p. 124)
Negueruela et al. (2004) found that L2 speakers did not shift to an L2 way of thinking for speaking. Participants indicated a preference for their L1 patterns while using the L2 by producing path and manner gestures consistent with L1 patterns. Negueruela et al. (2004) emphasized that this does not mean that L1 speakers will never shift into L2 patterns of thinking for speaking. In addition, the authors questioned if there is a direct relationship between verbal proficiency and TFS patterns that a specific language promotes.
McCafferty (1998, 2002) and McCafferty and Ahmed (2000) are three relevant studies of non-verbal speech in which the majority of participants were non-native speakers of English. Each study had different configurations of participants: these included participants who were non-native speakers of English paired with non-native speakers from different first language foundations (i.e., learned English in a naturalistic or instructional-only context; mainly monolingual or bilingual)
In the McCafferty and Ahmed (2000) study, the focus was on the appropriation of metaphoric gestures. The purpose of the study was to investigate the appropriation of abstract gestures (i.e., different types of metaphoric gestures) and look for any relationships between several combinations of learning contexts and foundational L1 and English proficiency levels. In terms of the context and instruments, participants were paired up and asked to consider a list of seven discussion-types of questions about marriage. During the data analysis, the frequency of metaphoric gestures and other types of gestures were coded (e.g., nodding of the head; shrugging of the shoulders).
According to McCafferty and Ahmed (2002), their findings support the idea that those living in another culture appropriate some of the metaphoric gestures common in that particular culture. With regards to inner speech going through transformation, the authors suggested that it would be unlikely, considering the close connection between gesture, private speech, and inner speech, that participants' inner speech would not be transformed by the appropriation of a gesture from a different culture than their own. Of course, the authors mention limitations to these conclusions and suggest that further research in non-verbal speech explore the relationship of gesture and language learning.
In a related study McCafferty (1998) investigated eight adult ESL learners' use of gesture in a narrative recall of a video and picture narration task that involved six sequential drawings. These participants were enrolled in an Intensive English Program at a university in the US. Four of these participants were from Venezuela and four from Japan.
In terms of the relationship of gesture and the psychological predicate, McCafferty (1998) found that participants “did indeed use gestures that were very much tied to what it was they were struggling with" (McCafferty, 1998, p. 82). McCafferty (1998) also found that beats were quite frequent in the data, and these beats occurred in examples where the participants seemed to have difficulty with a particular utterance. Other results included differences in the types of gestures the Japanese and Venezuelans used.
McCafferty’s (1998) findings support the notion that language is embodied. Moreover, with regards to the relationship of gesture, thought, and speech "the three (thought, gesture, and speech) are intertwined" (McCafferty, 1998, p. 93). McCafferty (1998) also mentioned the need for much more research with non-verbal forms of expression and second language learners.
McCafferty (2002) was concerned with the role of gesture in supporting a zone of proximal development (ZPD) (Vygotsky, 1978), the difference between what learners can do alone and what they can do with the assistance of a more capable other (Vygotsky, 1978). The participants in this study, a native English speaker who was a graduate assistant and a college-aged gentleman from Taiwan (J and B from McCafferty, 2004), used gesture to create zones of proximal development.
McCafferty (2002) emphasized the ZPD as a situated and co-constructed activity. Gestures were crucial to the participants working within each other’s ever-changing ZPD. The participants’ ability to imitate (see Tomasello, 1999; Lantolf, 2003) each other's gestures was also a prominent feature of their interactions and contributed to the establishment of the ZPD.
Overall, the research on gesture emphasizes the central role of gesture in human communication. Prior research also points to some fundamental differences in gesture as people move to different cultural and language-use situations. In addition, gesture has been found to be rich in examples of the psychological predicate and/or Growth Points (McNeill, 1992). Most important, gesture provides a window into human cognition (McNeill & Duncan, 2000). With regards to the present study on the relationship of gesture, language, and a concept map, there does not appear to be any prior research on the relationships of concept maps and gestures used in a formal presentation by a teacher of English as a Foreign Language, who is also a learner of English.
Method
Participants
Recall that the data for this paper was taken from a larger study, with Chou selected as a telling case. A telling case is one "in which the particular circumstances surrounding a case serve to make previously obscure theoretical relationships suddenly apparent" (Mitchell, 1984, p. 239). The interesting circumstances with Chou were her natural and schooled literacy with the symbolic Chinese writing system, and, at the beginning of the MATEFL course, she demonstrated the least explicit knowledge and experience with graphic organizers as learning or teaching tools for content in English. When compared with the other nine participants in the study during this first of two presentations students completed using graphic organizers, Chou’s gestures were distinctly more noticeable. Specifically, Chou’s gestures were more noticeable in the way they co-occurred and contrasted with spoken English and the concept map she alternately held in each hand as she spoke.
As mentioned earlier, Chou was a native speaker of Chinese and teacher of English from southern Yunnan, People’s Republic of China. Although not addressed in this paper, it is worth noting that Chou’s first language was Hani, a language of the Hani people who are one of the minority nationalities of southern Yunnan (Lewis & Bibo, 1996). At the time of the study, Chou was 44 years old, had been teaching English for 22 years, and had begun studying English at the age of 20. In a series of interviews conducted as part of the larger study, she acknowledged that she primarily used Chinese as a vehicle to teach English. With regards to the class presentation she made from which the data for the present paper was selected; this was the first major presentation she had given in the English language outside of China to an audience.
It is beyond the scope of this paper to address results from all participants; however, results from the wider study and the selected participants’ interactions over the length of the entire study naturally influenced interpretations of data presented in this paper. The participants included a native speaker of English, a native speaker of Dutch who completed his BA in English at Berkley in the U.S.A, a native speaker of Turkish, and five native speakers of Thai. Data from the other participants in the study provided reference points for triangulation to contrast and verify the data from Chou (Huberman & Miles, 1998).
Primary Researcher’s Role in the Study
The primary researcher, a white, American male, was 46 during the data collection period. He had taught for several years at Northern University and other venues in Northern City during the first half of the nineties through the fall of 1996. He was also married to a Thai from Northern City and had an extensive relationship with the surrounding community during this period. When he returned to collect data for this study, he was no longer married and had been away from Northern City for almost 6 years.
The primary researcher was a full participant observer in the study (Spradley, 1980) as teacher/researcher. Participants were fully aware that the primary researcher was collecting data for research, specifically that he was interested in graphic organizers, and they graciously allowed him to place two tape recorders in the room and have a research assistant operate a video camera from the back of the classroom. With the exception of the first class, the video camera and tape recorders were present the entire sixteen-week semester.
The Site
The major portion of the research took place with students in an advanced reading course that was a part of the MATEFL (Master of Arts in Teaching English as a Foreign Language) program at a university in Northern Thailand, which is called Northern University for the study. Northern University is located in a large city, which is called Northern City (also a pseudonym). Northern City is strategically located south of an area of northern Thailand called the Golden Triangle, which encompasses the border-areas of Laos, Burma, (Myanmar) and Thailand. In the past several years, the term Quadrangle Economic Zone has been used in the local and international press. This term includes the three countries already mentioned as part of the Golden Triangle, and China, which is commonly known to be prompting tremendous economic growth throughout the region.
Data Collection
Data for the larger study consisted of audio from two Panasonic tape recorders, video from a Sony analog camera, document data, follow-up interview data, and reflective fieldnotes typed within 24 hours of class meetings. As mentioned, data for the present study was video and audio data, and the concept map, with additional data from the larger study used for triangulation (Huberman & Miles, 1998). The research assistant, located at the back corner of the small classroom, had no specific instructions with regards to where or how long to zoom in or out on Chou for the data presented for this paper. As mentioned earlier, there was never any initial intention to focus on gesture until late in the data collection period.
Data Analysis
For the current paper, there are two segments of data presented in the Results/ Discussion with detailed gesture transcription: Segment One, a one-minute and twenty-five second segment that takes place at the beginning of Chou’s presentation, and Segment Two, an eighteen-second segment at the end of her presentation. Additionally, an Oral transcription (i.e. no gesture transcription) for Chou’s entire presentation before questions from the audience is in the Appendix. The entire presentation (excluding question/answer period) lasted approximately three-minutes and forty-seven seconds. This was the shortest of the nine oral presentations given six weeks into the semester.
Both Segment One and Segment Two are episodes of microgenesis. This notion of a genesis of sign creation and use, expressed here by the term microgenesis, is essential to understanding the overall approach to the data. Briefly defined, microgenesis is the creation and use of signs for mediating human activity (e.g., semiosis; see van Lier, 2004 p. 62; see also Robbins, 2003, p.123), which occurs in front of “one's eyes” (Vygotsky, 1978, p. 61) over short periods of time; as short as seconds or minutes, or over longer, successive periods of learning (Wertsch, 1985; 1991; Wells, 1999; see also Werner, 1978; Vygotsky, 1978, 1986; Cole, 1996).
Coding the Gestures
For determining the types of gestures created (i.e., iconic, metaphoric, beats, illustrators), a general analytic-inductive approach to the data was taken (Huberman & Miles, 1998). As mentioned earlier, categories were borrowed from McNeill (1992) and others (i.e., McCafferty, 1998, 2000, 2004), and coding was accomplished by moving back and forth between the transcriptions of gesture, video, audio, the gesture categories; then re-questioning parameters of the categories and adapting the categories slightly to effectively delineate the types of gestures found in the data. However, it needs to be emphasized that gestures were often a combination of one or more categories, for example, a gesture can easily be an iconic gesture that incorporates beats.
Transcription protocol
There are differences in transcription protocol for each segment, with Segment Two providing more detail; that is, a more precise description concerning on which part of the word(s) the gesture co-occurred. Transcription protocol is adapted from McNeill (1992), McCafferty (2004), and Wells (1999).
Descriptions of gesture in Segment One and Two are written in italics. Oral language is in regular text. Words on which gesture co-occurred or related in some other way worth noting are underlined. In Segment Two, brackets indicate in which part of the utterance the gesture mainly co-occurred with speech. This is similar to what is known as the gesture phrase, which is bounded by beginning and ending points; that is, the arms and hands at rest, the gesture occurs, and then the arms/hand at rest again (McCafferty, 2004; McNeill. 1992). Even though other parts of the body (i.e., eyes, face, eyebrows) are involved in the gesture, the emphasis in these transcriptions is on the hands and arms. Numbers presented in the transcriptions of gesture do not have any intended relationship with the line numbers in the full oral transcription presented in the Appendix.
Results/Discussion
So that the Discussion of gestures is clearly related to the transcription, the Results and Discussion are presented together in the context of the research questions. To support a clear presentation of the data and interpretation, Segment One and Two are parsed into smaller segments with discussion points immediately prior and following transcript examples. As mentioned earlier, the Appendix, which is referred to often in the Results/Discussion, displays the entire 3 minute and 47 second presentation.
Recall that the research questions were as follows:
Question One: How did gesture exemplify figure, ground, path, and manner (i.e., motion events), and using the motion event as a unit of analysis, was their any gestural evidence in a shift in thinking for speaking in the data?
Question Two: How did gesture and the concept map support and/or constrain Chou’s English language and overall content during this oral presentation?
Segment One
Container metaphors and thinking for speaking.
As mentioned earlier, Segment One is one-minute and twenty-five seconds long and occurs at the very beginning of Chou’s presentation. Of course, there are some huge differences between McNeill’s (1992) and McNeill and Duncan’s (2000) data and the data from Chou. Chou is not producing a narrative from viewing a cartoon and using Chinese as her L1; she is in a formal presentation context using a concept map to mediate her talk using English, her L2. She is operating at the nexus of the formal rule system of language, langue, and using language for concrete human activity; parole (Chandler, 2002). Indeed, there is an interesting synthesis of a variety of semiotic resources unfolding in the data. Her less formal L2 oral speech is guided by formal L2 writing represented by the Concept Map and the article Chou read prior to the oral summary, in addition to gesture. Take a moment to compare the Appendix and the concept map in Figure 1. Note how closely the concept map guided her speech.
With regards to evidence of a shift in thinking for speaking from Chinese to English, there is not strong enough evidence to clearly support that a shift in thinking for speaking occurred; that is, in this data, a strong example of a shift in thinking for speaking would be indicated by Chou producing a container metaphor, which is not common for speakers of Chinese, but is common for speakers of English (McNeill & Duncan, 2000). Recall that a container metaphor occurs when a bounded space is created by a speaker and offered as a container to the audience (McNeill, 1992).
In McNeill’s (1992) example of a Chinese speaker he said two things about metaphoric gestures for Chinese speakers and Turkana speakers of northeastern Kenya (Turkana speakers also do not produce containers) that have relevance for interpreting Chou’s metaphoric gestures. In discussing metaphors that Chinese speakers and Turkana speakers do make, McNeill said that “…narratives contain metaphoric gestures of other kinds, but not gestures in which abstract ideas are presented as bounded and supported containers” (McNeill, 1992, p. 151). McNeill (1992) also presented data from a Chinese speaker where she “…created a boundless substance [in her lap] that she then patted down (this is a metaphoric gesture also used by English narrators, but it is not the conduit). The gesture creates an image of a substance without form. The metaphor is that an abstract idea is a mass of some kind, a concrete substance, but it is not supported in the speaker’s hand” (ibid, p. 152).
In Chou’s gestures in Segment One, she creates a space in front of her next to the concept map. It can be argued that this space has substance and form. It can also be argued that rather than patting a formless space down, she returns to this space, specifically, when she noticeably pauses, as if searching for the word “information”, and she seems to find the word as she recreates the same gesture that she used on two other words which together represented the topic of “information” (i.e., “talk about” and “our ah organ”)
With regards to the possibility of a container metaphor, in several moments in the data, Chou’s right hand appears to hold something about the size of a grapefruit in her hand; her fingers slightly curled, bounding the round space in her hand, her palm angled toward the audience. In the transcript this is described as a “styrofoam grapefruit” (see below, line 3, Segment One). It appears as if Chou might be bounding an idea (the “information” about “organs” to be talked about) in her hand (palm down) and holding it out for the audience in three different instances: on talk about (line 3, Segment One), on our ah organ (line 5, Segment One), and on information (line 5, Segment One). She clearly signifies this space by moving her hand in a slow circle on talk about, our ah organs, and information, even adding a beat on organs. Immediately following information at the end of line 5, she closes her hand and simultaneously moves her hand toward her chest on receive on line 6. It’s interesting to note the inward movement toward her body on receive, which she also indicated with a quick flip of her hand, which she performed on line 5 on receive from before organs.
Although not clearly a container gesture, in Segment One there is an example of Chou bounding a space off to the side of her concept map and appearing to hold this space out to the audience over several words, returning to this same space with the same gesture, except the palm is angled slightly down toward the audience and her fingers are slightly curled. An English speaker would create a container with the palm facing up (McNeill, 1992). In addition, there are other differences in Chou’s gesture that match more of what would be expected from native English speakers. These will be presented shortly.
Transcript: Segment One, Lines 1-6
1: Mine ah is ah is concept map and the ah the article is from uh mapping
Chou is holding the concept map in both hands as she begins her talk
2: ah human’s mind and comprehension—ah human’s mind and understanding
3: So I choose this article because—ahh this article is talk about sense—sensing information
The first very noticeable gesture appears on choose. Chou’s right hand opens on choose, fingers spread and close, spread again, and her hand moves in a small circle on talk about. While moving in a circle, her hand is angled toward the audience; it’s as if she were palming something about the size of a grapefruit and as light as balsa or styrofoam. Her hand stops, momentarily hovering in the air after sense and returns to holding the concept map on information.
4: So—ah there are ah five senses—ah there, there are ah five ah information
5: ah we receive from our ah organs—so ah the first infor—ah-information
Her left hand spreads open on five senses, then returns to holding the concept map, then her right hand immediately leaves the concept map to hover next to it on there are. Her right hand gently flips, palm up, on receive, then flips, palm down, on from. On our ah organs, she holds her hand palm out, fingers spread, her hand angled to the floor from the audience, and she moves her hand in a slow circle. The movement stops with a beat on organs, and her hand quickly flexes open again after organ. On first, the index finger of her right hand is held up momentarily. Her hand drops on the false start, infor (line 5); then rises and fingers spread again on information, the palm outward and angled more toward the audience than before. Again, her right hand could be holding the styrofoam grapefruit.
6: we receive—sensory information we receive—uh visual information—ah we receive from
On the first receive at the beginning of line six, her fingers close from the styrofoam grapefruit metaphor. and move slightly toward her chest
Microgenesis, sign creation, and supported oral speech
In the next example from Segment One below, the most noticeable gesture that supported Chou’s speech was the co-occurrence of a concrete deictic gesture when Chou refers to the organs of the eyes in line 7. Recall that a concrete deictic gesture for this study is when the participant points toward an object in the immediate environment, which in this moment, is her eyes. This general pointing, with variation, is repeated later in Segment One to refer to her ears and her nose (see description under line 10).
Chou again repeats a very similar gesture to refer to her eyes during the question period following her three minute and forty-seven second talk. What is particularly unique about the first concrete deictic, which is the repeated variation occurring later in her talk, is the closing of her hand while pointing toward the eyes. This closing of her hand and pointing toward her eyes, which is described in the following segment of transcription as a puppet/parrot-like movement, represents the genesis of a sign into a repeatedly used symbol that supports speech and content; that is, microgenesis. Indeed, a clear transformation of her gesture into a semiotic resource is unfolding, semiosis, a movement from gesticulation to symbol.
This second transcript from Segment One begins with the introduction of the puppet-like movement of her hand with her fingertips pointing toward her face the first time the word “eye” occurs on line 7. The second time this gesture occurs is on the second time the word “eye” occurs on line 7. When the word “organ” and “ear” occur on line 10, the parrot/puppet-like movement of her left hand with the fingertips pointing inward is repeated. Following the first 1 minute and 18 seconds (see Appendix for the entire oral transcript of Chou’s talk), this parrot/puppet-like movement of the hand, with the fingertips inward, and with slight variations, is repeated on the words “nose” on line 13 (see Appendix), and then on the word “our” in the phrase “our organ” on line 16 (see Appendix).
During a question and answer period following her presentation, the parrot/puppet-like gesture occurred three out of the four times the word “eyes” occurred. Most noticeable in all occurrences of this gesture was her hand movement away from the head, similar to a pulling-away action of a trombone slide, with the fingers pointing toward her face; a combination of a deictic gesture and a beat, which is functioning to mediate Chou’s oral speech and overall presentation.
Transcript: Segment One, Lines 6-10
6: we receive—sensory information we receive—uh visual information—ah we receive from
7: the organs of the eyes—ah organs of the eyes—so umm—we use the eye –
On the first receive at the beginning of line six, her fingers close from the styrofoam grapefruit metaphor. and move slightly toward her chest. On ah we receive, at the end of line six, Chou begins to point at the concept map with her left hand and performs soft beats with her finger over the left section of the concept map. Her finger is moving over the concept map as if following a line of print on a page. On umm we use the eye, Chou’s hand moves from pointing at the concept map to a position closer to and pointing toward her face on the word eye. Her hand opens and closes once as it moves away from her face, clearly signifying her eye; the hand opening and closing, parrot-like, simultaneously pointing at and imitating the eye, like a hand-puppet. This movement becomes smoother, and less marked on the second we use the eye, which appears at the beginning of line 8 below.
8: ah we use the eye ahh and integrate—integrate movement ah shape, color, etceteras
On the first integrate, on line 8, which is not phonetically clear, Chou’s right hand moves from pointing to the map to the fingers coming together with her hand flicking in a quick beat a few inches to the left of the concept map, as if flicking an insect away. This beat is repeated immediately following movement, although this beat seems awkward and rushed this time; the flick of her hand seems added as an afterthought to the word. On etceteras, ending this sentence, she flicks her hand with a slightly more marked movement than before.
9: The second ahh sensory information we can receive from auditory information
10: We use the organ—ears—so integrate noise, sounds, and speech. And ah so ah
The camera zooms out. On organs, and on ears, with a pause in between, Chou’s performs the parrot/puppet-like movement with her left hand as she performed on line 7 with we use the eye. On organs, her hand is closer to her eye, however, when she says ear, her hand is positioned slightly more toward her ear, noticeably closer to the ear than the eye. Her hand spreads slightly, and performs beats on integrate, noise, and speech, the marked nature of the beats soft on integrate, the hand spread wider and the beat more marked on noise; this series of gestures ending with a soft beat on speech. Her left hand then returns to holding the concept map after speech.
Segment Two
As mentioned previously, Segment Two is the last eighteen seconds of Chou’s presentation and this is transcribed with more of an emphasis on precision than breadth (see Kendon, 2005; McCafferty, 2002; McNeill, 1992). The difference in protocol more precisely illustrates speech and gesture synchronicity; brackets indicate in which part of the utterance the gesture mainly co-occurred with speech This bracketed area marks what is known as the gesture phrase, which is bounded by beginning and ending points; that is, the arms and hands at rest, the gesture occurs, and then the arms/hand at rest again (McCafferty, 2004; McNeill. 1992). Again, as in the previous protocol, the underlining signifies that the word co-occurred with the described gestures. Also, as in the presentation of Segment One, Segment Two is parsed into smaller units with discussion points breaking the transcript so the relationship between Discussion and Results is clearer. Refer to the Appendix (end of line 19 to the end of line 21) for the unbroken eighteen second segment at the end of Chou’s speech.
Manner gestures and the support of meaning
In Segment Two, the content of Chou’s oral speech and her eye gaze indicate that she is looking at the bottom section of the concept map marked “Kinesthetic Sense” (see Figure 1). The most striking example of manner was exhibited in Segment Two. This segment opens with Chou trying to describe the kinesthetic feeling of “ground-ride”. She bounces her hand up and down lightly on “sense” and “ground-ride” (see lines a and b). Take note of what seems to be the only manner-verb appropriate here, but not orally stated: bounce. Chou is making a bouncing motion that co-occurs with the words “sense” and “ground ride”. This phrase, ground ride, does not appear anywhere on the concept map, and it may be that “sense” and “ground ride” are the only lexical items Chou can think of at this moment to describe the bounce humans feel in cars and planes. In this example, she substitutes a gesture to describe manner because she does not know or cannot remember the manner verb bounce. With regards to the support of meaning, Chou supports the meaning for “sense” and “ground ride” with the bouncing movement of her hand.
a). so this is ah ah kine[sthetic sense]
opens hand and bobs it up and down twice on sense
b) and sometimes we feel ah [the ground ride]
looks up from concept map and her hand beats once on ground and twice on ride
Figure, ground, and path expressing Kita’s spatio-motoric thinking
In the next transcript example from Segment Two, Chou positions herself on the ground by explicitly positioning a plane overhead with her body and hands: she spins her hand, moving higher above her head, in small circles that become beats as she says, “uh in the plane ah on the plane so” (see transcript c below). The co-occurrence of the beats with the switching of the prepositions suggests they are connected. Recall that beats have been found to be associated with difficulties in speech and become self-regulating tools (McCafferty, 1998, 2004). Beats also add emphasis to what is being said (McNeill, 1992). Additionally, this gesture acts as an illustrator (Argyle, 1998; McCafferty, 2002) by locating the plane above her head.
This location of the plane above Chou’s head in this segment is also important in expressing Kita’s (2000) notion of a spatio-motoric type of thinking. This is when Chou moves her hand to the area above her forehead when she begins talking about feeling movement in a plane. Chou very clearly displays her bodily orientation to the physical environment with the movement of a plane positioned in the area above her head. In terms of figure and ground, she is positioning herself as ground and the object as figure. It can also be inferred that the path is in the air above her and the figure (the plane) is moving away. It also needs to be mentioned here that right before this 18 second final segment, Chou had visibly shifted from displaying herself as driving a car a moment before the beginning of Segment Two (see Appendix, lines18 and 19) by holding an imaginary car steering wheel in her hands.
By illustrating herself behind the wheel of a car, Chou was positioning herself in the action happening in a virtual environment. This is one of Kita’s (2000) hypotheses, that the gesture is not only imagistic: “a gestural sign is formed by the cognitive system that is also used in the movement in the physical environment” (Kita, 2000, p. 170). Chou is in the action in an imagined physical environment, as in putting herself in the car, or in positioning herself as ground with a figure, the plane, passing overhead. When positioned behind the wheel of a car, and at one point swaying her entire body to illustrate the movement of the car, she was positioning herself as figure and the ground as passing outside the car. These actional gestures were all related to supporting broader oral ideas expressed in English and derived from the concept map held in her hand, and what she remembered from her reading.
c) [or ground ride or in uh in the plane ah on the plane so]
still looking up from the concept map, her hand lightly bounces down at a deeper level in the space in front of her on ride and immediately moves upwards, chopping the air once in front of her on the conjunction or. Her hand continues upwards, spiraling distinctively. With her hand held momentarily flat and straight, she finishes the prepositional phrase in the plane. The spiraling circular movement continues upwards and changes into more of an up and down movement as says ah on the plane; her hand notably marks plane by a distinctive beat that is the apex of all the movement that occurred at this higher level of space in front of her. As she says on the plane Her left hand moves to a foot above her forehead, slightly in front of her, before dropping down to grasp the concept map at the end of plane so.
d) ah [finish ]
left hand moves from holding the paper to palm up and open, facing the ceiling, as if designating an empty space as someone might say there isn’t anything left.
Also clearly displayed in Segment Two and other parts of Chou’s presentation were moments of the psychological predicate. Defining a psychological predicate (and hence a GP) requires reference to the context; that is because the psychological predicate and its context are mutually defining. As mentioned earlier, the GP:
1. marks a significant departure in the immediate context; and
2. implies this context as a background. (McNeill & Duncan, 2000, p. 145)
One of the salient moments in the data where the psychological predicate/growth point is easily identifiable is in Segment Two. Chou is describing how the body senses movement “kinesthetic sense” (see line a, Segment Two). Recall that a moment prior to Segment Two, she described driving a car (see line 19 in the Appendix) and had even placed her body in the car by holding a wheel in her hand and turning it from side to side. Chou’s left hand then moves from the level of the ground (where she was driving a car) to a higher level above her head to describe how one’s body feels the ground-ride “in the plane ah on the plane” (Segment Two, line c).
As in the description of the GP from McNeill and Duncan (2000), Chou moves from the immediate context, feeling the “ground-ride” in a car, to high above her head (i.e., in the sky) in the context of feeling the “ground-ride” in the plane. This sequence of gestures implies a larger scene as the background with Chou at ground level and the plane in the sky above her head. This was just one of many moments in the data where Chou uses gesture to provide a background for the context of the current speech or her gesture signified an immediate departure from the current context.
Conclusions
Overall, the data from the present study demonstrate the importance of gesture in supporting a common literacy activity for L2 learners: the oral summarization of academic text. Both Segments One and Two have numerous examples of the gesture and the concept map working together to support Chou’s English language, and many of the gestures co-occurring with speech are illustrators that support content. Most interesting are the concrete deictic gestures introduced during the first minute and eighteen seconds of her presentation (see Segment One), which are repeated beyond their introduction and become prominent illustrators for “eyes”, “nose”, and “ears”. Moreover, thinking for speaking in these data has been found to be very different than originally expected from McNeill and Duncan’s (2000) notion of thinking for speaking.
During the entire 3 minute and 47 second talk and question and answer period that followed, when Chou shifts to looking toward the audience from simultaneously talking and referring to the concept map, her gestures generally become more prominent and synchronized with speech. All through her presentation, the concept map becomes a crucial part of her thinking for speaking in many moments in the data. As a unique set of data, these data clearly illustrate a nexus of langue (formal writing) and parole (conversation) in thinking for speaking. The data illustrate how thinking for speaking in an academic setting becomes a synthesis of a wide variety of semiotic resources that exist in a dynamic co-constructed state with her audience. The major semiotic resources that support the English language in these data, beyond signals the audience supplied to Chou, are the concept map and her gestures.
Also of interest were moments in the data where the Growth Point/psychological predicate could be inferred. These moments provided a view into what Chou determined as salient in her next moment of speaking. Identification of the Growth Point/psychological predicate allows a glimpse into the summarization process of a non-native speaker of English as she worked to make herself clear to the audience.
Although there was no conclusive evidence of a transformation of Chou’s L1 patterns of thinking for speaking (i.e., Chinese patterns), there was the possibility of the creation of a container gesture in Segment One, which is not expected of speakers of Chinese (McNeill, 1992). In addition, Chou’s patterns of gesture followed closely with subject-predicate relations and there was not a pattern of delays between verbs and gesture as found by McNeill and Duncan (2000) and McNeill (1992). However, thinking for speaking is very different in these formal circumstances than has been described in other studies (e.g., Negueruela, Lantolf, Jordan, & Gelabert, 2004).
An investigative focus on figure, path, ground, and manner (i.e., motion events), provided an additional insight into the links between language and gesture. One strong example in the data was Chou’s emphasis on manner in Segment Two to describe the “ground-ride”; she provided insight into a lexical item, the word bounce, with which she was having difficulty producing. In other episodes in the data, figure and ground revealed Chou’s cognitive/physical orientation to the images she was portraying for her audience. In these examples, the embodied nature of language is clearly evident (see also McCafferty, 2004).
As in other studies on gesture, this paper demonstrates the effectiveness of investigating gesture to make judgments about language and cognition. Moreover, as proposed by McNeill (1992), gesture provides a window on cognition. Further study of gesture, language, and cognition is needed, particularly in the ESL/EFL field.
Limitations
As in all research there were many limitations to this study. Despite extensive triangulation of the findings with a variety of data from the larger study, and an objective approach to the data, the findings are ultimately subjective. In addition, generalizability of the findings is limited. To counter a variety of limitations, data interpreted for the present paper are displayed for readers to make their own judgments about verifiability of the findings; that is, a sincere attempt has been made to create a transparent path from the data to the Discussion and Conclusions (Huberman & Miles, 1998; see also Altheide & Johnson, 1998).
Implications for teaching and learning and avenues for further research
For ESL/EFL classes, a possible path for using gesture data for the teaching and learning of English as a second or foreign language may be to teach students about the different categories of gesture they may encounter and/or can use while listening to a lecture or during their own presentations. With the increased access to video cameras in a variety of teaching contexts, it would be interesting to have ESL/EFL students film themselves talking and study their use of gesture and language. Could a kind of metacognition of gesture interpretation be created for language learners? In addition, because the co-occurrence of language and gesture may have connections to language proficiency, gesture may provide an avenue to language assessment (Stam, 2002; see also McCafferty, 2002), not necessarily in a formal sense, but to gain a general idea of what language forms and lexical items are troubling learners.
It is also interesting to note the proposed differences in L1 language patterns expressed by the motion event and that thinking for speaking may be expressed through gesture. Would it be worthwhile to make ESL/EFL learners more aware of the differences in thinking for speaking patterns in their L1s and the target language? Slobin (2003) proposes the idea that if a language emphasizes path or manner than speakers will attend to one over the other; is this always the case even when semiotic resources in the L2 dominate the thinking for speaking context? If language learners become more aware of how figure, ground, path, and manner are expressed by gesture and in their writing, would this knowledge aid them in language learning, in listening, speaking, reading, or writing? How is thinking for speaking affected by the variety of semiotic resources available to language learners using the target language? What exactly is the concept of thinking for speaking in the variety of multimedia contexts available to assist communication in the target language? These and many other questions provide many practical research opportunities for ESL/EFL teachers and researchers, concerning the synthesis of gesture, speech, and semiotic resources such as concept maps and other media that support communication.
Appendix (see pdf)
References
Altheide, D., & Johnson, J. (1998). Criteria for assessing interpretive validity in qualitative research. In N. Denzin and Y. Lincoln (Eds.), Collecting and interpreting qualitative materials (pp. 283-312). Thousand Oaks, CA: Sage. Argyle (1998) Bodily Communication (2nd ed.). Madison, CT: International Universities Press.
Chandler, D. (2002) Semiotics: The basics. London, UK: Routledge
Cole, M. (1996). Cultural psychology: A once and future discipline. Cambridge, MA. Harvard University Press.
Dillon, D., O'Brien, D., Heilman, E. (2000). Literacy research in the next millennium: From paradigms to pragmatism and practicality. Reading Research Quarterly, 35, 10-26.
Engestrom, Y, & Miettinen, R. (1999a). Introduction. In Y. Engestrom, R. Miettinen, and R. L. Punamaki (Eds.), Perspectives on activity theory (pp. 1-16). N.Y.: Cambridge University Press
Engestrom, Y. (1999b). Activity theory and transformation. In Y. Engestrom, R. Miettinen, and R. L. Punamaki (Eds.), Perspectives on activity theory (pp. 19-38). N.Y.: Cambridge University Press.
Gee, James (2003). What video games have to teach us about learning and literacy. N.Y.: Palgrave Macmillan.
Huberman, M., & Miles, M. (1998). Data management and analysis methods. In N. Denzin and Y. Lincoln (Eds.), Collecting and interpreting qualitative materials (pp. 179-210). Thousand Oaks, CA: Sage.
John-Steiner, B. & Meehan, T. (2000). Creativity and collaboration. In C. Lee and P. Smagorinsky (Eds.), Vygotskian perspectives on literacy research: Constructing meaning through collaborative inquiry (pp. 31-48). New York: Cambridge University Press.
Kendon, A. (2004). Gesture: Visible action as utterance. Cambridge UK: Cambridge University Press.
Kita, S. (2000). How representational gestures help speaking. In McNeill (Ed.), Language and gesture (pp. 162-185). New York: Cambridge University
Kozulin, A. (1998). Psychological tools: A sociocultural approach to education. Cambridge MA: Harvard University Press.
Lewis, P. & Bai, B. (1996). Hani-English English-Hani Dictionary. New York: Kegan Paul International.
Lantolf, J. (1994) Sociocultural theory and second language learning: Introduction to the special issue. Modern Language Journal, 78, 418-420.
Lantolf, J. (2003) Intrapersonal communication and internalization in the Second Language Classroom. In A. Kozulin, B. Gindis, V.S. Ageyev, and S. Miller (Eds.), Vygotsky’s educational theory in cultural context (pp. 349-370). NY: Cambridge University Press.
McCafferty, S. (1994). Adult second language use of private speech: A review of studies. Modern Language Journal, 78, 421-435.
McCafferty, S. (1998). Nonverbal expression and L2 private speech. Applied Linguistics, 7, 73-96.
McCafferty, S. (2002). Gesture and creating zones of proximal development for second language learning. Modern Language Journal, 86, 192-202.
McCafferty, S. (2004). Space for cognition: gesture and second language learning. International Journal of Applied Linguistics, 14, 148-165.
McNeill, D. (1992). Hand and mind: What gestures reveal about thought. Chicago: The University of Chicago Press.
McNeill, D., & Duncan, S. (2000). Growth points in thinking-for-speaking. In D. McNeill (Ed.), Language and gesture (pp. 141-161). New York: Cambridge University Press.
Merkley, D., & Jeffries, D. (2000). Guidelines for implementing a graphic organizer. The Reading Teacher, 54, 350-357.
Mitchell, J.C. (1984). Case studies. In R.F. Ellen (Ed.), Ethnographic research: A guide to general conduct (pp. 237-241). San Diego, CA: Academic Press.
Mohan, B. (1986). Language and Content. Reading, MA: Addison-Wesley.
Negueruela, E., Lantolf, J., Jordan, S., & Gelabert, J. (2004). The “private function” of gesture in second language speaking activity: a study of motion verbs and gesturing in English and Spanish. International Journal of Applied Linguistics, 14, 113-147.
Novak, J. D, & Gowin, D. B. (1984). Learning how to learn. New York: Cambridge University Press.
Peirce, Charles S. (1991) Peirce on Signs: Writings on Semiotic by Charles Sanders Peirce. Ed. James Hoopes. Chapel Hill: The University of North Carolina Press.
Peregoy, S., & Boyle, O. (2001). Reading, writing, & learning in ESL: A resource book for K-12 teachers. New York: Longman.
Robbins, D. (2003). Vygotsk’s and A. A. Leontiev’s semiotics and psycholinguistics: Applications for education, second language acquisition, and theories of language. Westport CT: Praeger Publishers.
Robinson, D. (1998). Graphic organizers as aids to text learning. Reading Research and Instruction, 37, 85-105.
Slobin, D. (2003). Language and thought online: cognitive consequences of linguistic relativity. In D. Gentner and S. Goldin-Meadow (Eds.), Language in mind: advances in the study of language and thought (pp. 157-192). Cambridge: MA: MIT Press.
Spradley, J. (1980). Participant observation. Fort Worth, TX: Harcourt Brace Jovanovich
Stam, G. (2002). What gestures can tell us about second language acquisition. Paper presented at the First Congress International Society Gesture Studies (ISGS) University of Texas, Austin June 5, 2002.
Talmy, L. (1985). Lexicalization patterns: Semantic structure in lexical forms. In Shopen (Ed.), Language typology and syntactic description, vol. III: Grammatical categories and the lexicon (pp. 57-149). Cambridge: Cambridge University Press.
Talmy, L. (1991). Path to realization: a typology of event conflation. In Sutton, L.G., Johnson, C. & Shields, R. (Eds.), Proceedings of the 17th Annual Meeting of the Berkeley Linguistics Society (pp. 480-520). Berkeley, CA: Berkeley Linguistics Society.
Tomasello, M. (1999). The cultural origins of human cognition. Cambridge, MA: Harvard University Press.
Van Leer, L. (2004). The ecology and semiotics of language learning: A sociocultural perspective. Boston: Kluwer Academic Publishers.
Van Leeuwen, T. (2005). Introducing Social Semiotics. NY: Routledge.
Varela, F., Thompson, E., & Rosch, E. (1991). The embodied mind: Cognitive science and human experience. Cambridge, MA: MIT Press.
Vygotsky, L. (1978). Mind in society: The development in higher psychological processes. Cambridge MA: Harvard University Press.
Vygotsky L.S. (1981). The genesis of higher mental functions. In J.V. Wertsch, (Ed.), The concept of activity in Soviet psychology (pp. 144-188). Armonk, N.Y.: M.E.Sharpe.
Vygotsky, L.S. (1986). Thought and language. Cambridge, MA: MIT Press.
Wells, G. (1999). Dialogic inquiry: Toward a sociocultural practice and theory of education. New York: Cambridge University Press.
Werner (1978). Microgenesis and aphasia. In S. Barten and M. Franklin (Eds.), Developmental processes: Heinz Werner’s selected writings Vol 2 (pp. 429-444). NY: International Universities Press, Inc.
Wertsch, J. (1979). The regulation of human action and the given-new organization of private speech. In G. Ziven (Ed.), The development of self-regulation through private speech (pp. 79-98) New York: John Wiley & Sons.
Wertsch, J. (1985). Vygotsky and the social formation of mind. Cambridge, MA: Harvard University Press.
Wertsch, J. (1991). Voices of the mind: A sociocultural approach to mediated action. Cambridge MA: Harvard University Press.
Wertsch, J. (1998). Mind as action. New York: Oxford University Press.