The author, Kim Hyun Sook has worked for the Korean Education department for 15 years. She recently obtained her Masters from Monash University in Australia. Ms. Kim is also a lecturer in English to Korean university students. A further research proposal (Developing Communicative Competence through
Kagan's Cooperative Learning Structures ) was chosen for award by the Busan Teachers' Association. Other projects include working as a Teacher Trainer for Korean teachers and assisting in special project research for the advancement of English education in Korea. pdf version

 

This study aimed:
- to identify the types of speaking assessment tasks used by Korean Junior Secondary School English teachers and the ways in which those assessments were administered;
- to investigate Korean teachers' perceptions of the practical constraints in Korean EFL classrooms which affect assessment of speaking.

This study was conducted, using qualitative methods, with ten Korean Junior Secondary School English teachers. All ten participants responded to a questionnaire which asked for opinions on speaking assessment as well as professional, educational and personal background. Four of the ten also participated in an interview with the researcher to clarify their perceptions of speaking assessment.

The study found that the speaking assessment tasks used by Korean Junior Secondary School English teachers were those which:
- gave the students less psychological burden;
- were time-saving and designed for the convenience of construction and administration;
- did not demand the teacher to take the role of an interviewer.

As well, the study found that when assessing the speaking skills of their students, Korean Junior Secondary School teachers were not concerned with the validity and reliability of their assessments. Nor were the teachers equipped with an adequate theory of speaking assessment. As a consequence, the teachers had little confidence in conducting speaking assessment.

The study revealed the practical constraints of the Korean context which affected the assessment of speaking in the EFL. Participants reported such constraints as large classes and time-consuming, excessive work in addition to face-to-face classroom teaching, lack of training in conducting speaking assessment, lack of effective and efficient assessment instruments, difficulty in eliciting students' responses.

The conclusion is reached that although Korean teachers need to have assistance and encouragement to try new ways of communicative assessment in their EFL classrooms, they need to make themselves aware of the shift in social and educational needs and to make conscious and persistent efforts to introduce more communicative speaking assessment in spite of practical difficulties. For their part, educational administrators need to show greater sensitivity to the teachers' complaints of excessive workload and to reflect teachers' points of view in their decision-making.

Chapter One: Introduction

1.1 Background to the research
Testing oral proficiency has become one of the most important issues in language testing since the role of speaking ability has become more central in language teaching with the advent of communicative language teaching (Nakamura, 1993). As Bostwick and Gakuen (1995) state, assessment can be used to improve instruction and help students take control of their own learning. That is more likely to be accomplished when assessment is authentic and tied to the instructional goals of the program.

However, there are many difficulties involved in the construction and administration of any speaking assessment. There is a great discrepancy between the predominance of the Communicative Approach and the accurate measurement of communication ability (Hughes, 1989).

With the widespread adoption of communicative language teaching (CLT) in ESL countries (Hartley and Sporing, 1999), it is replacing the traditional grammar-centred, text-centred, and teacher-centred methods in Korean Junior Secondary Schools. With training in CLT offered by in-service teacher education programs, Korean Junior Secondary English teachers have been trying to expand CLT in their classrooms (Li, 1998).

However, although many English teachers in Korean Junior Secondary Schools are interested in CLT, communicative assessment has received little attention. If it is important to know if a person can speak a second language, then it should be important to test that person's speaking ability directly (Jones, 1977). Despite the interdependence of communicative teaching and communicative assessment (Bachman, 1990), speaking assessment in the Korean Junior Secondary Schools does not assess students' oral proficiency from the perspective of language use and communication. As Nagata (1995) pointed out, rote memorization of text dialogs has been a common practice for speaking assessment in Korea. It seems that Korean English teachers do not concern themselves much with matters of validity and reliability in relation to speaking assessment (Li, 1998).

However, the need for classroom teachers to be equipped with some measurement tools to evaluate students' oral proficiency is becoming more and more important (Nagata, 1995). Speaking assessment has become a vital part of all the examinations in all the Korean Junior Secondary Schools; every school is required, by the Ministry of Education and Human Resources Development, to perform students' speaking assessment at least once each year. The schools referred to in this study take a speaking assessment twice a year.

Korean Junior Secondary School English teachers, however, find it difficult to assess students' oral proficiency in a way which reflects authentic interaction (Li, 1998). Thus, there is a contradiction between their communicative language teaching and their assessments of speaking which do not reflect authentic oral interaction.

1.2 Research aims
There seems no prominent research on speaking assessment from the point of view of the teacher in Korea. In particular, the perceptions of Korean Junior Secondary School English teachers in conducting speaking assessment remain largely unexplored. Yet, teachers' perceptions of the feasibility of a communicative assessment in a particular context are crucial in determining the ultimate success or failure of that assessment (Kelly, 1980).

Thus, this study focuses on Korean Junior Secondary School English teachers' perceptions of speaking assessment.

Chapter Two: Literature Review

This chapter reviews prominent studies to support the present research. The principal areas to be discussed are communication and communicative competence, Communicative Language Teaching (CLT), speaking assessment, and the teachers' perceptions of speaking assessment.

2.1 Communication and communicative competence
As a basis for investigating communicative competence, the researcher begins by defining what communication is. Morrow (1977) describes seven features which characterize communication. According to him, communication

An adequate description of the nature of communicative competence is crucial to instrument content and instrument method (Weir, 1990). According to Ellis (1994:696), communicative competence is "the knowledge that users of a language have internalized to enable them to understand and produce messages in the language".

Several models of communicative competence have been proposed (Ellis, 1994). Chomsky (1965) proposed the concept of grammatical or linguistic competence and highlighted cognitive aspects of human language acquisition and learning. He distinguished between competence (one's underlying knowledge of the language) and performance (the realization of language in specific situations). On the other hand, Hymes (1971), who coined the term 'communicative competence' emphasized social, interactive, and negotiating process of language. Hymes expanded Chomsky's notion of competence into communicative competence by including both grammatical rules and rules of language use (Hymes, 1971; Taylor, 1983).

Canale and Swain (1980) identified three distinctive components of communicative competence: grammatical competence, sociolinguistic competence and strategic competence. Grammatical competence includes one's knowledge of lexical items, morphology, syntax, semantics, and phonology in a language. Sociolinguistic competence encompasses the knowledge of rules governing the production and interpretation of language in different sociolinguistic contexts. Lastly, strategic competence is defined as one's capability to sustain communication using various verbal or nonverbal strategies when communication breakdowns occur. This model was updated by Canale (1983). He proposed a four-dimensional model of communicative competence: grammatical, sociolinguistic, discourse, and strategic competence.

Bachman (1990) suggested a theoretical framework for communicative language ability. It includes knowledge structures, strategic competence, psychophysical mechanisms, context of situation, and language competence. Language competence is further divided into organizational competence (grammatical and textual competences) and pragmatic competence (illocutionary and sociolinguistic competences).

Bachman (1990), in his schematization of 'language competence' takes a broader view of the role of strategic competence than Canale and Swain do. Bachman separates strategic competence from what he calls 'language competence'.

Agreement on what components should be included in a model of communicative competence is never unanimous (Weir, 1993). In spite of many disputes by applied linguists (Lluda, 2000), this notion of communicative competence outlined above has proven useful in suggesting specifications for content, formats, and scoring criteria in communication-oriented language proficiency assessment (Bachman and Palmer, 1984). However, "it must be emphasized that they are still themselves in need of validation" (Weir, 1990:8).

2.2 Communicative Language Teaching (CLT)
According to Richards and Rodgers (1986), Communicative Language Teaching (CLT) starts with a theory of language as communication. The classroom goal of instruction is focused on developing learners' communicative competence. Thus, learners are encouraged to communicate with target language through interaction from the beginning of instruction.

In CLT, meaning is most important. Larsen-Freeman maintains that "Almost everything that is done is done with a communicative intent" (1986:132). Accordingly, the process of meaning negotiation is essential in CLT (Paulston, 1974). In order to encourage learners to communicate better, errors should be tolerated with little explicit instruction on language rules (Larsen-Freeman, 1986). Naturally, CLT favours small group activities by students to maximize the time each student has to negotiate meaning. CLT employs information-gap activities, problem-solving tasks, and role-plays through pair and group work (Larsen-Freeman, 1986).

Another feature of CLT is its "learner-centredness view of second language teaching" (Richards & Rodgers, 1986:69). According to Savignon (1991), every individual student possesses unique interests, styles, needs and goals. Therefore, it is recommended that teachers develop materials based on students' demonstrated needs of a particular class.

CLT emphasizes the use of authentic materials in teaching language (Widdowson, 1996). It also encourages giving learners the opportunity to respond to genuine communicative needs in real-life situations. This is to help learners develop strategies for understanding language as actually used by native speakers (Canale and Swain, 1980).

2.3 Speaking assessment
Testing oral proficiency has become one of the most important issues in language testing since the role of speaking ability has become more central in language teaching (Hartley and Sporing, 1999). Assessment needs to be theory driven. The concept of validity, reliability and efficiency affect assessment design (Bachman, 1990). In this section the various types of validity will be discussed. As well, it will be discussed how the concept of validity relates to those of efficiency and reliability.

2.3.1 Validity
Spolsky (1975) stated that validity is the central problem in foreign language testing.
Validity is concerned with whether a test measures what it is intended to measure (Weir, 1990). A test of speaking ability in a classroom setting is usually an achievement test. An achievement test should have content and face validities (Davies, 1983). Since content validity asks if the test content matches the content of the course of study (Bachman, 1990), what teachers can do is to match the course objectives and syllabus design with the test items. This attitude by teachers is crucial in a classroom test because teachers may tend to use test tasks different from the course objectives especially when oral aspects are involved (Nakamura, 1993).

Face validity pertains to whether the test 'looks valid' to the examinees, the administrative personnel and other technically untrained observers (Bachman, 1990). Face validity is a must in a classroom speaking test, because the students' motivation is promoted for speaking if a test has good face validity (Hughes, 1989).

Language testing can be put on a scientific footing through construct validity (Hughes, 1989). Bachman (1990) also highlighted that construct validity is the most fundamental validity for a speaking test. Construct validity examines if the test matches a theoretical construct (Bachman, 1990). This cannot easily be handled by classroom teachers because of the abstract nature of language abilities (Nakamura, 1993).

2.3.2 Reliability
The concept of reliability is particularly important when considering communicative language testing (Porter, 1983). Reliability is concerned with the extent to which we can depend on the test results (Weir, 1990).

Rater reliability is important to overall test reliability. What raters need to do for this purpose is to achieve high inter-rater reliability for these assessments. The degree of inter-rater reliability is established by correlating the scores obtained by candidates from rater A with those from rater B. The concern of the rater is how to enhance the agreement between raters by establishing explicit guidelines and maintaining adherence to them for the conduct of this rating (Bachman, 1990).

Although reliability is something raters need to try to achieve in the tests, it may not be the prime consideration all the time (Bachman, 1990). It is said that there is a reliability-validity tension. Reliability offers a possible compromise. It is occasionally essential to sacrifice a degree of reliability to enhance validity (Davies, 1990). For example, in certain circumstances, reliability and validity are mutually exclusive. However, if a choice has to be made, validity is more important for speaking assessment (Bachman, 1990).

2.3.3 Test efficiency
A valid and reliable test is useless if it is not practical (Bachman, 1990). "This involves questions of economy, ease of administration, scoring and interpretation of results" (Bachman, 1990: 34). The context for the implementation of a test is a vital consideration. Classroom tests should not require costly specialized equipment or highly trained examiners or raters (Weir, 1993). The tasks should be the most efficient way of obtaining the information about the test takers. There is much pressure on teachers to make tests as short and practical as possible because teachers cannot afford to spend much time in assessing students' communicative ability. However, "this should never be allowed to put at risk test validity" (Weir, 1993: 22).

To sum up, there is a need to develop test formats that provide overall balance of reliability, validity and test efficiency in the assessment of communicative skills (Bachman, 1990). Authenticity as a concept has also been a major concern in language testing (Bachman, 1990). Therefore, it is considered necessary to briefly examine authenticity.

2.3.4 Authenticity
According to Bachman (1990), authenticity is defined as a quality of the relationship between features of the test and those of the non-test target-use context. There are two approaches on authenticity; the real-life approach and the interactional ability approach. 'Real-life(RL) approach' tries to develop tests that mirror the 'reality' of non-test language use. This approach has been considered as naive because the test setting itself does not exactly resemble its real-life setting (Spolsky, 1985). Also "this approach does not distinguish between language ability and the context in which this ability is observed, since non-test language performance constitutes the criterion for authenticity and the definition of proficiency" (Bachman, 1990: 302).

In the second approach, the authenticity of language tests arises from their 'situational' and their 'interactional' authenticity. 'Situational authenticity' refers to the relationship of features of the test method to particular features of the target-use situation. 'Interactional authenticity' mentions the extent to which an examinee's language ability is engaged in the test task. Thus, the emphasis in this model shifts from "attempting to sample actual instances of non-test language use to that of determining what combination of test method facets is likely to promote an appropriate interaction of a particular group of test takers with the testing context" (Bachman, 1990: 317).

Assessment can be used to improve instruction and help students take control of their learning (Bostwick and Gakuen, 1995). Accordingly, it is also necessary to briefly examine 'backwash effect' as a concept.

2.3.5 Backwash effect
This term describes the effect of testing on teaching: "Assessment should be supportive of good teaching and have a corrective influence on bad teaching" (Hughes, 1989:2). Backwash can be harmful or beneficial. Positive backwash happens when students study and learn those things which teachers intend them to study and learn (Hartley and Sporing, 1999). On the other hand, negative backwash means the converse. For example, if teachers measure writing skills only through multiple-choice items, then there will be pressure to practice such items, rather than writing itself. In this case, the backwash would be negative.

Bachman (1990) highlighted that positive 'backwash effect' will result when the testing procedures reflect the skills and abilities that are taught in the course. Hartley and Sporing (1999) support the rationale and validity of assessing students communicatively who have been taught communicatively. By assessing communicatively, teachers would expect the backwash to be beneficial. If teachers wish students to learn to communicate effectively in a variety of practical situations, teachers should test them on these skills. This conscious feedback loop between teaching and testing, in terms of content and of approach, is a vital mechanism for educational development (Bostwick and Gakuen, 1995).

2.3.6 Elicitation
The key to effective oral proficiency testing lies in matching elicitation techniques with the purposes and constraints of the testing situation. In the case of school-related speaking assessment, tests are usually brief and consist of a single elicitation procedure (Madeson, 1980).

There is a great range of test types, depending on the content of instruction. For example, some tests use a simple question and answer procedure to assess communicative matters such as amount of information conveyed, comprehensibility, appropriateness of vocabulary, and fluency. Other formats include the guided interview, evaluation during group interaction, oral reports, dialogues and role-play, skits and drama. While the latter five are high in communicative face validity, they are difficult to assess with any consistency (Madeson, 1980).

2.3.7 Scoring
The techniques for eliciting speech samples must be linked with appropriate scoring procedures (Madeson, 1980). The decision as to whether to use a global or specific scoring procedure depends on the purpose of the test (Jones, 1977).

There are two contrasting ways of grading student speech; holistic scoring and objectified scoring. Holistic scoring concentrates on communication and tends to be selected when the teacher evaluates a wide variety of criteria simultaneously. The limitation of holistic scoring is that some teachers find it confusing to evaluate many things simultaneously (Bachman, 1990); it may be only those teachers with considerable experience and training who can use holistic scoring effectively.

On the other hand, specific scoring procedures attempt to identify smaller units such as appropriateness, fluency, grammar, vocabulary and pronunciation. Objectified scoring (Bachman, 1990) can be used by teachers with little specialized training and by highly trained teachers who prefer evaluation which is consistent and easy to use. For most teachers, objectified scoring is a practical alternative. However, it is possible to lose perspective of the overall performance. Even on a speaking test with objectified scoring, it is good to indicate a very general impression of a student's performance. This can be done simply by an indication that the person is "high", "mid", or "low". The general rating can verify the teacher's objectified score (Bachman, 1990).

2.3.8 Further considerations for scoring
There are three important suggestions on scoring. One is to use a scoring sheet. At the left raters can number the test item. Next to the test number is a short version of the cue. At the right are at least three boxes for raters to check- the first for 2-point answers, the next for 1-point responses, and the next for "0" or unacceptable answers (Bachman, 1990).

Another suggestion is to score the speaking test immediately if possible. Usually the scoring of a speaking test is more accurate when it is done during the process of the test itself. So if they feel comfortable testing and scoring at the same time, it is recommended that teachers handle both together. Usually, however, it is difficult for teachers to handle both. The alternative method is to determine the score immediately after the test has been administered (Jones, 1977). Furthermore, an interviewer should not be seen making notes about an interviewee's performance, either during the interview or any other time (Hughes, 1989). If the examiner is making notes during the test, it can distract the examinee and create unnecessary anxiety (Nagata, 1995).

The third suggestion is that if the resources are available, the ideal method is to have an examiner and a scorer present during the test. The examiner can administer the test, while the scorer, located in a place so that he or she cannot easily be seen by the examinee, can record the information for the score (Bostwick and Gakuen, 1995). Hughes (1989) also recommends that a second tester be present for an interview. This is because the difficulty of conducting an interview and keeping track of the candidates' performance.

2.4 Teachers' perceptions
It is generally perceived that oral testing is difficult (Jones, 1977) and it is a perplexing problem for many language teachers (Nagata, 1995). Li (1998) found that the main obstacles cited by Korean English teachers included a lack of effective and efficient assessment instruments. According to him, the Korean teachers found it disconcerting that there were no prescribed, ready-made assessment tools for communicative competence. Korean teachers also found it difficult to balance content and language when scoring an oral exam (Myung-sook, 1995, cited in Li, 1998). Some obstacles had little to do with pedagogical issues, but included large classes and a lack of time to conduct speaking assessments (Min-ju, 1995, cited in Li, 1998).

It seems that teachers need to have assistance and encouragement in trying communicative assessment. The accurate measurement of oral ability takes considerable time and effort to obtain valid and reliable results. Nevertheless, where backwash is an important consideration, the investment of such time and effort may be considered necessary (Bachman, 1990).

This chapter has discussed the literature which supports the rationale and validity of using communicative speaking assessment and shows the practical constraints in conducting communicative speaking assessment. Communicative speaking assessment can be difficult and expensive. However, teachers should always remember that it is not impossible (Bachman, 1990). In any attempt to improve education, teachers are central to changes (Frymier, 1987). Teachers are the end users of a classroom innovation. Teachers' perceptions are a crucial factor in the ultimate success or failure of that innovation (Kelly, 1998). Therefore, the study investigated Korean Junior Secondary School English teachers' perceptions of speaking assessment.

The methodology of the study will be outlined in the next chapter.

page 2 continued