banner

December 2008
Volume 12, Number 3

Contents  |   TESL-EJ Top

Testing Spoken English for Credit within the Indian University System

Shreesh Chaudhary
Indian Institute of Technology, Madras
<shreeshhotmail.com>

Abstract

Courses in Spoken English (SE) are yet to be acceptable in Indian universities because conducting session-end tests in SE is assumed to be logistically difficult and academically problematic. This article argues that it need not necessarily be so; session-end tests can be conducted just as in other courses. With voice recording, preferably a computer network in a local area, this can be done relatively easily, meeting all the demands of a standard test with respect to validity, reliability and ease of administration and evaluation. The article begins with a description of some reasons behind the unpopularity of SE tests, particularly at universities, and then talks about the features of a standard academic test. Finally, it reports the results and organization of a session-end test for a credit course in SE at the Indian Institute of Technology, Madras. This article also defines the items to be tested in a course in SE, and argues that there is greater need to test the features of continuous speech than those of the sounds in isolation. Tempo, phrasal pause, and word-stress deserve greater attention than segmental features. Recorded speech, particularly on the computer, can be as easily evaluated and stored for verification as a written text. Use of headphones can facilitate mass recording without compromising privacy, as required in an academic test. Testers can use more than one set of questions to test the predetermined aspects of SE.

Introduction

One of the continuing puzzles of English language teaching in India is that spoken English is not a part of the teaching and testing curriculum in schools and colleges. English has, strictly speaking, never been only a library language in India. In a few domains, English has also been a spoken language, and its mispronunciation and miscomprehension has attracted and continues to attract criticism. Its "good" use also brings opportunities and success. Yet, English is and has usually been taught as if it were a dead language. Teaching and testing reading and writing skills as well as vocabulary and grammar is emphasized, but little or no attention is paid to its sounds and diction. A mistake of pronunciation does not attract as much unwelcome attention as a mistake of spelling does.

During the colonial period, the British did not teach or test speaking in English, perhaps, because they hardly required Indians to speak it. They did, however, require clerks to write, copy and translate letters, minutes, resolutions and a variety of other written English documents. They made several copies of what they wrote, keeping some in India and sending others home by different vessels, hoping that at least some copies would reach the home office. Emphasis was placed on good handwriting, correct spelling, use of various marks of punctuation, use of lower and upper case letters, and other conventions of writing. The bias for teaching written English at the cost of everything else continues. The new curriculum recommended to the country by the University Grants Commission in 1989 is hardly "new" in this sense. (For details of the recommended curriculum, see Sethi, 1989.)

It is not that spoken English (SE) was never taught and tested. In the early nineteenth century, when public examinations began in Calcutta, examinees were given verses, globes, etc., from which they were supposed to read aloud to the examiners and then answer questions based thereon. See Sinha (1978) for some accounts of these oral and public examinations in Calcutta in the first half of the nineteenth century. One, for instance, is reproduced below.

The Calcutta School Society set a pattern of examinations of the students studying in the different seminaries of Calcutta. Reference is made in a local Bengali paper dated 8th March 1823 to one such examination held in the house of Raja Gopi Mohan Deva, where David Hare and Gour Mohan Vidyalankar acted as examiners. Mr Larkin himself conducted the examination. The boys fared very well. Among them, Shri Hara Mohan Babu, Shri Kshetra Mohan Mukhopadhyaya, Sri Rupnarain Dey, were examined in geography with English globe, and they recited English poems to the satisfaction of the examiners . . . . (Sinha, 1978, pp. 35-36)

Examinees were asked questions in English relating to the globe, and they then had to answer these questions orally referring to the globe wherever necessary. Questions were likely to have answers referring to location, direction, route, and geographical phenomena. Recitation of poems was evaluated for delivery and diction, which meant correct pronunciation of individual sounds in a word with proper stress and intonation.

But a major obstacle in teaching and testing SE, especially in second/foreign language situations, is the absence of an accepted[1] list of items to be taught and tested. "Speaking," as Harris (1977, p. 81) observes, "is a complex skill requiring the simultaneous use of different abilities which often develop at different rates . . . . Five components are generally recognized in analyses of the speech process." Harris lists them as follows:

  1. Pronunciation, including segmental features, vowels and consonants, and the stress and intonation patterns
  2. Grammar
  3. Vocabulary
  4. Fluency
  5. Comprehension

Of these, pronunciation is the most difficult to assess, arising partly from the theoretical aspects of the subject, explained in the following section. But, in good part, difficulty in testing SE is due also to logistics--the practical aspect of testing.

Testing Spoken English

There is no generally agreed upon set of features to look for while assessing pronunciation. In assessing writing, for instance, most examiners would agree that, besides content, they look for correct spelling, correct use of conventions of punctuation, small and capital letters, gaps between words and other aspects of conventions in writing. There does not seem to be such a generally agreed and accepted list of assessable features in pronunciation. As a result, the teaching and testing of SE is hampered.

The practical aspect has to do with the administration and validity of tests. Written examinations can be administered to hundreds of people at the same time without requiring different exam forms, and examinees cannot help or hinder the other examinees if they do their own work. This may not be the case with the testing of SE. The speaking module of international tests like Cambridge University's International English Language Testing System (IELTS) is administered to examinees individually, which makes the process expensive and time-consuming. It is generally thought to be impossible to administer tests in SE to large groups of people at the same time. Additionally, it is possible for examinees to help or hinder the performance of other examinees, even if they concentrate only on their own work. Because answers are oral, hundreds of people speaking simultaneously can reduce the classroom to a cacophony of unassessable noise.

Regardless of the conditions set out above, such problems perceived in the design and administration of an SE test can be seen as imaginary. After having administered such tests to undergraduate students for over a decade, both the theoretical and logistical problems have been overcome. Other colleagues in other classes with some basic equipment could also administer this test.

Test Design

The test is given at the end of a course in spoken English. The course teaches the skills of seminar presentation and some aspects of pronunciation of English, such as tempo, word stress, pauses, long vowels, and fricative consonants. The following are believed to be relatively easily achievable and crucial for worldwide comprehensibility.

  1. Fluency and coherence in extended discourse, and
  2. Accuracy in pronunciation of some vowels and consonants, word stress, and, to an extent, rhythm.

(For the rationale of teaching and testing these items, see Bansal (1966), Chaudhary (1996) and Chaudhary (1997).)

For teaching some aspects of English pronunciation, Chaudhary (1993) is used. A copy of the test form is given in Appendices I and II. Question 1 is intended to register the examinees' identity and to test their pronunciation of numbers and abbreviations in their roll numbers. At this institute, the code for the branch of engineering is prefixed to the roll number. The roll number of a student admitted in the mechanical engineering branch of the bachelor of technology programme, for instance, may be ME02435.

Questions 1, 2, and 3 involve discourse on familiar topics. A typical answer is likely to have a variety of vocabulary items and sentences, and may help test fluency and coherence.

Question 4, in the manner of an older version of the speaking module of the IELTS, tests elicitation skills and pragmatic competence of examinees in spoken English. In this part, it is not the examinee who answers questions, instead, he or she asks questions on a given topic. Many speakers of English in India are often found to have good control of sentence grammar, yet in asking questions they do not always and strictly follow the syntactic constraints of the English language. Thus, rather than asking, "Has the taxi arrived?" they often ask, "The taxi has arrived?" with a rising tone. Such interrogative sentences can be confusing for the listener. The course, therefore, aims at teaching and testing such elicitation skills. A conventional Indian course in SE pays little attention to teaching how to make requests, offer and ask for help and suggestions, or apologise and accept apologies. Clearly, these pragmatic skills are useful and necessary for the learner. During the pragmatic component of the course, this aspect is also highlighted and taught. Question 4 helps test this elicitation skill, which may be considered a subset of pragmatic skills. In a polite English request, the form of sentence is often like that of a question. The intention of the question is to test the extent of knowledge and skill in this aspect of SE.

The topics for questioning, given in Appendix II, are written on strips of paper, and each examinee chooses one strip. It is a blind draw, and usually no changes are allowed. Examinees are given about a minute to think about the topic before they begin asking questions. Since the examinees do not have any one to answer them--they are only talking in the recording mode on their desk in the language laboratory--they are asked to pause for a few seconds after every question. Besides testing the range of grammar and vocabulary, this also helps test the examinee's fluency and accuracy with regard to the discourse of a very formal kind.

Question 5 tests fluency and accuracy in relation to discourse on a relatively unfamiliar topic. The examinee may have to speak about a typical Indian wedding, or about a place of tourist interest, for example. Examinees are given a minute to think before they are asked to record their answers. In addition to testing the usual linguistic and phonetic features of spoken discourse, this question also tests the examinee's presentation skills, the teaching of which is one of the objectives of the course. It is given substantial attention through practice and systematic observation and feedback during the course.

Question 6 is designed so that it tests the pronunciation of certain consonants, for example, /f/, /v/, /z/, /w/, and certain long vowels, for example, /i:/. /ei/, /a:/, /u:/, which are supposedly difficult for many Indian learners. (See Bansal (1966), Wells (1982), Chaudhary (1989) and references therein for details.) Mainly this question is intended to test examinees' proficiency with regard to the "correct" word stress, which is crucial to global intelligibility of speakers. Thus, the questions are so designed that all items mentioned in 1 and 2 and taught during the semester are tested.

Test Administration

The test is administered in the Multi Media Laboratory of the Department of Humanities & Social Sciences of the Indian Institute of Technology Madras. The laboratory has typical facilities for recording and listening, and can hold up to 30 examinees at a time.

Before the test, during class time, examinees are made familiar with the procedure and constraints of an oral examination of this kind. They are, for instance, told that all examinees will have to answer the questions in the given order and each question within the given time. To ensure this, the recording time is centrally controlled.

For questions 4 and 5, the group of topics is offered by the administrator to all examinees, each of whom picks up a strip without any knowledge of its contents. Only rarely has there been a request from an examinee for the change of the topic (These requests were accepted, even though the possibility of any change was ruled out during the briefing session.) Care is taken in the selection of topics. Totally unfamiliar topics are avoided. Topics for question 5 are circulated only after topics for question 4 have been returned. For question 6, examinees read the given words aloud, trying for "correct" enunciation of vowels, consonants, and stress. Most examinees do not require the full-allotted time of five minutes for this question.

Because each examinee uses a headset and speaks into a microphone attached to a personal computer, examinees do not disturb other examinees' recordings, even though they sit side by side, and there is some noise in the laboratory. They are also advised not to speak too softly or too loudly. The recordings, as a result, are always of assessable quality.

The problem of examinees influencing one another's answers, voluntarily or otherwise, is also not a significant issue. Questions that are designed to test only fluency and coherence on a familiar topic may not be answered exactly at the same time to the same extent by all examinees. Moreover, only limited time is allowed within which answers must be recorded. Thus, examinees are under pressure to attend to their own tasks and record the answers in time.

For questions 4 and 5, which test fluency on relatively unfamiliar topics, examinees speak on different topics, and no examinee can, therefore, voluntarily or otherwise, influence another examinee's answer. The entire test takes less than half an hour and the answers are recorded on the hard disc of a personal computer. For evaluation, these answers are later copied into a folder on a compact disc. Student evaluation of the course has always shown very high levels of appreciation for the course and the test.

Evaluation

The examiner listens to the recordings on the CD one by one. Attention is paid to the amount of information, errors of grammar, flow of speech--that is, the absence of hesitation, repetition, false starts, and so forth--length of vowels and articulatory features of some "identified" consonants, such as /v/, /w/, and so on, as mentioned above.

Temp and pause length constitute another evaluation parameter. Many students, as many others in India, have a false notion of fluency, believing that an extremely rapid flow of speech indicates fluency. Given other features of an average Indian pronunciation of English, such as accurate pronunciation of consonants, inadequate length of long English vowels, and inappropriate stress on the word and sentence, very rapid flow can render the speech almost unintelligible. Therefore, these students are encouraged to speak at a slower tempo. Usually, a tempo between 270 to 300 words per minute (see Powers, 1985), or four to six syllables per second (see Usha, 1995), is considered ideal for intelligibility across cultures and disciplines. These tempos are used as benchmark for evaluation. It is possible to evaluate tempo of speech of each examinee with the help of a computer. But, to save time, examiner judgment is relied upon. Everything else being equal, a relatively slow and steady tempo is given high marks.

In evaluating pauses, attention is paid to their occurrence at the end of natural word groups, that is, at the end of phrases, clauses, and sentences. There is little difficulty in objective evaluation of speech with reference to this criterion.

In evaluating word stress, standard varieties are followed as a model. In cases where more than one stress pattern is possible, such as in research, donate, laboratory, and so on, which speakers of British English and many Indians generally pronounce one way, and speakers of American English, and many Canadians, Japanese, or others, pronounce another way, the advantage is given to the examinee. Unless the speech is considered as deviating far from the norm, the given stress pattern is accepted as "correct."

Conclusion

The test in this course also, then, becomes like tests in other courses at many colleges and universities in India: it is not difficult to administer, it is reliable and valid, gives the same question to all examinees, and evaluates their answers using the same criteria. It may profitably be tried elsewhere, and may help make teaching and testing of speaking a part of the curriculum and may give students a more complete English language education.

More importantly, it can give an additional medium for university examinations, that is oral, and not just written. It can also give another dimension to university examinations. Students need not attend a three-hours or longer written examination. Neither do the evaluators need to use subjective criteria, as in the evaluation of essay questions. Here, the test takes much less time and evaluation is done much more objectively.

Notes

[1] "Accepted" means accepted generally by English teachers worldwide.

About the Author

Shreesh Chaudhary is Professor in the Department of Humanities and Social Sciences, Indian Institute of Technology, Madras. He received his degrees at CIEFL (now EFLU) in Hyderabad, India. He has published many articles on English education in India, as well as on testing. He is the author of three books, including Some Aspects of the Phonology of Indian English (1989, Ranchi: Jayswal).

References

Bansal, R.K. (1966). Intelligibility of Indian English. Unpublished Ph.D. Dissertation, University of London.

Chaudhary, S. (1989). Some aspects of the phonology of Indian English. Ranchi: Jayaswal Press.

Chaudhary, S. (1993). A new course in better spoken English. New Delhi: Sterling Publishers.

Chaudhary, S. (1996, December). Speaking to the global village: Towards globally usable accents of English, CIEFL Bulletin 8(1), 29-58

Chaudhary, S. (1997, April). Testing spoken English as a second language. English Teaching Forum 35(2), 22-25

Harris, D. P. (1977). Testing English as a Second Language. Bombay: Tata-McGraw Hill & Co. Ltd.

Powers, D.E. (1985). A survey of academic demands related to listening skills. Research Report No. 5848, Princeton, New Jersey: Educational Testing Service.

Sethi, J. (Ed.). (1989). Report of the Curriculum Development Centre in English. University Grants Commission, New Delhi.

Sinha, S. P. (1978). English in India. Patna: Janaki Prakashan.

Usha, R. (1995). Linguistic aspects of listening by speakers of English as a Second Language, Unpublished Ph.D. Dissertation, Indian Institute of Technology, Madras.

Wells, J.C. (1982). Accents of English, Cambridge: Cambridge University Press.

Comment/view comments on this article. There are currently comments.

© Copyright rests with authors. Please cite TESL-EJ appropriately.

Editor's Note: The HTML version contains no page numbers. Please use the PDF version of this article for citations.