The Impact of TOEFL on Instructors’ Course Content and Teaching Methods

November 2019 – Volume 23, Number 3

Yangting (Tina) Wang
The University of Texas at San Antonio
<yangting.wangutsa.edu>

Abstract

Even though there are books that guide students in preparing for TOEFL exams, the literature on TOEFL test preparation courses at the university level is sparse (Huang, 2018). The current study aimed to fill the research gap by investigating the impact of TOEFL ITP assessment on teachers’ TOEFL course content and teaching methods. Five different TOEFL classes and four TOEFL instructors participated in the study. Following a qualitative research design (Glesne, 2015), fourteen classroom observations, fourteen student interviews, and four teacher interviews were collected. A combination of the Communicative Orientation of Language Teaching (COLT) observation scheme (Spada & Frohlich, 1995) and Miles & Huberman’s (1994) inductive approach were used to analyze classroom observations. The results from the observations were triangulated with student and teacher interviews. Findings indicated that teachers relied on the TOEFL textbooks to teach and practice test-taking strategies. They followed a “teach-model-practice-explain” structure during their instruction and spent the majority of class time conducting teacher-centered activity and student-alone activity. Additionally, teachers’ beliefs shaped their teaching methods, and their different beliefs led to students’ varied attitudes toward the TOEFL class. Pedagogical implications of implementing TOEFL classes at English language programs were discussed.

Keywords: TOEFL ITP test preparation, test washback, course content, teaching methods, Intensive English Program

Introduction

Test of English as a Foreign Language (TOEFL) is the English proficiency criteria for international students to gain university admissions in English-speaking countries. More than 35 million people all over the world have taken TOEFL tests (ETS, 2018). At the research site, international students who do not meet the English proficiency standard set by the university are “conditionally” admitted and need to take English language courses at the Intensive English Program (IEP). These students need to pass a TOEFL cut score of 550 in order to successfully graduate from the IEP, move their English proficiency hold, and start attending regular courses at the university. Since many universities require a cutoff score in TOEFL for admission; international students seek and need guidance in TOEFL test preparation. Under this circumstance, an increasing number of TOEFL test preparation courses are offered by various ELPs on and off university campuses in the United States. Little empirical research has examined the effects of such test preparation courses, and resources for teachers are much needed at the university level (Huang, 2018). This study aims to investigate the washback effects of TOEFL ITP on TOEFL classroom content and teaching methods. The purpose is to understand how TOEFL is taught in ESL classrooms and to provide suggestions on how teachers can improve their TOEFL teaching practices.

The TOEFL courses that the students took in the research site are to prepare them for the TOEFL Institutional Testing Program (ITP). To better understand this assessment, it is important to distinguish it from the popular TOEFL Internet-based test (IBT). The TOEFL ITP and IBT are both English proficiency test measures. However, they are different in three fundamental ways. First, ITP is institutional-based and ITP scores are only accepted by institutions internally (i.e. ESL program) to make placement, entrance, or exit decisions (ETS, 2018). However, IBT is accepted by most American universities externally as an English admission criterion which can be taken by anyone who wishes to be admitted into a university. Second, the TOEFL ITP is a paper-based test, whereas IBT is Internet-based. Third, the ITP test consists of three sections: listening comprehension, structure and written expression, and reading comprehension, whereas the IBT has four sections: speaking, listening, reading, and writing.

Literature Review

Language assessment has traditionally been centered around psychometrics and relatively less attention has been paid to test consequences (Green, 2013). However, there has been an increasing number of studies on the social-cultural aspects of assessment, more specifically, the impact of language tests (McNamara & Roever, 2006). Washback, according to Hughes (1989), focuses on the social dimensions of tests. It refers to the effects of a test on language learning and teaching. Messick (1996) defined washback as “the introduction of a test influences language teachers and learners to do things they would not otherwise do that promote or inhibit language learning” (p. 241). Washback can be either negative or positive (Taylor, 2005). Positive washback brings beneficial teaching practices, whereas, negative washback is when the test content and format cover a narrow range of language ability and, thus, restrict the teaching situation.

Washback effect is complex and context-dependent (Green, 2013) and not fully developed to date. Although ample studies have attempted to examine the washback of tests on teachers’ course instructions, empirical research on the washback effects of different tests and in various contexts is still underexplored (Barnes, 2016). The current study, thus, investigates the washback effects of TOEFL at the university level IEP in the Southwest United States. Messick believed that “a test might influence what is taught but not how it is taught” (p. 2). According to Barnes (2016), many researchers have demonstrated that tests can influence classroom content but not the teaching methods, whereas, some believed that tests can bring about change in both content and methodology. The two aspects pertinent to the study, washback on teachers’ course content and washback on teaching methods, are discussed in the following sections.

Washback of Test Preparation Courses on Teachers’ Course Content

Some studies investigated the mediating effect of TOEFL test materials on teachers’ course instruction. Hamp-Lyons (1998) discussed the problems with TOEFL test preparation textbooks. She stated that the textbooks were designed for autonomous learning which many international students are not capable of doing. And teachers cannot effectively use the textbook either as it only consists of practice tests and no syllabus, activities, or lesson plans like the regular language classroom textbooks. Wall and Horák (2008) investigated the impact of the new change of the TOEFL test, TOEFL IBT, on six teachers’ preparedness on TOEFL classroom instruction in Central Europe and Eastern Europe. They found that teachers’ knowledge about the new TOEFL was limited. Additionally, teachers relied greatly on the new test-preparation materials and commercial TOEFL preparation publications in their course instruction (Wall & Horák, 2008; 2011). Similarly, Barnes (2016) observed both TOEFL IBT and general English classes by four teachers in Vietnam and found that TOEFL IBT textbooks play an important role in teachers’ course instructions. In another study investigating effects of the Hong Kong Certificate of Education Examination (KHCEE) on secondary school teachers, Cheng (1999) found that the teaching materials used were exam practice books tailored to prepare for the KHCEE. After the test reform on KHCEE, the content of the teaching materials was also changed to meet the exam format.

Washback of Test Preparation Courses on Teaching Methods

Other studies focused on the washback effects of the TOEFL test on teaching methods. Alderson and Hamp-Lyons (1996) collected data from teachers’ interviews and classroom observations. Both teachers’ TOEFL preparation classes and non-TOEFL regular ESL classes were observed. The differences in teaching methods implemented in the two different class types were identified based on the number of turn-takings, test-taking time, teacher talk, frequency of laughter, pair work activity, and metalanguage use. The results indicated that test preparation classes included more test-taking strategies and had limited interactive activities in comparison to the non-TOEFL classes. Similarly, Barnes (2016) found that the majority of the four teachers focused primarily on giving instructions and having students conduct individual practice. Moreover, there were fewer opportunities for students to conduct pair or group work for the TOEFL class than the general English classes.

Additionally, it was found that the washback effects on teaching methods change from one teacher to another (Spratt, 2005) and that the quality of instruction depends on teachers’ attitudes and beliefs (Alderson & Hamp-Lyons, 1996; Watanabe, 2004). For example, Alderson and Hamp-Lyons (1996) found that teachers resented the TOEFL class, and their negative attitudes led the class to become less interesting. They further indicated that TOEFL class instruction highly depended on individual instructors’ teaching and there may be no uniform method for teaching TOEFL. In Watanabe’s (2004) study on teacher perceptions of the National Center for University Entrance Examinations in Japan, he found that there were several teacher factors that mediated the washback effect. Teachers had biased assumptions about what should be emphasized in class. Moreover, if teachers were not familiar with how to teach certain language skills, they were less likely to teach them in class. These teacher factors may hinder beneficial washback (Watanabe, 2004).

Since research regarding what is taught in a TOEFL preparation class and how is still ongoing and the washback effects of TOEFL test preparation on teachers’ classroom practices are limited, the current paper fills the gap by investigating the effects of TOEFL ITP on teachers’ course content and teaching methods. The study offers pedagogical implications to ESL instructors at similar ELPs on how to effectively teach TOEFL. The research questions are as follows:

1. What is taught in a TOEFL class?
2. How is TOEFL taught in a class?

Method

Study Context

This study is part of a larger study that investigates the test washback on TOEFL preparation courses. It took place at an IEP in a public university in the Southwest United States. Data were collected in the 2018 spring and fall semesters. The IEP is a relatively small local program, consisting of five ESL instructors, one director, one coordinator, and around 40 students. Four types of classes are offered at the IEP. There are three regular ESL classes: writing/grammar, reading/vocabulary, and oral communication; and one test-preparation class, which is the TOEFL class. The three non-TOEFL classes are each 120 minutes long and offered three times a week. Each of the three non-TOEFL classes offers five levels to accommodate five different proficiency level students. The TOEFL class is shorter, it lasts 80 minutes and is held twice a week. Because TOEFL lessons were more advanced and were considered not suitable for beginning level students by the IEP, students in levels 1 and 2 take an American culture class instead of the TOEFL class. Therefore, only levels 3, 4, and 5 students take the TOEFL class. Because of the number of students enrolled, there is only one TOEFL class per level. To maintain visa status in the United States, all international students are required to take both the TOEFL/American culture class and the non-TOEFL classes each semester. Class size is 15 students maximum.

Participants

A pilot study was conducted at the research site in spring 2017 where one level 5 TOEFL class was observed eight times and five students were interviewed. A few interview questions were added in order to gain a more comprehensive understanding of student attitudes. Participants in the current study included one male teacher Mr. Z and his two classes in spring 2018; three female teachers, Ms. T, Ms. E, Ms. C, and their three classes in fall 2018. The four instructors all come from the U.S., speak English as their first language, have more than one year of teaching experience, and hold a master’s degree in Teaching English to Speakers of Other Languages (TESOL). The reason for collecting two semesters of data is to be able to compare different teaching practices among teachers. In spring 2018, a total of 26 students were in Mr. Z’s class. They are between the ages of 18 to 40 with a mean of 24 years old. Among them, 12 were male and 14 were female. The student demographic is similar to the ones in the fall semester. Students were told that their participation is completely voluntary, will not influence their grades, and that their information will be kept confidential.

Data Collection

Because many previous scholars used qualitative methodology in investigating the washback effect on teaching (i.e. Barnes, 2017; Green, 2006; Wall & Horák, 2008; 2011) and since qualitative studies offer a rich account of studied phenomena, the current study follows a qualitative research design (Glesne, 2015) to describe and understand TOEFL instructors’ teaching practices. Wall and Alderson (1993) indicated the importance of gathering empirical data from direct classroom observations. Therefore, instead of solely collecting data through questionnaires and interviews (Hughes, 1988), a wider range of qualitative data including classroom observations, student interviews, and teacher interviews was collected.

Classroom observation. As shown in Table 1, in spring 2018, classroom observations were conducted eight times with one teacher in order to provide in-depth understanding regarding TOEFL instruction at the IEP. The fall semester observations aimed to offer a more complete picture of teachers’ instruction. Therefore, all of the TOEFL levels (level 3, 4, and 5) taught by three different instructors were observed. Classes were audio recorded upon teachers’ consent. Following Glesne’s observation guidelines (2005), the observer tried to take notes of everything that happened in the classroom. To not interfere with the setting, the observer had no interaction with the participants. Field notes were typed during the observations. Detailed memos were then added after each observation (Emerson, Fretz, & Shaw, 1995).

Table 1.Data Collection in Spring and Fall 2018.

	N (Spring 2018)	N (Fall 2018)
Teachers observed	1 (Mr. Z)	3 (Ms. E, Ms. C, Ms. T.)
Classroom observations	8	6
Classes observed	one TOEFL Level 4 (four times) one TOEFL Level 5 (four times)	one TOEFL Level 3 (twice) one TOEFL Level 4 (twice) one TOEFL level 5 (twice)
End-of-semester interviews	6 (3 for each level)	6 (2 for each level)

Teacher interviews. To ensure that the study’s interpretation and preliminary data analyses results were appropriate and accurate, by the end of the semester, the four instructors were interviewed about their perceptions toward the class. The following questions were asked, “What do you think about the TOEFL class?” “Do you teach strategies based on the textbook?” “What is the structure of your TOEFL class?”

Student interviews. Face-to-face individual interviews were scheduled with the six students who showed interest in participating in the study by the end of the spring semester. By the end of the fall semester, the first two students in one of the three levels who showed interest were invited for the interview. Table 2 presents the background information of the 12 student interviewees. The interview consists of seven open-ended questions (See Appendix 1). Based on participants’ responses, some follow-up questions were asked during the interview. Each interview lasted 20-40 minutes.

Table 2. Information about Student Interviewees.

Pseudonym	Age	Country of origin	Time studied at the IEP (months)	Levels at the IEP	Semester (teacher)
Chris	26	China	36	5	Fall (Mr. Z)
Eric	27	China	36	5
Maria	38	Columbia	24	5
Hannah	21	Japan	7	4
Monika	21	Japan	7	4
Sonny	30	Korea	18	4
Lynda	24	Taiwan	4	5	Spring (Ms. T)
Asset	19	Kazakhstan	7	5
Von	21	China	4	4	Spring (Ms. E)
Moka	21	Japan	4	4
Marie	47	Mexico	7	3	Spring (Ms. C)
Anna	30	Mexico	10	3

Data Analysis

To investigate TOEFL teachers’ classroom content and teaching methods, the Communicative Orientation of Language Teaching (COLT) observation scheme was used (Spada & Frohlich, 1995). Five categories from the scheme were employed: time (percentage of average class time spent in each segment), participant organization (who is organizing the participation?), activity type (e.g. teacher-student, student-student, role play, individual practice, discussion, etc.), content (what is being taught?), and materials (what teaching materials are used?). Additionally, Miles and Huberman’s (1994) inductive approach and Emerson et al.’s (1995) open and focused coding were used for qualitative analysis.

To answer the first research question on the impact of TOEFL ITP on what is taught in the classroom. Content and materials were analyzed based on the COLT scheme. The average number of strategies, examples, and practice conducted in each teacher’s class were calculated based on classroom observations. This result was triangulated with students’ and teachers’ perceptions of classroom content. Two themes emerged from student interviews: language skills and test-taking strategies. Examples were coded as open codes to describe and demonstrate the themes. Additionally, based on the teacher interviews, two different perceptions were identified: one is “positive,” the other is “boring.” Examples were analyzed as open codes to illustrate teachers’ varied perceptions.

For the second research question regarding TOEFL teaching methods, participant organization, and activity type were coded from the COLT scheme to investigate classroom interaction patterns and instructors’ teaching characteristics. Activity type was divided into teacher-centered activity, teacher-student interaction, student-student interaction, and student-alone activity. Based on participant organization, the teacher-student interaction was further divided into two main aspects “the teacher asks students’ questions” and “students ask teacher questions.” Teachers’ four major teaching components, “teach,” “model,” “practice,” and “explain,” were identified based on all lesson recordings and field notes. The average percentage of time spent teaching these four components for the four teachers were calculated. As with the first research question, the results from classroom observations were also triangulated with student and teacher interviews. Instructors’ teaching characteristics such as “clear instruction,” “interactive activities,” and “drills” were identified as focused codes to summarize students’ interview transcripts. Specific examples were used to demonstrate the focused codes. Teachers’ perception of how they teach TOEFL class was summarized.

Results

RQ1: What is Taught in a TOEFL Class?

Instructors of the same level course use the same textbook as their teaching materials. TOEFL level 3 teachers use Longman introductory course for the TOEFL test (Phillips, 2004), level 4 teachers use The complete guide to the TOEFL test PBT edition (Rogers, 2011), and level 5 teachers use a mixture of the level 4 textbook and reading practice examples from the TOEFL IBT.

Table 3 shows a summary of the content and the average number of strategies, examples, and practice conducted in each teacher’s class. Classroom observations demonstrated that all four teachers created and used their PPT slides to teach test-taking strategies based on the TOEFL textbook. Teachers then used the exercises from the textbook to practice the new strategies. For example, one of Mr. Z’s classes on listening comprehension (Observation 2, level 4, 3/6/2018) taught students’ three strategies from the textbook respectively: lesson 14 “anticipate questions,” lesson 15 “overview different types of questions,” and lesson 16 “detail questions including information questions and inference questions.” To teach lesson 14 “anticipate questions,” he told students that they needed to guess the type of questions before listening based on the multiple-choice options given. Students need to first predict whether the questions will ask about a topic, a setting, or a person, and then choose the correct questions listed that matched the multiple-choice answers. For each lesson/strategy, he first modeled an example and then asked the students to do the corresponding exercise in the textbook, exercise 14, on their own.

Another example comes from Ms. C’s class on structure (Observation 1, level 3, 10/23/2018), she taught skills 46, 47, 48, and 49 respectively in her lesson, modeled one example for each, and had students practiced the related exercises. Because the class content is exercise after exercise with no break in between, when it came to the last exercise (skill 50), some students did not wish to continue anymore. Excerpt 1 illustrated a conversation that happened between two students and Ms. C.

Excerpt 1 (10/18/2018)

S1: No

S2: No more, teacher

S3: My brain stopped

Ms. C: Come on. I know it’s a lot. I’m sorry. But this is the last one.

After knowing that Ms. C was going to teach skill 50, S2 said, “no more” indicating that she had enough exercises and did not wish to continue the practice. S1 and S3 also rebelled against the teacher. S3 commented that he felt as if his brain had stopped working because of all the exercises. The teacher comforted the students by saying that they had only one skill left for the day. The excerpt shows that when teachers use the textbook to teach and practice strategies one after another, the class became drill-oriented and students felt fatigued in the class.

Table 3. A Summary of TOEFL Class Content based on Classroom Observations

Per class	Mr. Z (Level 4 & Level 5)	Ms. T (Level 5)	Ms. E (Level 4)	Ms. C (Level 3)
Content focus (classes observed)	Listening comprehension (2) Reading comprehension (4) Written expression (2)	Listening comprehension (1) Reading comprehension (1)	Structure (2)	Structure (2)
No. of strategies taught (average)	3	2	3	4
No. of examples modeled (average)	2	2	2	4
No. of sections of practice given (average)	3 (32 items)	3 (33 items)	2 (18 items)	4 (40 items)

Students’ perceptions of what is taught in the TOEFL class. Students’ perceptions support findings from classroom observations that teachers focused on the language skills that were relevant to what was assessed in TOEFL ITP. When being asked about what specific language skills are taught in class, all of the six students in Mr. Z’s class indicated that the skills that Mr. Z focused on are grammar, listening, reading, and vocabulary, and that Mr. Z spent little time on speaking or writing practice. Four of the students expressed that they wished to practice speaking in TOEFL class. For instance, Hannah said, “It’s more focus on grammar and reading listening, but no writing or speaking. I think that we can work more speaking.”

In terms of grammar skills, Maria said, “There’re grammar skilling they focus on different things for example, the difference between make and do, the different proposition we have to use. It’s very specific, the book and work and that specific topics are very important for English improvement.” Monika appreciated the fact that she learned English grammar. She said, “I can learn the grammar by English, such as singular, plural, before I never knew this. I learned lots of grammar points in TOEFL.” For vocabulary, Sonny said, “yes we worked a lot with vocabulary, yes very good.” Hannah talked about how she learned academic vocabulary from taking the class “I can learn formal things or business things from TOEFL, that is what I don’t learn from normal life. Because in my life, I don’t use the TOEFL words, or the scientific words. It’s kind of good chance to learn specific words.” In regard to listening, Monika said, “we worked with different conversation and we tried to response to different questions.”

Similar to Mr. Z’s class, students indicated that Ms. T, Ms. E, and Ms. C’s classes focused mainly on teaching grammar, reading, and listening skills. Marie discussed how Ms. C’s class centers on grammar, “all the time we are just grammar, grammar, and grammar, we need more speaking.” Though the class focused on grammar skills, Von indicated that she did not learn much from the class because the grammar skills taught in the class are the ones she already learned back in China. She said, “actually, I didn’t really learn much from the class. The grammar the teacher taught, we already learned them in elementary and secondary school in China.”

Based on the interviews, students reported that what they learned most in the class were TOEFL ITP test-taking strategies. This verified that the content of the TOEFL textbook focused on strategies. Different from the fall semester teachers, Mr. Z sometimes taught strategies that were not from the textbook. For example, Chris commented that Mr. Z taught him how to time himself for the different test sections and how to take notes during listening. He said that these tips were not from the textbook but had helped him a lot. On the other hand, the other teachers based their strategies entirely from the textbook. Anna in Ms. K’s class said, “The strategies are the same from the textbook. She tried to explain but it’s the same with the textbook.” It seemed that the strategies from the textbook were not enough for students to prepare for TOEFL ITP. She added, “if we want to improve our scores, we need more strategies.”

Additionally, among the 16 student interviewees, reading strategies were commented on the most. They talked about how reading strategies helped them improve their reading speed and accuracy (See examples [1], [2], [3], [4]). Both Chris and Lynda said that they learned how to read and answer the reading questions faster ([1]). Moreover, Sonny indicated how she loved learning different strategies to help her improve test scores ([4]). She trusted the teacher to accurately teach her how to understand the purpose of reading and identify different types of comprehension questions.

[1] Lynda: “What I learned is TOEFL skills. To answer faster in reading.”

[2] Asset: “I learned a lot of strategies in TOEFL test. We learn about critical thinking. Strategies like reading articles, research work, skimming, looking for key words.”

[3] Maria said, “The teacher taught us to try to find the answer in the passage, then to read all the passage and respond. Because time is limited, that was a good strategy.”

[4] Sonny said, “I love learning the strategies. Strategy is good for the score, it’s important. But if I don’t have the teacher, nobody knows about this. Reading tips, this sentence, inference question, purpose. where I have to read, where I need to focus. And I knew which type of question it is.”

Teachers’ perceptions of what is taught in the TOEFL class. Teachers’ perceptions supported findings from the classroom observations and student interviews that the majority of teachers felt that they “teach to the test” and what is taught in the class is based on the test strategies and practice materials that are from the TOEFL textbook. It also offers more information regarding how the TOEFL class is perceived by the teachers.

Mr. Z and the three teachers in the fall had different attitudes toward the course. Mr. Z’s attitude is very positive. He said, “TOEFL class is fun. I like teaching the class and test-taking strategies to students.” However, Ms. T, Ms. E, and Ms. C all expressed that “TOEFL class is very boring.” Ms. E indicated the reason why she believed that the class is not interesting. She said, “Because you ‘teach to the test,’ compare to my oral communication class, we have lots of time to do activities, move around, but this class is very boring.” Besides, Ms. E, Ms. C, and Ms. T all indicated the time constraint of the class ([5]). Her response indicated that, because of the time limit of TOEFL ITP class, she felt in a rush to cover all the materials. Ms. C’s  “rushing to cover all strategies” belief furthered supported the classroom observation in which the class is filled with drills and in one example, even when students said, “no more,” to the exercise, she still teaches one more strategy and convinces the students to do one more exercise before class ends. Ms. C indicated that she did not enjoy teaching the class because its primary purpose is for students to pass the test.

[5] Ms. C: “I felt restricted with the time and course curriculum that I had to talk about all the strategies and complete the entire book in one semester. If I don’t do that, students may complain that I did not cover the whole strategies before they took the test.”

RQ2: How is TOEFL Taught in a Class?

Participant organization and activity type of TOEFL classes. The classroom observations indicated that all four instructors teach their lessons using a comparable “teach-model-practice-explain” structure. In a typical lesson, the teacher starts with some housekeeping, which included chatting with students, giving announcements, and presenting the class agenda (Greetings). The teacher then started the instruction by using the PPT slides to teach one or multiple strategies from the TOEFL textbook (Teach). The teacher provided one to three examples from the textbook to model how to use the strategy to answer test questions (Model), then directed and started a timer for students to complete the exercises related to the strategy (Practice). Lastly, the teacher provided the correct answers and explained the incorrect responses to reinforce the strategy (Explain). Table 4 indicates the organizer of the different classroom activities and the average percentage of class time spent on the activities.

Table 4. Participant Organization and Activity Type

Participant organization as a percentage of class time (average class)	Mr. Z	Ms. T	Ms. E	Ms. C
Teacher-centered activity
Housekeeping/greetings	4.2%	3.8%	3.3%	2.7%
Teach strategies	28.6%	15.7%	42.8%	16.9%
Model strategies	7.1%	2.8%	22.4%	6.8%
Explain answers	8.6%	15.7%	12.5%	19.9%
Teacher-student interaction
T asks S questions	20.7%	6.4%	8%	11.6%
S asks T questions	5.7%	7.1%	2%	3.7%
Student-student interaction
Pair/group work	14.3%	0%	0%	0%
Student-alone activity
Individual practice	42.9%	50%	19%	35.6%

Table 4 shows that all four teachers spent about 40% (Mr. Z, Ms. T, Ms. C) to 80% (Ms. E) of their class time on average conducting the teacher-centered activity. They spent about 35.6 to 50% (Mr. Z, Ms. T, Ms. C) of the time conducting the student-centered activity, with the exception of Ms. E (19%). The reason Ms. E’s class had less time for individual practice is that she spent 80% of her class time lecturing and there was simply less time left for practice. Additionally, teacher-student interaction was mainly about asking and answering questions, and most of the time, the teachers led this type of interaction. Mr. Z in average had more teacher-student interaction (26.4%) than the instructors in the fall (6.4% to 11.6%) and he was the only teacher who allowed student-student interaction (14.3%).

Students’ perception of how TOEFL is taught. Students in Mr. Z’s class frequently reported that they liked Mr. Z’s teaching style because he gave clear instruction ([6], [7]). Unlike other TOEFL teachers they previously had who rushed through exercises, he spent time explaining the answer choices ([8], [9]). Additionally, he uses a variety of visuals ([10]), encourages pair/group work ([9]), and his class is interactive ([9]). The following quotes are examples from the interviews that demonstrated Mr. Z’s teaching characteristics.

[6] Monika: “he speaks English very clear and very easy to understand.”

[7] Eric: “Mr. Z takes time to explain answers and he explain it very well and clear.”

[8] Hannah: “I like the teacher, he gives us time to think a lot so we can think. I think it’s good. Sometimes, other teachers don’t give us opportunities to think, they just gave us answer. I don’t think that’s good.”

[9] Maria: “At the beginning, I didn’t like just go to the room and take the book and start to do the exercises. I didn’t like that. This class, the teacher showed us slides and he worked in a different ways, more interactive ways, not just take the book and do the exercises.”

[10] Sonny: “The teacher’s lecture is very clear…he explained about the sentence structure. and with the visualizing the drawing is very clear. Very helpful to understanding.”

[11] Maria: “very active during the class. He also helps us to work with our classmates so the classes were not boring…very dynamic class…Very nice very nice classes.”

As to the other teachers, students only had a few comments of appreciation. They emphasized more about how the teacher uses exercises to teach the class. Excerpt 2 indicated how Asset (Ms. T’s class) felt the TOEFL class was somewhat boring compared to the non-TOEFL/language courses because of the drills.

Excerpt 2 (11/1/2018)

Asset (Ms. T): “Sometimes the class is boring. The exercises are kind of boring and activities are also boring.”

Interviewer: “What kind of activities?”

Asset: “Activities like test, doing exercises. It’s like routine, you came to TOEFL class and you do the same thing every time.”

Interviewer: “What about other classes?”

Asset: “Other classes are okay. Like Oral Communication class, we have lots of speaking activities and playing activities like Kahoot.”

Students also indicated that there was little interaction in Ms. T, Ms. E, and Ms. C’s TOEFL class. Quotes [12], [13], and [14] from Marie, Moka, and Lynda showed that their teachers spent the majority of the time having students practice exercises. What Marie has shared is shocking: to maximize students’ practice time, Ms. C often told her students to “be quiet” and prohibited them from speaking with others ([15]). This showed that students had little opportunities to carry out conversations in class. Also, unlike Mr. Z, Marie said that Ms. C simply taught strategies to students. She did not make sure students understand these strategies and did not spend time to review them.

[12] Lynda: “Most of the time alone, doing exercise, on our own.” (Ms. T)

[13] Moka: “I just focus on what the teacher is saying. The majority of the time is doing exercise. It’s just like what we need to do, that’s it.” (Ms. E)

[14] Marie: “the teacher does not ask us whether we understand the information. She teaches us and that’s it. No review. I want review the strategies.”

[15] Marie: “The interaction is very low. We don’t interaction with the teacher. We only give answers. Because all the time, the teacher often say, ‘Be quiet! No speaking.’” (Ms. C)

Teachers’ perception of how TOEFL is taught. Instructors’ statements demonstrated that teachers’ beliefs shaped their teaching practices. Mr. Z said, “I hear students’ voices and I learn from them to see how I can improve the class to benefit students and I try to make the class interactive and fun. I gave time for students to do pair/group work so that they can work together to solve the problem.” As a result, Mr. Z’s class had more interactive activities and is more “dynamic” for students.

Ms. T, Ms. E, and Ms. C, on the other hand, had different beliefs. They all said, “Doing speaking activity will be a waste of time.” Ms. C further noted, “There is no time to do language-related activities because this class focuses on the test, and whether students’ scores can improve, not language skills. Students practice test-skills by doing exercises in class. Other classes, you focus on language skills and you can have activities build around it to improve their language.” The three teachers’ viewpoints led to a very test-oriented class. Students’ comments and classroom observation both support teacher interviews that the teachers did not have interactive activities in class when teaching TOEFL because they see it as “a waste of time.”

Discussion

The study used a qualitative research design to investigate the washback effect of TOEFL ITP on instructors’ course content and teaching methods at a tertiary level IEP in the United States. The results aligned with prior research findings and provided new insights on the impact of TOEFL ITP on teachers’ classroom content and teaching methods. Now I discuss each of the research findings and pedagogical implications in details.

For the first research question on what is taught in TOEFL class, classroom observation indicated that teachers relied on the TOEFL textbooks to teach and practice test-taking strategies. This aligned with previous research that the TOEFL strategies that the instructors used were based mostly from the textbook materials (Barnes, 2016; Wall & Horák, 2008; 2011). In terms of language skills, student interviews showed that all teachers focused on language skills such as grammar and reading that are most relevant to the test and spent little time on speaking and writing. Some students also reported that they are tired of learning grammar all the time and wanting to practice more speaking in class.

Furthermore, based on students’ interview responses, the knowledge that students learned the most in TOEFL preparation class is strategies. This finding was in line with Alderson and Hamp-Lyons’ results (1996) that TOEFL instructors focused more on strategies. The current results extended previous literature and provided more information on the different varieties of test-taking strategies that instructors teach based on student interviews. These include test-wiseness strategies and language learning strategies. Test-wiseness strategies are strategies which students use to take the test without knowing the knowledge of the content (Nikito, 2001). For example, Chris learned how to reasonably use their time in the different test sections which was termed by Nikito (2001) as time-using strategy. Students also learned language-learning strategies, which were used to improve their learning of a second language (Cohen, 1998). For example, Asset learned about skimming skills in reading and Sonny learned how to identify the purpose of an article, etc.

Lastly, teachers’ perceptions on the class content demonstrated that while one teacher believed that the TOEFL class was fun, the majority of the teachers felt restricted by the time. Most teachers believed that they “teach to the test,” and that the course was “boring.” Teachers’ attitudes toward the class support Alderson and Hamp-Lyons (1996) and Shohamy et al.’s (1996) results that the majority of the teachers disliked the time pressure in their classes and had negative attitudes toward teaching TOEFL.

In regard to the second research question on how TOEFL is taught. The classroom observations indicated that all four instructors teach their course using a “teach-model-practice-explain” structure. This structure is related to the procedural scaffolding model “teach, model, practice, apply” from Edevarria, Vogt, Short’s (2000) Sheltered Instruction Observation Protocol Model which was widely and successfully used for teaching K-12 English language learners in the United States. However, Edevarria et al. (2000) suggested that at the “practice” stage, students should work with their peers collaboratively to solve problems and then “apply” the strategies to their independent learning. In the TOEFL class, “practice” and “apply” stages were conducted together where students apply the strategy by practicing the corresponding exercises. Only Mr. Z conducted both peer activity and individual practice whereas the other teachers only had students completed exercise individually. The “teach-model-practice-explain” model, in general, seem to work for the population of adult ESL learners and for teaching TOEFL. However, all instructors should also consider including peer collaboration in their teaching to more effectively scaffold students’ learning.

Results were also in line with Barnes’ (2016) finding that teachers spent the majority of class time doing teacher-centered activities and individual practice. Additionally, results demonstrated that teachers’ beliefs greatly influence instructors’ teaching methods and that different teacher beliefs led to different washback outcomes for students. This finding extended Spratt (2005) and Watanabe (2004)’s results that washback effects of teaching methods varied across teachers and that teachers’ perceptions mediated washback effects. Because Mr. Z believed that the TOEFL class was fun and can be interactive he asked a lot of questions in class and encouraged pair work. Classroom observation demonstrated that there were more teacher-student interactions in Mr. Z’s class than other classes and there was also student-student interaction. And because he believed in improving students’ language learning in the TOEFL class, he took time to explain the answers clearly. As a result, students liked his teaching methods and believed it was “very nice” and “dynamic.” Teachers’ positive belief led to positive washback on students’ attitudes toward the class. On the contrary, since the three instructors in the fall believed that the TOEFL class time is too short, classroom observations showed that these instructors often ask students to do exercise after exercise with no break in between. And since interactive activities were “a waste of time,” students had no time to do peer work. Moreover, because the teachers believed that the goal of the class was only to improve student scores, they spent a large amount of time teaching test-taking strategies and language skills that are relevant to the TOEFL ITP and ignored the speaking activities and real-life English that students need. As a result, students felt that the class was “boring” compared to the other non-TOEFL/language classes. The washback of the TOEFL class on these teachers and their students in terms of their perceptions was negative.

Implications and Future Directions

Based on the two semesters’ classroom observations from four different TOEFL instructors and student interviews from twelve students, the following recommendations are provided for teachers and the English language programs to adopt in order to teach a more effective TOEFL class.

1. Make English language learning as the TOEFL class goal. Teachers should believe that the goal of TOEFL class is not only to help students improve test scores, but also improve students’ English language abilities. TOEFL class should not be seen as “something else,” teachers should embrace it as an integral part of the language program.

2. Spend time to explain. Teachers should think about how to more effectively use the limited time frame to teach strategies. Instead of rushing through exercises, teachers should take time to explain the incorrect answers to the exercises in an easy and clear way.

3. Make TOEFL class interactive and fun. Teachers should change their ideology that the TOEFL class is just for the test and for the scores, and it is supposed to be “boring.” Mr. Z is a great example that TOEFL class can also be fun. Teachers should make every effort to make the class interesting. They should not spend the majority of the class time doing teacher-centered activities (i.e. lecturing) and should not have students do multiple drills in one sitting; instead, they should allow more time for teacher-student and student-student interactions, promote pair and group discussions.

Additionally, while teaching the strategies, instead of only using PPT, teachers should use a combination of visual such as PPT slides, body language, and blackboard to engage students. If teachers all rely on the TOEFL textbook to teach, then the textbook needs to be revised as well to be more interactive in order to improve students’ learning experience. Apart from the three main recommendations, teachers should also consider other suggestions given by the students including practicing speaking skills in TOEFL class and learning more test-taking strategies that cannot be found from the textbook.

The study offers information on course content and teaching methods regarding TOEFL preparation courses and provides suggestions in teaching and engaging students in a TOEFL class. As Muñoz and Álvarez (2010) and Green (2013) pointed out, washback studies focused primarily on teachers and classroom practices, less attention has been paid to learners. Future study can also investigate the impact of TOEFL preparation courses on students such as student attitudes toward TOEFL courses, how their attitudes influence their learning motivation, TOEFL test scores, as well as language learning.

About the Author

Yangting (Tina) Wang is a Ph.D. candidate in the Cultural, Literacy, and Language program at the University of Texas at San Antonio. She specializes in language assessment and computer-assisted language learning. Her recent research projects include mobile-assisted language learning, validity of high-stakes assessment, and program evaluation on K-12 STEM education.

Acknowledgements

I would like to thank Dr. Becky Huang at the Department of Bicultural-Bilingual Studies at the University of Texas at San Antonio for her guidance on the earlier version of the manuscript. I also thank all the anonymous reviewers’ invaluable feedback. I am very grateful for the director, teachers, and students at the English language program to participate in the project.

References

Alderson, J. C., & Hamp-Lyons, L. (1996). TOEFL preparation courses: a study of washback. Language Testing, 13(3), 280–297. https://doi.org/10.1177/026553229601300304

Barnes, M. (2016). The washback of the TOEFL iBT in Vietnam. Australian Journal of Teacher Education, 41(7), 158-174.

Barnes, M. (2017). Washback: Exploring what constitutes “good” teaching practices. Journal of English for Academic Purposes, 30, 1-12.

Cheng, L. (1997). How does washback influence teaching? Implications for Hong Kong. Language and education, 11(1), 38-54.

Cheng, L. (1999). Changing assessment: Washback on teacher perceptions and actions. Teaching and teacher education, 15(3), 253-271.

Cohen, A. (1998). Strategies in learning and using a second language. New York: Addison Wesley Longman Inc.

Echevarria, J., Vogt, M., & Deborah, J. Short (2000). Making content comprehensible for English language learners: The SIOP model. London: Pearson.

Emerson, R. M., Fretz, R. I., & Shaw, L. L. (1995). Writing ethnographic fieldnotes. Chicago: University of Chicago Press.

ETS (2018). About the TOEFL ITP assessment series. Retrieved from www.ets.org/toeflitp

Glesne, C. (2015). Becoming qualitative researchers. An introduction (5^th edition). London: Pearson.

Green, A. (2006). Washback to the learner: Learner and teacher perspectives on IELTS preparation course expectations and outcomes. Assessing Writing, 11(2), 113–134. https://doi.org/10.1016/j.asw.2006.07.002

Green, A. (2013). Washback in language assessment. International Journal of English Studies, 13(2), 39-51.

Hamp-Lyons, L. (1998). Ethical test preparation practice: The case of the TOEFL. TESOL Quarterly, 32(2), 329. https://doi.org/10.2307/3587587

Huang, B. H. (2018). PD in testing preparation for university ESL. In J. I. Liontas, T. International Association and M. DelliCarpini (Eds.), The TESOL encyclopedia of English language teaching (pp.1-6). Hoboken, NJ: John Wiley & Sons. https://doi.org/10.1002/9781118784235.eelt0886

Hughes, A. (1988). Introducing a needs-based test of English language proficiency into an English-medium university in Turkey. In A. Hughes (Ed.), Testing English for university study. ELT Documents 127 (pp. 134-146). Modern English Publications in association with the British Council.

Hughes, A. (1989). Testing for language teachers. Cambridge: Cambridge University Press.

McNamara, T. F. & Roever, C. (2006). Language testing: The social dimension. Oxford: Basil Blackwell.

Messick, S. (1996). Validity and washback in language testing. Language Testing, 13(3), 241-256.

Miles, M. B., Huberman, A. M., Huberman, M. A., & Huberman, M. (1994). Qualitative data analysis: An expanded sourcebook. Sage.

Muñoz, A. P., & Álvarez, M. E. (2010). Washback of an oral assessment system in the EFL classroom. Language Testing, 27(1), 33–49. https://doi.org/10.1177/0265532209347148

Nitko, J. (2001). Educational assessment of students. New York: Merrill Prentice Hall.

Phillips, D. (2004). Longman introductory course for the TOEFL test, the paper test. New York City, NY: Pearson Education ESL.

Rogers, B. (2010). The complete guide to the TOEFL test: PBT edition. Boston, MA: Cengage Learning.

Shohamy, E., Donitsa-Schmidt, S., & Ferman, I. (1996). Test impact revisited: Washback effect over time. Language testing, 13(3), 298-317.

Spada, N., & Fröhlich, M. (1995). COLT–communicative orientation of language teaching observation scheme: Coding conventions and applications. National Centre for English Language Teaching and Research.

Spratt, M. (2005). Washback and the classroom: The implications for teaching and learning of studies of washback from exams. Language teaching research, 9(1), 5-29.

Taylor, L. (2005). Washback and impact. ELT Journal, 59(2), 154-155.

Wall, D., & Horák, T. (2008). The impact of changes in the TOEFL examination on teaching and learning in Central and Eastern Europe: Phase 2, Coping with change. ETS Research Report Series, 2008(2), i-105.

Wall, D., & Horák, T. (2011). The impact of changes in the TOEFL® exam on teaching in a sample of countries in Europe: Phase 3, The role of the coursebook phase 4, describing change. ETS Research Report Series, 2011(2), i-181.

Watanabe, Y. (2004). Teacher factors mediating washback. In L. Cheng, Y. Watanabe, & A. Curtis (Eds.), Washback in Language Testing: Research Contexts and Methods (pp. 129-146). Mahwah, NJ: Lawrence Erlbaum Associates.

Appendix 1

Student Interview Questions

1. Can you tell me a little bit about yourself (your home country, how long have you lived in the U.S? your English learning experience)?
2. What do you like and dislike about the class?
3. How does the TOEFL class influence your English skills, speaking, reading, writing, and listening?
4. Can you tell me about your interaction with your classmates and the teacher in your TOEFL class?
5. What do you think about the TOEFL test strategies taught in your class?
6. What do you think about the materials and activities used in the class?
7. Is there anything else you want to add about the TOEFL prep course?

[back to article]

Copyright rests with authors. Please cite TESL-EJ appropriately. Editor’s Note: The HTML version contains no page numbers. Please use the PDF version of this article for citations.