\nMichigan State University, USA
\n<buttiler
![\"atmark\"](\"http:\/\/www.tesl-ej.org\/atmark.png\")
msu.edu><\/p>\n\nAbstract<\/h3>\n
In this study, I investigate whether three A2 Key for Schools<\/em> practice tests from Cambridge Assessment English<\/em> for 6th-graders (11-12 years old) in Argentina measure growth and produce scores that are meaningful. Drawing data from three consecutive years, I analyze the scores of 80 children over a school year and consider whether the tests are beneficial for both students and teachers in terms of time and instructional impact. In this English as a Foreign Language (EFL) context, the results show no significant differences among the students\u2019 average performances across the Reading and Writing<\/em> section of the practice tests and some significant differences across the Listening<\/em> practice tests. I discuss the affective, pedagogical, and curricular implications and argue that because the tests are administered in tight succession, growth may be too small to measure. It is important for teachers and language program coordinators to ensure that the testing they do benefits instruction and positively impacts students in language classes. This is because testing should serve and inform instruction. In this study, I suggest how other teachers in charge of their elementary and middle school assessment programs can do similar analyses to ensure their students\u2019 language evaluation programming is maximally beneficial.<\/p>\n
Keywords<\/strong>: Language assessment, standardized tests, young learners, repeated testing<\/p><\/blockquote>\n
A popular saying states that \u201cpractice makes perfect\u201d, meaning that it is possible to become proficient in an activity or a skill if one exercises regularly. It refers to the idea that the more someone practices, the more likely they are to excel. But how often does an apprentice need to practice in order to improve certain skills? How much do practice effects impact the students\u2019 learning process? It is generally believed that the more exam practice foreign language learners do, the higher the chances of passing a test. Some researchers have pointed out that if students take the same or a similar exam more than once, they may obtain a higher score the second or third time, without necessarily showing improvement in the skills measured (Davies et al., 1999). In addition, as Green and Van Moere (2020) more recently claimed, language learners and instructors \u201cmay use repeated test taking as an opportunity to identify and exploit features of the test design with the intention of maximizing scores rather than improving target abilities\u201d (p. 479). Therefore, repeated testing may not necessarily reflect an increase in language proficiency.<\/p>\n
Many foreign language instructors and coordinators have their students take practice tests numerous times in preparation for standardized assessments such as Cambridge Assessment English<\/em> tests. An increasing body of research has examined language testing in relation to young learners in English as a Foreign Language (EFL) contexts (Bailey, 2008; Butler, 2015, 2016, 2017; McKay, 2006; Menken, 2018; Molloy, 2015; Nikolov, 2016; Wolf & Butler, 2017; Wolf et al., 2008; Wolf & Farnsworth, 2014). However, there is little systematic research on the impact of repeated language testing and the effects of elapsed time between tests taken, especially in young learners (Cho & Blood, 2020; Green, 2005; Gu et al., 2015; Knoch et al., 2020; Lin & Chen, 2020). Even though repeated exam practice can be seen as preparatory and not analytical and students enrolled for an official exam may benefit from this practice, children may experience anxiety and frustration if the test scores do not show measurable progress. This is why it is important that practice is accompanied by opportunities for students to obtain useful feedback on their performance.<\/p>\n
The purpose of this study is to investigate whether three A2 Key for Schools<\/em> practice tests from Cambridge Assessment English<\/em> for 6th-graders at a school in Argentina measure growth and produce scores that are meaningful. In this EFL context, the current study examines whether the three tests taken by three groups of students in May, June, and August of 2016, 2017, and 2018 are consequentially beneficial in terms of time, economics, and instructional impact. This study also aims at sharing simple statistical methods that EFL teachers and school administrators can use to chart their students\u2019 test scores over a period of time and conduct item analysis for different parts of the tests.<\/p>\n
In the context of this study, Cambridge proficiency tests are considered standardized assessments since they are administrated and scored in a consistent manner, using the answer keys and marking schemes provided by Cambridge Assessment English<\/em>. The purpose of these in-class practice tests is to have students practice for the official test administered by Cambridge at the end of the school year, to track progress, and to measure outcomes. These exams consist of three different papers or assessments: Reading and Writing<\/em>, Listening<\/em>, and Speaking<\/em>. This study will only focus on the first two. These assessments differ from multiple-choice exams in that they contain not only multiple-choice tasks but also gap fill, matching, open cloze, and guided writing tasks.<\/p>\n
This study will address the following research questions:<\/p>\n
\n
- Do the three practice tests that students take throughout the school year (May, June, and August) measure growth?<\/li>\n
- Are the three practice tests taken in preparation for the official test administration measuring progress effectively?<\/li>\n
- Are all nine parts of the Reading and Writing<\/em> assessment and all five parts of the Listening<\/em> assessment measuring students\u2019 skills effectively?<\/li>\n<\/ol>\n
Literature Review<\/h3>\n
In the last few years, there has been a growing interest in assessing young learners\u2019 English as a second or foreign language proficiency. Different standards-based tests (e.g., Cambridge Young Learners<\/em><\/a>,<\/em> TOEFL Primary<\/em><\/a>,<\/u><\/em> TOEFL Junior<\/em><\/a>,<\/u><\/em> Michigan Young Learners English<\/em><\/a>, MET Go!<\/em><\/a>, and PTE Young Learners<\/em><\/a>) are used among elementary, middle, and high-school children around the world to measure English language proficiency, monitor students\u2019 progress, and make decisions regarding placement, instructional planning, and promotion or retention, among others. Numerous ESL and EFL programs at schools and language centers have learners take similar tests repeatedly. Instructors and school administrators later use the test scores to inform students and their parents about the learners\u2019 language abilities. However, little is known about the effects of repeated language testing and the need for staggered testing times using standardized tests, especially in young learners.<\/p>\n
Previous research has mostly been conducted with adolescents and adult learners. Green (2005) analyzed writing scores for 15,380 test takers (ages 15 to 62) who took the official IELTS test twice. The period between tests ranged from 3 to 42 months. The researcher also focused on 476 students enrolled on ESL programs over periods of between three to ten weeks who took the IELTS writing test before and after the course. In this study, Green refuted the idea that 200 hours of instruction are needed to improve by one IELTS band score (as recommended by IELTS). Instead, he concluded that initial writing scores and region of origin proved to be stronger predictors of outcomes than course length. The author also noted a weak association between score gains and the period between tests. Barkaoui (2019) also examined the writing scores of adult EFL learners taking the Pearson Test of English (PTE) Academic multiple times. The scholar found that scores continued to rise for the first five months and then started to diminish. Overall, changes in writing scores over time were significantly associated with changes in test takers\u2019 English language proficiency, but not with the number of times taking the tests or the interval between tests.<\/p>\n
More recent studies investigated the effects of repeated test-taking and scores changes in the context of high-stakes English exams required for immigration purposes. Lin and Chen (2020) examined the writing performance of 562 students who took the Canadian English Language Proficiency Index Program\u2013General (CELPIP\u2013General) test at least three times, with a 30\u201340-day interval between the first and second attempts and a 90\u2013180-day interval between the first and third attempts. The authors observed that the test scores were relatively stable over the 6-month period whereas lexical features were the most likely to improve even after one month. Knoch et al. (2020) investigated scores from participants who took the Pearson Test of English-Academic up to 22 times looking to satisfy Australian immigration requirements. Information obtained during interviews revealed that test takers changed their preparation approaches depending on their previous test results. Tests scores showed that the most meaningful improvements from one test session to the next were recorded on the speaking part of the test, followed by fewer improvements in the reading and listening sections. Writing had the fewest number of large improvements. Thus, studies carried out with adolescents and adult learners show that factors such as region of origin, writing scores, vocabulary skills, and speaking skills can be associated with changes in test takers\u2019 language proficiency over time.<\/p>\n
As pointed out before, research on the effects of repeated testing on young learners is scarce. In a longitudinal study, Gu et al. (2015) examined how the TOEFL Junio<\/em>r, a standardized English proficiency test for children ages 11-15, could be used to measure changes in language ability due to learning. The authors analyzed test scores from 4,606 students from 15 countries who took the test more than once. The findings suggested that the TOEFL Junio<\/em>r test serves to measure progress in young learners. The authors explained that students scored higher in the second test and that \u201cthe longer the interval between testing was, the greater the score gain was\u201d (pp. 9-10). They argued that it does not seem possible that the observed gains were due to students having become more familiar with the test format the second time, which differs from Davies et al. (1999). Gu and colleagues stated that the test scores were higher due to changes in the target abilities (listening and reading) as a result of English language learning, which is consistent with Knoch et al.\u2019s (2020) secondary findings. More recently, Cho and Blood (2020) investigated score change and analyzed the effects of elapsed time between tests taken by EFL learners in Japan and Turkey who took the TOEFL Primary<\/em>, a test designed for students ages 8-13. In their study, the researchers found that rate of change varies depending on the initial scores obtained, test taker age, and test level difficulty, with a higher average performance for older students. To help explain why it may be difficult to measure score change across time, the authors also argued that young test takers have shorter attention spans and more developmental limitations than adults. Even though these studies looked at elapsed time between testing sessions, there was no reference to item analysis within the assessments. This can provide a more detailed insight into the students\u2019 performance on the parts of the tests that require different language abilities.<\/p>\n
Moreover, teachers and researchers have expressed their concerns about how children perceive and adapt to testing sessions. Based on the relationship between self-esteem, motivation, and assessment performance, some scholars have shown that it is highly important that young learners have positive experiences with assessments (McKay, 2006; Moss, 2013). Brumen and Cagran (2001) indicated that there is a strong positive correlation between the performance of young learners on assessments and their overall motivation. Research has also shown that affective factors can influence the performance of young learners more than they can impact the performance of adult test takers (Bachman & Palmer, 1996). In a study carried out in Turkey with students ages 9 to 13, Aydin (2012) found that learners experienced test anxiety related to time constraints, the physical setting, and difficulties in the comprehension of instructions. Cameron (2001) also pointed out the importance of clear instructions with young learners since their performance tends to vary depending on the clarity of the given activity. In relation to the type of task, Wolf and Butler (2017) stated that \u201cit is important that sample tasks and practice tasks be available for young learners and their teachers\u201d (p. 11). This highlights the significance of having students complete practice tasks focused on one test-target as well as practice testing in preparation for the official examinations. Finally, regarding score reporting, it has been suggested that assessment results for young learners should help students succeed (McKay, 2006). In this sense, students should have access to detailed and constructive feedback on their test performance, whereas teachers should be able to identify the learners\u2019 strengths and areas in need of improvement to adopt strategies that will advance the students\u2019 learning.<\/p>\n
Context<\/h4>\n
The current study was carried out using practice test scores from 6th<\/sup>-graders studying at a bilingual school in Argentina. In this country, the educational system consists of six years of elementary school (for children ages 6 to 11, called Grades 1 to 6) and six years of high school (for children ages 12 to 17, called Years 1 to 6). At elementary school, students who attend classes at this institution have language, science, and drama lessons in English, which, on average, represent 40% of their total class time. At high school, because students must take more specific courses in Spanish, only 25% of the total curriculum is devoted to courses in English such as language, literature, and biology. For these content areas taught in English, the school follows a Content and Language Integrated Learning (CLIL) approach.<\/p>\n
From Grade 3 in elementary school through the last year of high school, EFL teachers help students to prepare for the Cambridge Assessment English <\/em>official exams. These language proficiency assessments follow the Common European Framework of Reference for Languages (CEFR) developed by the Council of Europe in 2001 to describe language ability on a six-point scale, from A1 (beginners) to C2 (advanced). Elementary Grades 3 to 5 receive training to take the Young Learners English Tests<\/em> (YLE): Pre A1 Starters<\/em><\/a>, A1 Movers<\/em><\/a>, and A2 Flyers<\/em><\/a> respectively. In the sixth and last year of elementary school, students take A2 Key for Schools<\/em><\/a>, whereas in high school, learners prepare to take three higher level Cambridge Assessment English<\/em> qualifications (PET<\/em><\/a>, FCE<\/em><\/a>, and CAE<\/em><\/a>), equivalent to B1, B2, and C1 proficiency levels respectively. Taking the official tests at the end of the school year is not mandatory. Parents, however, are encouraged to pay the fees and have their children signed up. In preparation and, based on the assumption that parents will sign up their children for the official administration of the test on specific test dates (and normally 95% of elementary children take the test, but in high school approximately 70% do), all students receive equal training. In general, two to three hours per week are dedicated to exam practice in class and three practice tests are administered throughout the academic year with the stated goal that the more the students practice, the better they will do during the live administration of the test.<\/p>\n
Because the school commits to following a CLIL approach to teach science and English language, teachers are required to use textbooks that present authentic, multicultural literature and language practice that promotes the development of higher-order thinking skills. At the same time, in English language classes, students use textbooks designed for EFL learners taking international language proficiency exams. These textbooks usually have a strong focus on forms, as they present grammar and vocabulary exercises where students are instructed to fill in gaps and answer multiple-choice questions. These activities serve as practice tasks and complement the practice done during exam practice hours.<\/p>\n
Teachers\u2019 attitudes towards the coexistence of this CLIL approach and the exam practice approach adopted in language classes generally indicate that there does not seem to be a convergence between these two methods of instruction. While focus on specific forms is not the objective of the CLIL classes, exam practice hours are mainly about explicit instruction of grammar rules and practice of individual structures. Indeed, Nikolov (2016) pointed out that \u201cresearch projects on early CLIL tend to follow a different tradition unrelated to testing children or standards-based testing\u201d (p. 8). Moreover, Turner (2012) argued that \u201cteachers continue to grapple with the relationship between what they are doing in the classroom and the specifications of the external large-scale tests their students will eventually need to take\u201d (p. 69). In addition, a study carried out by Hill (2015), showed that the integration of instruction, the curricula, and assessment can lead to more effective teaching practices and better assessment results with positive washback effects. Therefore, a more integrated approach where the exam practice hours also focus on meaning and CLIL classes assist students in strengthening certain language forms could enhance students\u2019 learning and contribute to their performance on the test.<\/p>\n
Motivations Behind the Current Study<\/h4>\n
I was motivated to conduct this study both as a practitioner who saw the need to investigate the three research questions in my own school\u2019s context, and as a researcher in the field of applied linguistics. Whether multiple tests are beneficial and meaningful is an important question for me and the school students locally, and for the field of applied linguistics at large. The motivations behind this study relate to concerns that I had during my experience at this bilingual school regarding students\u2019 performances on three practice tests and the affective, pedagogical, and curricular implications of this approach and teaching practice.<\/p>\n
First, the time span between each of the three practice tests that students take ranges from thirty to forty-five days, with a three-month interval between the first and the third practice test. The students usually take the first test by mid-May, the second one in June, and the third and last practice test in August. Because the school year begins in March, students do the first practice test less than two months after the beginning of classes, a period that is usually not enough for teachers to present the different test items and for students to become familiar with the overall test structure and its parts. In addition, the last practice test is administered right after the two-week winter break, a time during which students do not usually do English homework or receive much exposure to the target language. As discussed above, in some contexts, test scores may be relatively stable over short time intervals, especially for reading, listening, and writing skills (Gu et al., 2015; Green, 2005; Knoch et al., 2020; Lin & Chen, 2020).<\/p>\n
Second, the learners are told that the scores will not influence their grades because the tests are taken for the sake of practice. However, when they are handed out the assessments, students are asked to sit individually, and instructed not to speak, ask any questions, or use dictionaries. This, from my experience, increases the children\u2019s anxiety and does not mirror their learning experiences where students are encouraged to participate, interact with their teachers and classmates, and resort to their notes and books when they need help. As indicated by Kormos et al. (2020), to ensure the validity of the interpretation of the scores, \u201cassessment tasks should be administered in an environment that is familiar and comfortable for young learners\u201d (p. 56). Hamid and Hoang (2018) highlighted the importance of \u201chumanizing\u201d language testing. They showed that test takers are calling for more friendly and closer-to-life tests to make the test-taking experience less stressful. Previous research also indicated that affective factors such as motivation, attitudes towards the test, and self-esteem influence a learner\u2019s assessment performance, especially in young students (Bachman & Palmer, 1996, 2010). Therefore, the test administration conditions are important to investigate, as they can influence the students\u2019 performance.<\/p>\n
Further studies have shown that there exists a strong correlation between young learners\u2019 performance on a test and their overall motivation (Brumen & Cagran, 2011). This relates to my third concern. Often times, when students get their graded tests back, they express that they see no significant differences among the first, second, and third practice tests scores. This usually leads to frustration and lack of motivation, making students feel that they are not making progress and that they will not be advised to take the official test. In addition, the students receive little and very general feedback on their performance on the practice tests. Not much time is devoted to specific and developmental feedback on different items of the test other than the corrections made by the teachers on the practice papers. These corrections may be somewhat helpful in the Reading and Writing <\/em>assessments but may have very little value particularly in the Listening <\/em>assessments as students do not listen to the audio tracks again to discuss the answers.<\/p>\n
Fourth, as proof that they have seen their children\u2019s tests and scores, parents are required to sign the practice tests and return them to the school. From my experience with interviews with parents, they sometimes expect that their children show considerable progress in a short period of time. According to Nikolov (2016), \u201cparents who want their children to get language certificates assume that the proficiency achieved at an early stage of language learning will be automatically maintained and built on over time\u201d (p. 3). Thus, the practice tests impact entire families and how the children are seen by their parents. In this sense, the practice tests may be causing unnecessary “parental worry” (Chik & Besser, 2011, p. 88) because the scores are not indicating significant growth or, in some cases, any growth at all. The worry may be unfounded because, when the tests are administered in tight succession, growth may be too small to measure. In addition, some parents who received the scores may worry not only that their children are not learning, but also that the teachers are not teaching. Concomitantly, the teachers who receive the scores may believe their teaching is not effective, but may not be sure why, which may demotivate them as educators.<\/p>\n
Finally, because there are approximately thirty students in each classroom and one instructor may teach more than one course, grading these practice tests represents a significant workload. This may have a high opportunity cost as teachers spend several hours and even days grading tests and therefore cannot dedicate that time to activities such as lesson planning or material design that could make their classes more enriching for students. A possible solution to this could be to have students self-correct the closed answer sections of their tests in class. Students are used to this self-correction practice as they do it for other grammar and textbook exercises. That way, teachers could go over different parts of the test, providing students with an opportunity to learn about their correct and incorrect answers, and saving out-of-school grading time.<\/p>\n
The Study<\/h3>\n
Participants and data collection<\/h4>\n
In this study, I analyzed practice test scores from elementary Grade 6 children (ages 11 and 12) who took A2 Key for Schools<\/em> level practice tests three times during their elementary Grade 6 academic year: first in May, then in June, and finally in August. Each child thus took three Reading and Writing<\/em> and three Listening<\/em> A2 Key for Schools<\/em> assessments. There were 80 children total in the study sample. The school is relatively small. There are approximately 25 to 30 students in elementary Grade 6 each year. Therefore, I was able to have 80 children in the sample by collapsing data from three consecutive years of Grade 6 practice test administration. These three years were 2016 (N<\/em>= 25), 2017 (N<\/em>= 26), and 2018 (N<\/em>= 29).<\/p>\n
The participants\u2019 first language is Spanish. They attend an all-girls bilingual school in the province of Buenos Aires, Argentina. In previous years (Grades 3 to 5), all students received training in standards-based testing and most of them took Cambridge Assessment English <\/em>tests designed for young learners (Pre A1 Starters<\/em>, A1 Movers<\/em>, A2 Flyers<\/em>). Hence, in Grade 6 the students were familiar with the format of some standardized tests already.<\/p>\n
Materials<\/h4>\n
For the current study, I analyzed the Reading and Writing<\/em> and Listening<\/em> scores obtained from the administration of three A2 Key for School Cambridge English <\/em>practice tests over three consecutive years (2016, 2017, and 2018). These tests were taken from the first edition of Cambridge Key English Test for Schools 1<\/em> (2010), a book containing a set of past official examination papers from University of Cambridge ESOL and published by Cambridge University Press. Because the data were not originally collected with the purpose of carrying out the current study, there is no information as to whether all groups of students took the exact same practice test for each of the different testing times every year. However, for the sake of practicality and valuing the data collected longitudinally, over a period of three years, I will assume that as standards-based assessments, the tests used are overall equated in terms of difficulty. Cambridge Assessment English<\/em> reported a 0.95 reliability for the total test score and a 3.23 Standard Error of Measurement (SEM) for the A2 Key for Schools<\/em> test (Cambridge Assessment English, 2021, Quality and accountability<\/em>).<\/p>\n
![](\"https:\/\/tesl-ej.org\/ej99\/a3pix\/figure-1.png\")
![](\"https:\/\/tesl-ej.org\/ej99\/a3pix\/figure-2.png\")
![](\"https:\/\/tesl-ej.org\/ej99\/a3pix\/figure-3.png\")
![](\"https:\/\/tesl-ej.org\/ej99\/a3pix\/figure-4.png\")