{"id":12864,"date":"2018-02-23T23:50:10","date_gmt":"2018-02-24T06:50:10","guid":{"rendered":"http:\/\/www.tesl-ej.org\/wordpress\/?page_id=12864"},"modified":"2018-02-28T04:01:37","modified_gmt":"2018-02-28T11:01:37","slug":"ej84a3","status":"publish","type":"page","link":"https:\/\/tesl-ej.org\/wordpress\/issues\/volume21\/ej84\/ej84a3\/","title":{"rendered":"A Cross-Sectional Survey on Japanese English-Major University Students\u2019 Confidence in the TOEIC Can-Do List"},"content":{"rendered":"

February 2018 – Volume 21, Number 4<\/h2>\n

Shinya Ozawa<\/strong>
\nHiroshima Shudo University, Japan
\n<ozawa\"atmark\"shudo-u.ac.jp><\/p>\n

Abstract<\/h3>\n

This survey was conducted to investigate how university students gain or lose confidence in English communicative domains over four years at university. Self-assessment has been a useful instrument for measuring learners\u2019 English proficiency, and the students in this study were required to self-assess their confidence levels in TOEIC Can-Do list items. The results revealed a difference between first- and third-year students\u2019 levels of confidence in the Reading domain, but the effect size was small. In the other communicative domains, there was no significant gain in confidence over the four years. Frustratingly, there was a decline in confidence from the third to the fourth year. Even when the limitations of this survey are considered, the results clearly demonstrate that there is a need to improve the curriculum in order to develop student confidence over their four years at university.<\/p>\n

Introduction<\/h3>\n

Following the introduction of the Common European Framework of Reference for Languages (CEFR) and CEFR-J, which was developed for use specifically in Japan, much attention has been paid to their use in language education in Japan. Among the research initiatives on the CEFR, CEFR-J, and other Can-Do lists, several studies have been conducted on the reliability of learners\u2019 self-assessments on these lists, most of which confirmed learners\u2019 reliability (e.g., Ross, 2006). However, there has not been much consensus or evidence on how a learner\u2019s self-assessment can change over time.<\/p>\n

In this study, first- to fourth-year English-major students at a local university in Japan were asked to self-assess their confidence in English Can-Do items, and their responses were analyzed. A longitudinal study would have been preferred, but a cross-sectional approach was adopted because of time constraints. The survey was part of a series of attempts that have been made to verify the effectiveness of the curriculum in the department of English Language and Literature at the university. The researchers hope that the results of this survey can serve as a basis for useful discussions about the future introduction of a new curriculum.<\/p>\n

Background of the Study<\/h3>\n

Since the introduction of the current curriculum in the department, no special attempts have been made to measure its effectiveness thus far. Recognizing this problem, we began our project in 2011 to better understand the general nature of the curriculum, and to suggest ways to improve it. An outline of the curriculum and issues in the department are summarized in the following section.<\/p>\n

Outline of the Curriculum<\/h4>\n

The current English department curriculum was implemented in 2007. It places emphasis on providing abundant English reading inputs for students. There are five compulsory English-skill courses in each semester of the first year and four in each semester of the second year, as shown in Table 1.<\/p>\n

In the Progress in English I\/II course, the same textbook is used in all the classes. Japanese teachers of English provide instruction for the first-year classes, in which students are required to read the textbook intensively. Native-English speakers teach the second-year Progress in English III\/IV classes, in which students are required to deliver oral presentations in English based on reading inputs. As indicated by their titles, the other writing and grammar courses underscore the importance of reading inputs as well. The English-skills courses in the first and second years provide the foundation for more academic-oriented English courses in the third and fourth years, during which students write a graduate thesis on a topic of their choosing.<\/p>\n

Table 1. First- and Second-Year Compulsory English-Skill Courses<\/em><\/p>\n\n\n\n\n\n\n\n\n\n
1st year<\/td>\n2nd year<\/td>\n<\/tr>\n
1st semester<\/td>\n2nd semester<\/td>\n1st semester<\/td>\n2nd semester<\/td>\n<\/tr>\n
Progress in English I<\/td>\nProgress in English II<\/td>\nProgress in English III<\/td>\nProgress in English IV<\/td>\n<\/tr>\n
Reading<\/strong>
\n& Writing I<\/td>\n		Reading<\/strong>
\n& Writing II<\/td>\n		Reading<\/strong>
\n& Writing III<\/td>\n		Reading<\/strong>
\n& Writing IV<\/td>\n<\/tr>\n
Reading<\/strong>
\n& Grammar I<\/td>\n		Reading<\/strong>
\n& Grammar II<\/td>\n		Reading<\/strong>
\n& Grammar III<\/td>\n		Reading<\/strong>
\n& Grammar IV<\/td>\n<\/tr>\n
Speaking I<\/td>\nSpeaking II<\/td>\nSpeaking III<\/td>\nSpeaking IV<\/td>\n<\/tr>\n
Listening I<\/td>\nListening II<\/td>\n–<\/td>\n–<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

One concern in the department is that there are no compulsory English-skill courses for the third- and fourth-year students. Only those students willing to continue working to improve their English skills choose to take elective courses such as Presentation and Discussion I\/II and Project Work I\/II. We, the teachers, believe that the students\u2019 English abilities continue to develop in their third and fourth years, since we require them to read academic textbooks in small-enrollment seminar courses. There is no reliable evidence, however, that students improve their English skills in their last two years.<\/p>\n

Investigation of Curriculum Effectiveness<\/h4>\n

All first-year students at the university are required to take the TOEIC Bridge test three times a year: in April as a placement test and in July and January as achievement tests. Since the proficiency levels of the English-majors are generally higher than those of students in other departments, the placement test was changed from TOEIC Bridge to TOEIC in 2011. Except for these tests, we have not implemented any other means of tracking students\u2019 development of their English skills. We commenced our project in 2011, with the purpose of investigating English-major students\u2019 English skills using three primary methods: semi-structured interviews with the students, the development of a corpus of English textbooks that we recommend our students to read, and a survey in which students self-assess their English skills by using a Can-Do list. The Can-Do list survey is the focus of this paper.<\/p>\n

Literature Review<\/h3>\n

This section summarizes some of the narrative and systematic reviews of the existing literature on self-assessments. Various previous studies have investigated the reliability of learners\u2019 self-assessment of Can-Do items. Before discussing the survey data, it is important to review this particular line of research.
\nBlanche and Merino (1989) reviewed and summarized all previous studies on self-assessment in foreign language skills. Their extensive literature review identified 16 studies published between 1979 and 1986. Overall, these studies found that self-assessments of proficiency correctly reflected external criteria, such as standardized test scores. In addition, it became clear that when self-assessment items are based on concrete situations, they correlate with learners providing more accurate assessments, with better learners tending to underestimate their proficiencies. The study was a seminal work that synthesized the previous literatures; however, as they pointed out, their study remained as a kind of \u201cprose-based\u201d (Blanche & Merino, 1989, p. 2) subjective narrative review.<\/p>\n

In an attempt to draw a more comprehensive and objective picture of self-assessment, Ross (1998) adopted meta-analysis as his procedure. He limited his search to studies that had focused on the correlations between self-assessment and other measures in four language skills, collecting 60 correlations in 10 studies. He found that the overall average correlation r was .63 and effect size g was 1.64; for reading, .61 and 1.56, listening, .65 and 1.71, speaking .55 and 1.33, and writing .52 and 1.23. Overall, correlations between self-assessments and other measures were quite high, with the productive skills of speaking and writing showing lower correlations than the other two skills. He claimed that one of the factors that affected the self-assessments was whether the learners had actually experienced the specific examples that they were being asked about.<\/p>\n

Studies comparing self-assessments with other criteria, such as test scores, have also been conducted in Japan. Saida (2008), for example, adopted the DIALANG test (Online Language Diagnosis System, n.d.) to investigate the correlation between the self-assessed English proficiency levels and actual performances of 130 first-year university students in Japan. In listening, writing, and reading skills, the rates of correspondence between the two were about 60% for each skill. These rates were lower than those found in Alderson\u2019s (2005) DIALANG pilot study, in which the rate for each skill was over 80%. Several studies that focused on the correlations between TOEIC scores and Can-Do self-assessments have also been conducted, and they showed the same tendency as above (Ito, Kawaguchi, & Ohta, 2005; Powers, Kim, & Weng, 2008; Powers, Yu, & Yan, 2013).<\/p>\n

Generally, we can assume that self-assessment is a reliable measure for predicting learners\u2019 proficiencies. However, it also became clear that various factors, such as learners\u2019 proficiency levels and actual experiences of the Can-Do items, could also influence responses. Brantmeier, Vanderplank, and Strube (2012), for example, invented self-assessment reading question items adopted from various course materials, so that their items reflected the objectives of a Spanish language course. The participants of the study were enrolled in beginner-, intermediate-, and advanced-level Spanish language courses at a private university in the United States. They completed the self-assessment survey during class, and advanced-level students were asked to take the online DIALANG test to investigate the correspondence between their self-assessment and their actual scores. The results showed that (a) beginner learners rated their speaking skills the lowest, (b) many groups rated reading ability the highest among their language skills, and (c) the advanced students were able to judge their levels appropriately. Result (c) appears to contradict previous findings that advanced students tended to underestimate their proficiencies. As Brantmeier et al. (2012) outlined, this could be because the self-assessment items in the study were criterion-referenced and corresponded with the course objectives. In other words, the learners could easily imagine the situations where their English skills were used (Little, 2014), and were therefore able to provide more accurate answers.<\/p>\n

There is also a need to investigate whether or not learners\u2019 self-assessments of their own proficiency change in accordance with their English language development. L\u00e9ger (2009) performed a qualitative investigation of how self-assessments in speaking, conducted by university learners studying French, changed over a 12-week period. She concluded that their self-assessments changed in a positive way; their confidence in self-assessment items increased by the end of the course. Nishida\u2019s research (2012) on elementary school students is one of only a few studies to investigate how students\u2019 self-assessments develop longitudinally. One hundred and six students aged 10 and 11 participated in the study and the data were collected four times over a one-year period. It became evident that most of the pupils\u2019 self-assessments declined after the second term of the year (cf., Chen, 2008). There are, however, few longitudinal studies that focus on how higher-education students rate their own skills. This gap in the existing research was a major factor in our decision to conduct the present survey.<\/p>\n

Survey<\/h3>\n

<\/a>
\nOur main aim in this survey was to investigate how university students self-assess their English skills over the course of four years of study at a university. It would have been ideal to adopt a longitudinal approach to determine the trajectory of the learners\u2019 developments. Due to time constraints, however, we adopted a cross-sectional approach for this study. The research question was how the university students develop their self-assessments of their English skills over the course of four years of study. The study\u2019s three hypotheses are as follows: (1) students gain confidence over four years of study at the university, (2) students rate their reading skills the highest, and (3) students rate items experienced in concrete situations higher than others.<\/p>\n

Procedure<\/h4>\n

The participants were English-major students at a private university in Japan1<\/a><\/sup>. All students from the first to fourth years were asked to answer the self-assessment questionnaire. The Can-Do statements developed by the Educational Testing Service (ETS; 2000) were used in the survey (Appendix A). There were 75 Can-Do statements covering five communicative domains, including both the business and social aspects of work: listening, speaking, interaction, reading, and writing. First, the Can-Do statements were translated from English to Japanese and tested by 10 postgraduate students at the university, and any statements the participants found confusing were revised. Next, the questionnaire was administered to 348 students, who rated their levels of confidence on a 5-point Likert scale: (5) Can do easily, (4) Can do with very little difficulty, (3) Can do with some difficulty, (2) Can do with great difficulty, and (1) Cannot do at all. Participants were also asked to confirm whether they had actually experienced the situations described in the items. Data from 257 students who answered all of the statements, and their responses were used for the analysis. The statistical analysis was conducted using RStudio 0.99 and HAD software (Shimizu, 2016).<\/p>\n

Results<\/h4>\n

The Kaiser-Meyer-Olkin measure of sampling adequacy was .95 and Barlett\u2019s test of sphericity was significant (\u03c72<\/sup> (2775) = 17656.38, p<\/em>=.0). Therefore, the data was considered to be appropriate for using factor analysis. The number of factors was set at five as suggested in the original version of the questionnaire, and exploratory factor analysis was implemented. Maximum likelihood was adopted as an extraction method because the result can be applied to other samples. The rotation method was Promax because each factor was assumed to correlate with each other. As a result, five factors were identified and labeled as Listening, Speaking, Interaction, Reading, and Writing. Those items with factor loadings below .4 and those that did not correspond with the assumed factors were eliminated from the analysis (Appendix B). Accordingly, 42 out of 75 items were selected. Cronbach\u2019s alpha coefficients for each factor were as follows; Listening = .91 (skewness = 0.64, kurtosis = 0.91), Speaking = .91 (skewness = 0.28, kurtosis = -0.27), Interaction = .94 (skewness = 0.71, kurtosis = -0.20), .Reading = .92 (skewness = 0.20, kurtosis = -0.09), and Writing = .94 (skewness = 0.25, kurtosis = -0.10), and the internal consistencies were confirmed as being high enough to be acceptable.<\/p>\n

The descriptive statistics of the ratings are shown in Appendix C. As can be seen from the table, the ratings in Speaking and Reading were higher than in the other domains, but the ratings as a whole were quite low and did not substantially improve over the years. The boxplots, with beeswarms, are provided in Appendix D in order to illustrate the individual participants\u2019 responses. The differences between each year in each domain were then compared using one-way analysis of variance (ANOVA). The results are shown in Table 2.<\/p>\n

No significant differences were found in Listening, Speaking, Interaction, and Writing; the only exception was in Reading. Following this, we adopted a Holm method and conducted post hoc analysis to make multiple comparisons in Reading. A significant difference was found only between the first- and third-year students. The effect size, however, was very small.<\/p>\n

As implied in the previous research, the difficulty and familiarity of the questionnaire \u2013 in other words, the participants\u2019 experiences of each item – may have had an impact on their responses to them. The boxplots in Appendix E illustrate two groups: 0 is \u201cnever experienced\u201d and 1 is \u201cexperienced.\u201d Although the number in each group was unbalanced and could not provide a precise answer to the question of respondents\u2019 relationships to them, there was an overall tendency in each item that already-experienced items were rated higher.<\/p>\n

Table 2. One-Way Analyses of Variance for the Ratings on Five Communicative Domains<\/em><\/p>\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
Variable and source<\/td>\nSS<\/em><\/td>\nMS<\/em><\/td>\nF<\/em><\/td>\np<\/em><\/td>\n\u03b72<\/sup><\/td>\n<\/tr>\n
Listening<\/td>\n<\/td>\n<\/td>\n<\/td>\n<\/td>\n<\/td>\n<\/tr>\n
Between<\/td>\n0.92<\/td>\n0.31<\/td>\n0.65<\/td>\n.59<\/td>\n.01<\/td>\n<\/tr>\n
Error<\/td>\n120.49<\/td>\n0.48<\/td>\n<\/td>\n<\/td>\n<\/td>\n<\/tr>\n
Speaking<\/td>\n<\/td>\n<\/td>\n<\/td>\n<\/td>\n<\/td>\n<\/tr>\n
Between<\/td>\n1.03<\/td>\n0.34<\/td>\n0.67<\/td>\n.57<\/td>\n.01<\/td>\n<\/tr>\n
Error<\/td>\n130.29<\/td>\n0.52<\/td>\n<\/td>\n<\/td>\n<\/td>\n<\/tr>\n
Interaction<\/td>\n<\/td>\n<\/td>\n<\/td>\n<\/td>\n<\/td>\n<\/tr>\n
Between<\/td>\n0.79<\/td>\n0.26<\/td>\n0.72<\/td>\n.54<\/td>\n.01<\/td>\n<\/tr>\n
Error<\/td>\n92.44<\/td>\n0.37<\/td>\n<\/td>\n<\/td>\n<\/td>\n<\/tr>\n
Reading<\/td>\n<\/td>\n<\/td>\n<\/td>\n<\/td>\n<\/td>\n<\/tr>\n
Between<\/td>\n5.54<\/td>\n1.85<\/td>\n3.63<\/td>\n.01<\/td>\n.04<\/td>\n<\/tr>\n
Error<\/td>\n128.62<\/td>\n0.51<\/td>\n<\/td>\n<\/td>\n<\/td>\n<\/tr>\n
Writing<\/td>\n<\/td>\n<\/td>\n<\/td>\n<\/td>\n<\/td>\n<\/tr>\n
Between<\/td>\n3.08<\/td>\n1.03<\/td>\n2.3<\/td>\n.08<\/td>\n.03<\/td>\n<\/tr>\n
Error<\/td>\n112.72<\/td>\n0.45<\/td>\n<\/td>\n<\/td>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

Discussion<\/h3>\n

Overall, through first to fourth year, our participants\u2019 levels of confidence did not increase as expected, and their confidence either declined or did not change between the third and fourth year. In many Japanese universities, fourth-year students are not required to take many courses, and this situation may partially explain their low self-assessments. Among the five communicative domains, Reading was rated second highest (2.97, SD<\/em> = 0.76) in the first year and highest in the fourth year (3.25, SD<\/em> = 0.72). This tendency corroborates the findings of Brantmeier et al. (2012), and might be due in part to the emphasis our curriculum places on reading skills. Even though there was a significant difference between first and third year, the effect size was still very small.<\/p>\n

Surprisingly, first-year students rated their speaking skills the highest (3.13, SD<\/em> = 0.68), which is contradictory to Ross\u2019 findings (1998). This may be partly because the questionnaire items addressed speaking scenarios that were familiar to the participants. For example, speaking contained items such as \u201cintroduce myself in social situations and use appropriate greeting and leave-taking expressions (Speaking, 1),\u201d and \u201cdescribe my daily routine (e.g., when I get up, what time I eat lunch) (Speaking, 8).\u201d It can be assumed that they rated this skill higher compared to the other, seemingly more difficult skills. On the other hand, the Interaction domain, which is similar to Speaking, was rated the lowest. This might have been due to the nature of the questionnaire items. In the Interaction domain, we included items such as \u201cconduct simple business transactions at places such as the post office, bank, drugstore (Interaction, 1)\u201d and \u201cexplain to a repairman what is wrong with an appliance that I want fixed (Interaction, 8).\u201d These items required that the participants not only had the experience of interacting with others but also that they could negotiate with specific knowledge. If they had not had the experience mentioned in the items, they might have thought that they were unable to perform those tasks. This again indicates that experience of or familiarity with the questionnaire items is a very important factor in self-assessment of language skills, which supports the findings of Ross (1998).<\/p>\n

Conclusion<\/h3>\n

This study explored how university students gained or lost confidence over a four-year period. The students demonstrated a tendency to rate reading skills higher than other ones, and whether they had experience of the items in question was found to be an important factor affecting ratings. Generally, the third-year students demonstrated more confidence than the first-year ones in the Reading domain, but not in the other domains. In addition, as many of the university teachers could readily guess, the fourth-year students seemed to have lost confidence in each domain, and that the ratings declined from the third to fourth year. We hope that these results will lead to further discussion on how we can ensure that we provide courses that deliver balanced instruction of the five English communicative domains, and how, under the new curriculum, we can enhance and maintain student confidence in the fourth year.<\/p>\n

There are two issues to be considered for future studies. First, because of time constraints, this study adopted a cross-sectional approach to investigate the university students\u2019 self-assessments on Can-Do items. A longitudinal study is also needed to reinforce this study\u2019s findings. Second, the Can-Do list in this study was designed for TOEIC test-takers, and some question items were unfamiliar to the students. Since familiarity and experience affect self-assessments, similar studies, which adopt criterion-referenced and concrete Can-Do items, might affect the accuracy of self-assessments.<\/p>\n

About the Author<\/h3>\n

Shinya Ozawa<\/strong> is a professor in the Department of English at Hiroshima Shudo University in Japan. His research interest lies in the integration of ICT in the university language classrooms.<\/p>\n

<\/a><\/p>\n

Note<\/h3>\n

1. The homogeneity of each group cannot be guaranteed because of the placement test change from TOEIC Bridge to TOEIC in 2011. As a reference, a t-test was conducted to compare the first- and second-year students who took TOEIC (t<\/em>= -0.473, df<\/em> = 240, p<\/em> = .636), and third- and fourth-year students who took TOEIC Bridge (t<\/em> = 0.160, df<\/em> = 252, p<\/em> = .873). It can therefore be assumed that the students were at similar English proficiency levels when they entered the university. [back<\/a>]<\/p>\n

References<\/h3>\n

Alderson, C. A. (2005). Diagnosing foreign language proficiency: The interface between learning and assessment<\/em>. New York: Continuum.<\/p>\n

Blanche, P., & Merino, B. J. (1989). Self\u2010assessment of foreign\u2010language skills: Implications for teachers and researchers. Language Learning, 39<\/em>, 313\u2013338. doi:10.1111\/j.1467-1770.1989.tb00595.x<\/p>\n

Brantmeier, C., Vanderplank, R., & Strube, M. (2012). What about me? Individual self-assessment by skill and level of language instruction. System, 40<\/em>(1), 144\u2013160. doi:10.1016\/j.system.2012.01.003<\/p>\n

Chen, Y. M. (2008). Learning to self-assess oral performance in English: A longitudinal case study. Language Teaching Research, 12<\/em>(2), 235\u2013262. doi: 10.1177\/1362168807086293<\/p>\n

DIALANG [Online language diagnosis system] (n.d.). Lancaster, UK: Lancaster University. Retrieved from https:\/\/dialangweb.lancaster.ac.uk<\/a><\/p>\n

Educational Testing Service. (2000). TOEIC Can-do guide: Linking TOEIC scores to activities performed using English<\/em>. Retrieved from https:\/\/www.ets.org\/Media\/Research\/pdf\/TOEIC_CAN_DO.pdf<\/a><\/p>\n

Ito, T., Kawaguchi, K., & Ohta, R. (2005). A study of the relationship between TOEIC scores and functional job performance: Self-assessment of foreign language proficiency<\/em>. Retrieved from http:\/\/www.toeic.or.jp\/library\/toeic_data\/toeic_en\/pdf\/newsletter\/1_E.pdf<\/a><\/p>\n

L\u00e9ger, D. S. (2009). Self-assessment of speaking skills and participation in a foreign language class. Foreign Language Annals, 42<\/em>(1), 158\u2013178. doi:10.1111\/j.1944-9720.2009.01013.x<\/p>\n

Little, D. (2014). The Common European Framework and the European Language Portfolio: involving learners and their judgements in the assessment process. Language Testing, 22<\/em>(3), 321\u2013336. doi:10.1191\/0265532205lt311oa<\/p>\n

Nishida, R. (2012). A longitudinal study of motivation, interest, can-do and willingness to communicate in foreign language activities among Japanese fifth-grade students. Language Education & Technology, 49<\/em>, 23\u201345. Retrieved from http:\/\/ci.nii.ac.jp\/naid\/110009470694<\/a><\/p>\n

Powers, D. E., Kim, H-J., & Weng, V. Z. (2008). The redesigned TOEIC (listening and reading) test: Relations to test-taker perceptions of proficiency in English<\/em> (ETS Research Report No. 08-56). Retrieved from https:\/\/www.ets.org\/Media\/Research\/pdf\/RR-08-56.pdf<\/a><\/p>\n

Powers, D. E., Yu, F., & Yan, F. (2013). The TOEIC listening, reading, speaking, and writing tests: Evaluating their unique contribution to assessing English-language proficiency<\/em>. Retrieved from https:\/\/www.ets.org\/Media\/Research\/pdf\/TC2-03.pdf<\/a><\/p>\n

Ross, S. (1998). Self-assessment in second language testing: a meta-analysis and analysis of experiential factors. Language Testing, 15<\/em>(1), 1\u201320.<\/p>\n

Ross, J. A. (2006). The reliability, validity, and utility of self-assessment. Practical Assessment Research & Education, 11<\/em>(10), 1\u201313. Retrieved from http:\/\/pareonline.net\/pdf\/v11n10.pdf<\/a><\/p>\n

Saida, C. (2008). The use of the common European framework of reference levels for measuring Japanese university students’ English, JACET Journal, 47<\/em>, 127\u2013140. Retrieved from http:\/\/ci.nii.ac.jp\/naid\/110007467221\/en<\/a><\/p>\n

Shimizu, H. (2016 ). Free-soft no tokei-bunseki soft HAD: Kino no syokai to tokei gakusyu kyoiku kenkyu-jissen ni okeru riyo no teian [The development of free statistical analysis software HAD: Introducing its functions and suggesting how to use for statistical learning, teaching\/researching practices], Journal of Media, Information and Communication, 1<\/em>, 59\u201373. Retrieved from http:\/\/jmic-weblab.org\/ojs\/index.php\/jmic\/article\/view\/6\/5<\/a><\/p>\n

Appendix A: TOEIC Can-Do Questionnaire<\/h3>\n

(adapted from \u201cTOEIC Can-do guide: Linking TOEIC scores to activities performed using English,\u201d by ETS, 2000, Appendix A)<\/p>\n

IN LISTENING, I CAN<\/p>\n

    \n
  1. understand simple questions in social situations such as \u201cHow are you?\u201d \u201cWhere do you live?\u201d and \u201cHow do you feel?\u201d<\/li>\n
  2. understand a salesperson when she or he tells me prices of various items<\/li>\n
  3. understand someone speaking slowly and deliberately, who is giving me directions on<\/li>\n
  4. understand explanations about how to perform a routine task related to my job<\/li>\n
  5. understand a co-worker discussing a simple problem that arose at work<\/li>\n
  6. understand announcements at a railway station indicating the track my train is on and the time it is scheduled to leave<\/li>\n
  7. understand headline news broadcasts on the radio<\/li>\n
  8. understand a client\u2019s request made on the telephone for one of my company\u2019s major products or services<\/li>\n
  9. understand a person\u2019s name when she or he gives it to me over the telephone<\/li>\n
  10. understand play-by-play descriptions on the radio of sports events that I like (e.g., soccer, baseball)<\/li>\n
  11. understand an explanation given over the radio of why a road has been temporarily closed<\/li>\n
  12. understand someone who is speaking slowly and deliberately about his or her hobbies, interests, and plans for the weekend<\/li>\n
  13. understand directions about what time to come to a meeting and the room in which it will be held<\/li>\n
  14. understand an explanation of why one restaurant is better than another<\/li>\n
  15. understand a discussion of current events taking place among a group of persons speaking English<\/li>\n<\/ol>\n

    IN SPEAKING, I CAN<\/p>\n

      \n
    1. introduce myself in social situations and use appropriate greeting and leave-taking expressions<\/li>\n
    2. state simple biographical information about myself (e.g., place of birth, composition of family)<\/li>\n
    3. describe the plot of a movie or television program that I have seen<\/li>\n
    4. describe a friend in detail, including physical and personality characteristics<\/li>\n
    5. describe my academic training or my present job responsibilities in detail<\/li>\n
    6. order food at a restaurant<\/li>\n
    7. talk about topics of general interest (e.g., current events, the weather)<\/li>\n
    8. describe my daily routine (e.g., when I get up, what time I eat lunch)<\/li>\n
    9. talk about my future professional goals and intentions (e.g., what I plan to be doing next year)<\/li>\n
    10. tell a co-worker how to perform a routine job task<\/li>\n
    11. telephone the airline to change my flight reservations to a different time and day<\/li>\n
    12. tell a colleague at work about a humorous event that recently happened to me<\/li>\n
    13. give a prepared half-hour formal presentation on a topic of interest<\/li>\n
    14. adjust my speaking to address a variety of listeners (e.g., professional staff, a friend, children)<\/li>\n
    15. tell someone directions on how to get to my house or apartment<\/li>\n<\/ol>\n

      IN INTERACTIVE SKILLS, I CAN<\/p>\n

        \n
      1. conduct simple business transactions at places such as the post office, bank, drugstore<\/li>\n
      2. telephone a restaurant to make dinner reservations for a party of three<\/li>\n
      3. give and take messages over the telephone<\/li>\n
      4. explain written company policies to a new employee<\/li>\n
      5. discuss with a co-worker the best way to accomplish a job task<\/li>\n
      6. discuss with an electronics salesperson the features I want on a new videocassette recorder<\/li>\n
      7. meet with a doctor and explain the physical symptoms of my illness<\/li>\n
      8. explain to a repairman what is wrong with an appliance that I want fixed<\/li>\n
      9. request information over the telephone (e.g., check airline schedules with a travel agent)<\/li>\n
      10. meet with a real-estate agent to discuss the type of house I would like to buy<\/li>\n
      11. talk to an elementary school class about what I do for a living<\/li>\n
      12. discuss world events with an English-speaking guest<\/li>\n
      13. discuss with my boss ways to improve customer service or product quality<\/li>\n
      14. telephone a department store and find out if a certain item is currently in stock<\/li>\n
      15. conduct an interview with an applicant for a job in my area of expertise<\/li>\n<\/ol>\n

        IN READING, I CAN<\/p>\n

          \n
        1. read, on storefronts, the type of store or services provided (e.g., \u201cdry cleaning,\u201d \u201cbook store\u201d)<\/li>\n
        2. read and understand a train or bus schedule<\/li>\n
        3. read and understand a restaurant menu<\/li>\n
        4. find information that I need in a telephone directory<\/li>\n
        5. read office memoranda written to me in which the writer has used simple words or sentences<\/li>\n
        6. read and understand traffic signs<\/li>\n
        7. read and understand simple, step-by-step instructions<\/li>\n
        8. read and understand an agenda for a meeting<\/li>\n
        9. read and understand a travel brochure<\/li>\n
        10. read and understand magazine articles like those found in Time or Newsweek, without using a dictionary<\/li>\n
        11. read and understand directions and explanations presented in computer manuals written for beginning users<\/li>\n
        12. identify inconsistencies or differences in points of view in two newspaper interviews with politicians of opposing parties<\/li>\n
        13. read highly technical material in my field or area of expertise with no use or only infrequent use of a dictionary<\/li>\n
        14. read and understand a popular novel<\/li>\n
        15. read and understand a letter of thanks from a client or customer<\/li>\n<\/ol>\n

          IN WRITING, I CAN<\/p>\n

            \n
          1. write a list for items to take on a weekend trip<\/li>\n
          2. write a one- or two-sentence thank-you note for a gift a friend sent to me<\/li>\n
          3. write a brief note to a co-worker explaining why I will not be able to attend the scheduled meeting<\/li>\n
          4. write a postcard to a friend describing what I have been doing on my vacation<\/li>\n
          5. fill out an application form for a class at night school<\/li>\n
          6. write clear directions on how to get to my house or apartment<\/li>\n
          7. write a letter requesting information about hotel accommodations for a future vacation<\/li>\n
          8. write a short note to a co-worker describing how to operate a standard piece of office equipment (e.g., photocopier, fax machine)<\/li>\n
          9. write a memorandum to my supervisor explaining why I need a new time off from work<\/li>\n
          10. write a letter introducing myself and describing my qualifications to accompany an employment application<\/li>\n
          11. write a memorandum to my supervisor describing the progress being made on a current project or assignment<\/li>\n
          12. write a complaint to a store manager about my dissatisfaction with an appliance I recently purchased<\/li>\n
          13. write a letter to a potential client describing the services and\/or products of my company<\/li>\n
          14. write a 5-page formal report on a project in which I participated<\/li>\n
          15. write a memorandum summarizing the main points of a meeting I recently attended<\/li>\n<\/ol>\n

            Appendix B: Factor Loadings for TOEIC Can-Do Questionnaire<\/h3>\n

            <\/p>\n

            <\/p>\n

            Appendix C: Comparisons of University Students\u2019 Ratings of Confidence in Five Communicative Domains<\/h3>\n\n\n\n\n\n\n\n\n\n
            <\/td>\n<\/td>\n1st yeara<\/sup><\/td>\n2nd yearb<\/sup><\/td>\n3rd yearc<\/sup><\/td>\n4th yeard<\/sup><\/td>\n<\/tr>\n
            Factor<\/td>\n<\/td>\nM <\/em>
            \n(SD<\/em>)<\/td>\n            		95% CI<\/td>\nM<\/em>
            \n(SD<\/em>)<\/td>\n            		95% CI<\/td>\nM<\/em>
            \n(SD<\/em>)<\/td>\n            		95% CI<\/td>\nM<\/em>
            \n(SD<\/em>)<\/td>\n            		95% CI<\/td>\n<\/tr>\n
            Listening<\/td>\n<\/td>\n2.32 (0.68)<\/td>\n[2.16, 2.48]<\/td>\n2.32 (0.71)<\/td>\n[2.15, 2.48]<\/td>\n2.44 (0.63)<\/td>\n[2.24, 2.64]<\/td>\n2.44 (0.72)<\/td>\n[2.27, 2.60]<\/td>\n<\/tr>\n
            Speaking<\/td>\n<\/td>\n3.13 (0.68)<\/td>\n[2.97, 3.30]<\/td>\n3.20 (0.65)<\/td>\n[3.03, 3.38]<\/td>\n3.11 (0.73)<\/td>\n[2.90, 3.32]<\/td>\n3.03 (0.80)<\/td>\n[2.86, 3.20]<\/td>\n<\/tr>\n
            Interaction<\/td>\n<\/td>\n1.67 (0.52)<\/td>\n[1.53, 1.81]<\/td>\n1.80 (0.64)<\/td>\n[1.65, 1.95]<\/td>\n1.82 (0.66)<\/td>\n[1.64, 2.00]<\/td>\n1.75 (0.62)<\/td>\n[1.60, 1.89]<\/td>\n<\/tr>\n
            Reading<\/td>\n<\/td>\n2.97 (0.76)<\/td>\n[2.80, 3.13]<\/td>\n3.05 (0.65)<\/td>\n[2.88, 3.22]<\/td>\n3.34 (0.72)<\/td>\n[3.13, 3.54]<\/td>\n3.25 (0.72)<\/td>\n[3.09, 3.42]<\/td>\n<\/tr>\n
            Writing<\/td>\n<\/td>\n2.18 (0.72)<\/td>\n[2.03, 2.34]<\/td>\n2.43 (0.59)<\/td>\n[2.27, 2.59]<\/td>\n2.45 (0.63)<\/td>\n[2.26, 2.64]<\/td>\n2.40 (0.70)<\/td>\n[2.24, 2.55]<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

            Note<\/em>. CI = confidence interval.
            \na<\/sup>n<\/em> = 73. b<\/sup>n<\/em> = 67. c<\/sup>n<\/em> = 47. d<\/sup>n<\/em> = 70.<\/p>\n

            Appendix D: Individual Responses in Each Communicative Domain<\/h3>\n

            <\/p>\n

            Note<\/em>. Boxplots show the maximum, third quartile, median, first quartile, and minimum of the data. Beeswarm plots show individual responses.<\/p>\n

            Appendix E: Experience Factors on Self-Assessment<\/h3>\n

            \"Experience<\/p>\n

            \"Experience<\/p>\n

            \"Experience<\/p>\n

            \"Experience<\/p>\n

            \"Experience<\/p>\n\n\n\n
            Copyright rests with authors. Please cite TESL-EJ appropriately.
            \nEditor’s Note:<\/strong> The HTML version contains no page numbers. Please use the PDF version<\/a> of this article for citations.<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n","protected":false},"excerpt":{"rendered":"

            February 2018 – Volume 21, Number 4 Shinya Ozawa Hiroshima Shudo University, Japan <ozawashudo-u.ac.jp> Abstract This survey was conducted to investigate how university students gain or lose confidence in English communicative domains over four years at university. Self-assessment has been a useful instrument for measuring learners\u2019 English proficiency, and the students in this study were […]<\/p>\n","protected":false},"author":3,"featured_media":0,"parent":12797,"menu_order":3,"comment_status":"closed","ping_status":"closed","template":"","meta":{"_genesis_hide_title":false,"_genesis_hide_breadcrumbs":false,"_genesis_hide_singular_image":false,"_genesis_hide_footer_widgets":false,"_genesis_custom_body_class":"","_genesis_custom_post_class":"","_genesis_layout":"","footnotes":""},"class_list":["post-12864","page","type-page","status-publish","entry"],"featured_image_src":null,"featured_image_src_square":null,"_links":{"self":[{"href":"https:\/\/tesl-ej.org\/wordpress\/wp-json\/wp\/v2\/pages\/12864","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/tesl-ej.org\/wordpress\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/tesl-ej.org\/wordpress\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/tesl-ej.org\/wordpress\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/tesl-ej.org\/wordpress\/wp-json\/wp\/v2\/comments?post=12864"}],"version-history":[{"count":6,"href":"https:\/\/tesl-ej.org\/wordpress\/wp-json\/wp\/v2\/pages\/12864\/revisions"}],"predecessor-version":[{"id":12944,"href":"https:\/\/tesl-ej.org\/wordpress\/wp-json\/wp\/v2\/pages\/12864\/revisions\/12944"}],"up":[{"embeddable":true,"href":"https:\/\/tesl-ej.org\/wordpress\/wp-json\/wp\/v2\/pages\/12797"}],"wp:attachment":[{"href":"https:\/\/tesl-ej.org\/wordpress\/wp-json\/wp\/v2\/media?parent=12864"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}