\nHimeji Dokkyo University, Japan.
\n<gilbert.dizon.jr
![\"atmark\"](\"http:\/\/www.tesl-ej.org\/atmark.png\")
gmail.com><\/p>\nAbstract<\/h3>\n
The Internet has made it possible for teachers to administer online assessments with affordability and ease. However, little is known about Japanese English as a Foreign Language (EFL) students\u2019 attitudes of internet-based tests (IBTs). Therefore, this study aimed to measure the perceptions of IBTs among Japanese English language learners with the technology acceptance model (TAM), the most influential and widely employed model to predict a user\u2019s adoption of new technologies (Lee, Kozar, Larsen, 2003). However, TAM has yet to be utilized in the context of English education in Japan. Thus, the study also sought to determine the correlational relationships between three TAM variables \u2013 perceived usefulness (PU), perceived ease of use (PEOU), and the behavioral intention (BI) to use IBTs. An adapted TAM questionnaire was administered to 80 Japanese university students. The results showed that the relationships between PU, PEOU, and BI were positively and highly correlated. More importantly, students\u2019 attitudes towards the PU, PEOU, and BI to use IBTs were found to be moderately favorable, with PU and BI more favorably perceived than PEOU. The findings verified that the four proposed hypotheses were supported, providing evidence that IBTs are viewed positively by Japanese EFL students. <\/p>\n
Keywords<\/em>: technology acceptance model, internet-based test, online assessment, L2 assessment <\/p>\n As Roever (2001) noted, there has been considerable interest in internet-based tests (IBTs) in the language testing community due to their advantages over traditional paper-based tests (PBTs). Not only do IBTs provide immediate feedback, improved scoring accuracy, and a richer interface (Noyes & Garland, 2008), but they also offer more flexibility and convenience (Roever, 2001). However, as this interest grows and IBTs become more widespread in second language classrooms, so does the need to examine students\u2019 attitudes towards them as helpful and easy to use tools for assessment. This study utilizes the technology acceptance model (TAM), the most widely used theoretical model to explain a user\u2019s technological acceptance. However, TAM has yet to be employed in Japan to study English as a Foreign Language (EFL) students\u2019 views of a given technology. Therefore, the purpose of this study is twofold: first, to examine the correlational relationships between three TAM variables, perceived usefulness (PU), perceived ease of use (PEOU) and behavioral intention (BI) to use IBTs; and second, to measure Japanese EFL student perceptions of IBTs with a TAM questionnaire.<\/p>\n The increase in the usage of technology in language assessment has provided insight into L2 students\u2019 attitudes towards computer-based tests (CBTs) and IBTs. Much of this research has focused on standardized exams, namely, the TOEFL test. One such study is Striker and Wilder\u2019s (2001) investigation of examinees\u2019 opinions of the TOEFL CBT. A total of 689 test takers in three cities, Buenos Aires, Cairo, and Frankfurt, were surveyed in the spring and summer of 1999. The results of the study indicated that the students had moderately favorable attitudes about the exam. These findings confirm the results from previous studies on other CBTs (Powers & O\u2019Neill, 1993; Schmitt, Gilliland, Landis, & Devine, 1993; Schmidt, Urry, & Gugel, 1978). In all of these studies including Striker and Wilder\u2019s (2001), favorable attitudes were found once the test takers had completed testing. Additionally, in Powers and O\u2019Neill\u2019s (1993) study, students were also asked about their views of CBTs prior to testing and were found to have more favorable attitudes after completion compared to their pre-test opinions. As Striker and Wilder (2001) pointed out, this suggests that their test taking experiences on the computer may have been better than they had anticipated.<\/p>\n One study which has measured test takers\u2019 opinions of IBTs is Nobandegani\u2019s (2012) investigation of the internet-based TOEFL exam. One hundred examinees of the TOEFL iBT in Iran were asked to specify their level of agreement to statements based on research of the advantages and disadvantages of CBTs. According to her findings, the disadvantages slightly outweighed the advantages; specifically, computer familiarity, computer anxiety, and difficulty with computer equipment were all perceived as limitations of IBTs. On the other hand, the participants indicated that IBTs offered several benefits such as administrative flexibility, accuracy and immediacy of scoring, as well as greater standardization. While IBTs may have significant advantages, these findings illustrate that their disadvantages need to be carefully addressed before implementing them into a teaching context. <\/p>\n In a much larger study of the TOEFL iBT, Striker and Attali (2010) measured the attitudes of 762 examinees in China, Columbia, Egypt, and Germany. They found that test takers in three of them had moderately favorable attitudes towards the test, with participants in Germany viewing it neutrally. This is in contrast with Striker and Wilder\u2019s (2001) prior study on students\u2019 attitudes towards the computer-based TOEFL exam in which all three areas held positive views of the test. Striker and Attali (2010) also looked at the test takers\u2019 opinions of the different sections of the exam. Although the participants in all four of the countries had favorable views of the listening, reading, and writing sections, the speaking portion was perceived to be less favorable or unfavorable. <\/p>\n At the classroom level, there have been relatively few studies examining L2 students\u2019 opinions of IBTs or online assessment. One exception is Hirshel\u2019s (2012) investigation of Japanese university students\u2019 views on online forums, glossaries, and quizzes with regard to potential opportunities for learning and interaction, awareness of lexical and grammatical patterns, as well as comprehensible output. A total of 23 EFL students participated in the study. An online survey including seven open-ended questions was distributed, and students were also interviewed by the researcher to obtain more detailed responses. Based on his findings, the perception of online quizzes was generally positive, with one student remarking \u201cI think that [the vocabulary quizzes are] very useful for me. Because there are various kinds of questions or divide some parts. It make me master how to use the certain vocab\u201d (p. 105). <\/p>\n Research framework: Technology acceptance model <\/p>\n A wide-range of theoretical models have been introduced to measure a user\u2019s degree of technological acceptance. However, TAM has emerged as \u201cthe most influential and commonly employed theory for describing an individual\u2019s acceptance of information systems\u201d (Lee, Kozar, & Larsen, 2003, p. 752). Illustrating the impact that it has had on the information systems (IS) field, Lee et al. (2003) found that by 2003, the first two articles concerning TAM by Davis (1989) and Davis, Bagozzi, and Warshaw (1989) had received 698 journal citations in the Social Science Citation Index. Because of its prominence in IS and its established validity and reliability, TAM was chosen as the framework for the present study. <\/p>\n TAM is an adaption of the Theory of Reasoned Action (TRA) by Fishbein and Ajzen (1975). While TRA attempts to explain the reasons for behavior in a variety of domains, TAM aims to understand an individual\u2019s behavioral intention (BI) towards the use a technology. The final version of TAM developed by Venkatesh and Davis (1996) consists of four variables: perceived usefulness (PU), perceived ease of use (PEOU), external variables, and BI (Figure 1). Davis (1989) refers to PU as \u201cthe degree to which a person believes that using a particular system would enhance his or her job performance\u201d (p. 320). According to Davis (1989), PEOU is \u201cthe degree to which a person believes that using a particular system would be free of effort\u201d (p. 320). A user\u2019s BI is jointly determined by his or her PU and PEOU of the technology. External variables include factors such as user training, user involvement in design, system characteristics, and the nature of the implementation process (Venkatesh & Davis, 1996). <\/p>\n Of the two variables that directly affect BI, PU has been found to be more influential. Lee et al. (2003) investigated the relationship between the three main variables in TAM (PU, PEOU, and BI) in 101 studies and found that 74 of them revealed a significant relationship between PU and BI. In contrast, the researchers found that only 58 studies showed a significant relationship between PEOU and BI. Nevertheless, PEOU was found to be a significant antecedent of PU with 69 of the studies showing a significant relationship between PU and PEOU. As Lee et al. (2003) stated, this suggests that PEOU can indirectly impact the acceptance of technology through PU. <\/p>\n While numerous studies have investigated learners\u2019 willingness to adopt e-learning systems using TAM (e.g., Hsu, Wang, & Chiu, 2009; Lee, Cheung, & Chen, 2005; Ong, Lai, & Wang, 2004; Park, 2009; Saade\u0301 & Bahli, 2005), fewer studies have employed the model in the context of second language learning. One recent study is Soleimani, Ismail, and Mustaffa\u2019s (2014) examination of mobile assisted language learning (MALL) among postgraduate ESL students in Malaysia. The twenty-five participants were asked nine questions concerning their level of agreement according to 5-point Likert scale and the results were analyzed using descriptive statistical analysis. Soleimani et al. (2014) found that the mean values of PEOU (m=3.68), PU (m=3.40), and BI (3.13) were high, indicating a positive perception towards MALL by the students. <\/p>\n Another TAM study conducted in a L2 context is Mah and Er\u2019s (2009) investigation of the correlational relationships between PEOU, PU, and BI with Pearson\u2019s correlation coefficient. The study consisted of a target sample of 58 participants from a total population of 918 Malaysian ESL students. Prior to the study, a pilot test involving 37 students was performed which verified the reliability of the 15-item questionnaire with Cronbach\u2019s alpha. All three of the variables analyzed, PEOU, PU, and BI, had Cronbach\u2019s alpha values of over 0.9, indicating a very high level of internal consistency. Mah and Er (2009) found that there were strong links between the three relationships examined (PEOU and PU; PU and BI; PEOU and BI) to write web logs in the ESL classroom, thereby providing validity to TAM questionnaires as research tools in second language educational contexts. <\/p>\n Based on the research objectives and the aforementioned literature, the following four hypotheses were proposed: <\/p>\n H1: There is a positive and significant relationship between PU and BI to use IBTs among Japanese EFL students. Data were collected with a questionnaire containing items related to the PU, PEOU, and BI to use IBTs. The correlational relationships of the TAM variables were measured with Pearson\u2019s correlation coefficient using SPSS. Per Boone and Boone\u2019s (2012) recommendation concerning Likert scale data, a descriptive statistical analysis detailing the mean and SD of each construct was carried out to measure the level of acceptance of IBTs among the students. <\/p>\n The participants in the study included 80 male and female first-, second-, third-, and fourth-year students who were enrolled in one or more English language classes taught by the researcher during the 2014 spring semester at Himeji Dokkyo University, representing a non-random sample of the target population. The students\u2019 L2 English proficiency levels ranged from beginner to intermediate. More than half of the students (n=45) were part of the Faculty of Foreign Studies. The remaining 35 students belonged to the Faculty of Pharmacology and were required to take an English for Specific Purposes (ESP) class as a departmental prerequisite. Only students who took both the midterm and final exams were included in the study. <\/p>\n All students in the present study had summative assessments (i.e., midterm and final exams) administered through the Internet via classmarker.com<\/a> (Figure 2). The testing website was chosen because of the key features it offered, specifically, multiple interface languages including Japanese, access to exams via password-protected links, as well as the ability to randomize test items and answers. Exams were created by the researcher and were comprised multiple choice, cloze, grammar, and essay questions based on the content of the respective classes. Although the medium of the exam items and answers was English, the medium of the website was Japanese in order to enhance ease of use and avoid any misunderstandings regarding the navigation and use of the website. Summative assessments were monitored and conducted in class due to the stakes involved and to reduce the likelihood of cheating.<\/p>\n As Hackbarth, Grover, and Yi (2003) noted, \u201cUsers generally perceive a system easier to use as they gain more knowledge and confidence through direct experience in using the system\u201d (p. 221). Therefore, students were given several opportunities to familiarize themselves with the website used for testing. A week prior to taking the midterm exam, a brief orientation was conducted in each class to explain how to access and navigate the exams as well as to detail some of the perceived student benefits of IBTs such as increased scoring accuracy and immediate feedback (Noyes & Garland, 2008). Following the orientation, the students took a non-graded practice quiz which allowed them to become comfortable with the test interface. Finally, formative assessments were administered to further increase familiarity and help them review for the midterm exams. Formative assessments were also administered a week before the final exams for these same reasons.<\/p>\n A 10-item self-report questionnaire measuring PU, PEAU, and BI was used in the study (Appendix A). All items were adapted from existing TAM literature (Appendix B), thereby ensuring validity. While there may be some concerns regarding validity when using recycled items, Hyman, Lamb, and Bulmer (2006) asserted that the degree of validity is likely to be high if the questions have been pre-tested to confirm that they are accurate measures of the concepts of interest. The reliability of the questionnaire was verified with Cronbach\u2019s alpha (\u03b1) using SPSS. Because the values of PU (\u03b1= .849), PEOU (\u03b1= .863), and BI (\u03b1= .729) are higher than 0.7, the internal consistency of the three subscales is considered acceptable. Table 1 shows the Cronbach\u2019s alpha of each construct.<\/p>\n Table 1<\/strong>Introduction<\/h3>\n
Literature Review<\/h3>\n
Computer-based and internet-based tests<\/h4>\n
![](\"http:\/\/tesl-ej.org\/ej78\/a2pix\/image1.jpg\")
\nFigure 1<\/em>. Technology acceptance model<\/p>\nResearch hypotheses<\/h3>\n
\nH2: There is a positive and significant relationship between PEOU and BI to use IBTs among Japanese EFL students.
\nH3: There is a positive and significant relationship between PEOU and PU of IBTs among Japanese EFL students.
\nH4: There is a positive perception among Japanese EFL students concerning the usefulness, ease of use, and BI to use IBTs. <\/p>\nMethodology<\/h3>\n
Research design<\/h4>\n
Participants<\/h4>\n
Online Assessments<\/h4>\n
![](\"http:\/\/tesl-ej.org\/ej78\/a2pix\/image2.jpg\")
\nFigure 2<\/em>. ClassMarker homepage<\/p>\nLearner training<\/h4>\n
Instrument<\/h4>\n
\nCronbach\u2019s alpha of PU, PEOU, and BI Constructs <\/em><\/p>\n