August 2023 – Volume 27, Number 2
https://doi.org/10.55593/ej.27106a10
Masoomeh Estaji
Allameh Tabataba’i University, Tehran, Iran
<mestaji74
gmail.com>
Zahra Banitalebi
Allameh Tabataba’i University, Tehran, Iran
<zahra.banitalebigmail.com>
Abstract
Despite a large number of studies on teachers’ assessment literacy (AL) and the increasing interest in digital assessment, there is a paucity of research on teachers’ AL in digital environments (TALiDE). To bridge this gap, the current study explored the effects of the COVID-19 pandemic on TALiDE, the roles, and competencies underlying TALiDE, and whether technology needs to be considered in the conceptualization of AL. The data were collected from an open-ended questionnaire and a set of interviews investigating the perspectives of 27 participants (12 experts and 15 EFL teachers). The findings, inferred from content and thematic analyses of data, reflected teachers’ raised awareness as a result of the COVID-19 pandemic experience, which resulted in an understanding of new approaches to assessment and increased value of TALiDE. Furthermore, compared to the existing literature on AL, roles and competencies underlying TALiDE corresponded to five main subthemes: teacher as a knower, implementer, confident technology user, moral agent, and learner. Finally, findings speculated a potential role for technology in the AL concept. This study can provide teacher trainers, educational policymakers, and course designers with implications for designing teacher development courses and programs to integrate TALiDE into their training practices.
Keywords: Assessment literacy, COVID-19 pandemic, teacher education, technology-enhanced assessment
Assessment data nowadays act as a code for measuring quality (Broadfoot & Black, 2004). Widely addressed by educational researchers, standards for quality teaching and assessment, and the current accountability framework of education (DeLuca et al., 2016), assessment literacy (AL) is known as a professional requirement for teachers. Generally, teachers’ competence in educational assessment is called AL (Stiggins, 1991). Such competence includes several aspects of knowledge and skills in doing classroom assessments, such as ensuring reliability and validity, designing appropriate assessment activities, and grading and reporting the results skillfully (DeLuca et al., 2016). Although several definitions have been proposed for AL, beginning with Stiggins (1991), a unanimous definition of AL is not currently available. This lack of consensus is attributed to the dynamicity and fluidity of the concept of AL that develops with the associated literature over time (DeLuca et al., 2016; Williams, 2015). As Willis et al. (2013) highlighted assessment literacies in plural form can better acknowledge different skills that teachers need as they engage in different assessment discourses. Since teachers are the agents of classroom assessment, such discourses mainly depend on the context of their classrooms.
Recently, classroom assessment has undergone remarkable changes due to technological enhancements and expansion promoted by the COVID-19 pandemic (Farhady, 2022; Timmis et al., 2016). For example, there has been an increasing shift to online teaching since the COVID-19 pandemic. According to Liang and Creasy (2004), the unique characteristics of digital environments require appropriate assessment skills. As an example of such digital environments, online classrooms require teachers to have extended knowledge and skills to take the most of the particularities of technological facilities and shortcomings to deliver quality assessment practices. Adaptable to environments beyond online classrooms, teachers’ AL in digital environments (TALiDE), or what Eyal (2012) calls digital AL (DAL), refers to the competencies that enable teachers as assessors in technology-enhanced environments to carry out quality assessments (Eyal, 2012). Eyal (2012) stresses that teachers in digital environments need a set of competencies that incorporate general skills to design, implement, and evaluate assessment tasks, along with skills and knowledge to tailor these assessment practices to a technology-enhanced environment. For example, teachers in digital environments need competency to select and use a range of assessment tools that align with their educational goals as well as technological skills to use learning management systems (Eyal, 2012). While Eyal’s (2012) attempt to identify TALiDE competencies is worthwhile, new assessment scenarios and potentials, such as automatic feedback, have been introduced to classroom assessment since the COVID-19 pandemic (Guerrero-Roldán & Noguera, 2018). Moreover, the role of technology in language assessment and its requirements for language teachers’ AL needs further investigation (Coombe & Davidson, 2022; Farhady, 2019). To understand TALiDE in language education, this study aimed to investigate experts’ (i.e., researchers in the field of assessment and teacher educators) and English as a Foreign Language (EFL) teachers’ perspectives regarding teachers’ roles and competencies in digital assessment. A post-COVID-19 understanding of TALiDE can inform teacher educators’, course designers’, and educational policymakers’ decision-making regarding language teacher professional development endeavors.
Literature Review
Assessment Literacy
First introduced by Stiggins (1991), AL has evolved to include a complex of professional skills over the years (DeLuca et al., 2020). Despite the fact that the multidimensional and evolving nature of AL prevents reaching a solid definition of teachers’ AL (Inbar-Lourie, 2013), attempts have been made to identify the various facets of this construct (e.g., DeLuca et al., 2016; Giraldo, 2018; Pastore & Andrade, 2019; Xu & Brown, 2016). For example, Xu and Brown (2016) developed the Teacher Assessment Literacy in Practice (TALiP) model consisting of a multitude of dimensions, namely teachers’ knowledge base, conceptions of assessment, institutional and socio-cultural contexts, AL in practice, learning, and assessment identity. They highlighted that an accumulation of assessment knowledge does not lead to AL, rather AL development entails a set of assessment competencies on which teachers’ development, larger socio-cultural context, and conceptions of assessment are influential.
Similar to Xu and Brown’s (2016) conceptualization of AL, Pastore and Andrade (2019) presented a model of AL including the conceptual, praxeological, and socio-emotional dimensions. The conceptual dimension includes teachers’ knowledge and conceptions about assessment, while the other two dimensions derive from the practical and social aspects of assessment, respectively. Reviewing the recent standards on assessment and measures developed for AL, DeLuca et al. (2016) argued that the existing measures are inadequate as they are ignorant of the recent developments in assessment. Thus, they developed a model of AL, embodied in approaches to classroom assessment, which includes four factors: assessment purposes, assessment processes, assessment fairness, and measurement theory (DeLuca et al., 2016).
Concerning language teachers’ AL, Giraldo (2018) conducted a conceptual review of the literature and proposed a three-component model of AL: knowledge, skills, and principles. These components, in return, give rise to other subdimensions. Knowledge in this framework is composed of three dimensions, namely the awareness of applied linguistics, theory and concepts, and language assessment context, which highlight the competencies to compare theories of applied linguistics, knowledge of reliability and validity, and cognizance of assessment culture. As the second component, skills include four dimensions: instructional skills, skills for language assessments, skills in educational measurement, and technological skills. All these skills focus on planning, utilizing, and interpreting the assessment tools in the actual assessment practices. Lastly, the principles call attention to the concerns of fairness, bias, and ethics in language assessment. Figure 1 shows the major components of AL models outlined in this subsection.
Figure 1. Major Models/Frameworks of AL
While acknowledging these major models of AL, where relevant, this study mainly focused on the model proposed by Xu and Brown (2016) as it was found to be more comprehensive, encompassing many of the elements mentioned in other models or frameworks.
Technology-Enhanced Assessment
Technology-enhanced assessment is a broad notion that embraces diverse techniques that use technology to assist the performance and delivery of assessment practices (Timmis et al., 2016). With the emergence of COVID-19 and the consequent intensive shift to online teaching, the importance of assessment in technology-enhanced environments has gained renewed attention (Cooper et al., 2022). Many scholars examined how the outbreak of the COVID-19 pandemic affected teachers’ classroom assessment practices, featuring the need for teachers to develop their TALiDE (e.g., Cooper et al., 2022; García-Peñalvo et al., 2021; Ghanbari & Nowroozi, 2021; Ibna Seraj et al., 2022; Lee et al., 2022; Pu & Xu, 2021; Tian et al., 2021; Zhang et al., 2021).
Cooper et al. (2022), referring to this phenomenon as “emergency remote assessment” (p. 1), conducted interviews with a group of 17 school teachers in Canada to understand their lived experiences with online assessment. The results showed that teachers’ engagement in feedback provision happened to those with a strong technology background before the pandemic. Additionally, those less versed in adopting technology in their assessment reported some concerns regarding grading. Finally, they concluded that the pandemic affected classroom assessment in a way that would never be the same, providing implications to move toward technology integration, prioritizing feedback, and developing collaborative tasks and equity.
Similarly, García-Peñalvo et al. (2021) argued for reconsideration in the design of online education that takes into account the shift away from the transmission to transformation. It means that, rather than transferring content to the digital environment, an appropriate digital transformation in which assessment takes advantage of a rich digital environment should happen. Ghanbari and Nowroozi, (2021) highlighted a similar concern that teachers did not know whether to adopt the previous face-to-face assessment mechanisms or increase the load of formative assessment.
Furthermore, Lee et al. (2022), relying on a survey of 782 students, pointed out that teachers’ skills in using formative assessment were found less satisfactory. Teachers could assist self and peer assessment and use technologies to communicate in real-time. Moreover, teachers need to work on giving detailed and timely feedback, avoiding cheating, designing more challenging application questions, and revising the assessment rubric for presentation in virtual contexts.
Tian et al. (2021) also investigated the changes in AL of three language teachers in Asia during the COVID-19 pandemic and its resultant school shutdowns through reflective dialogues. They found that over time teachers became more concerned with the practitioners’ roles regarding language assessment and the gap between their received instruction in assessment and their real practices. Other studies, focusing on assessment language teachers’ AL (Alavi et al., 2022; Pu & Xu, 2021; Zhang et al., 2021), signified the importance of disseminating TALiDE and reported that for assessment in digital environments, teachers lack enough ability to use technology in designing, implementing, and evaluating assessment tasks, which all point to the dimensions of TALiDE.
The Study
More recently, scholars argued that the realities of the COVID-19 pandemic demanded modifications in classroom assessment, which impose long-term effects on post-pandemic practices as well (Cooper et al., 2022). According to Brookhart (2011), the field of AL needs to heed contemporary classroom assessment realities if it is to accurately reflect the teachers’ assessment practices. As digital assessment is currently becoming commonplace, a deeper understanding of AL in digital assessment is indisputable to ensure the representativeness of AL “in light of transformations in the assessment landscape” (Gotch & French, 2014, p. 17). Furthermore, since the professional development programs are responsible for preparing competent teachers for current assessment situations in which technology plays a role, a greater understanding of the particular roles and competencies the teachers need in classroom assessment in digital environments would help teacher educators and policymakers to improve the quality of courses and programs in the situated contexts of practice. Such a requirement for teachers to be competent in the 21st century is also accentuated by the technological pedagogical content knowledge (TPACK) model, which highlights the intersection of teacher knowledge categories and professional knowledge about digital technologies for application in classroom situations (Mishra & Koehler, 2006).
Overall, the importance of AL is vivid, and AL in digital environments is not an exception, especially with the ever-developing role of technology. In light of critiques directed at current AL definitions and measurements (Gotch & French, 2014), this study argued that transformation to online teaching might affect the teachers’ roles and competencies in digital assessment and the representativeness of the AL concept. To do so, the present study was a current state analysis of experts’ (i.e., researchers and teacher educators) and EFL teachers’ perspectives concerning the effect of the COVID-19 pandemic on their TALiDE, the roles and competencies required by teachers to deliver effective classroom assessment in digital environments, and the status of technology in AL conceptualization. To this end, the following questions were addressed in this study.
- How do teachers’ experiences of the COVID-19 pandemic affect their assessment literacy in digital environments (TALiDE)?
- What roles and competencies are required for teacher assessment literacy in digital environments (TALiDE) from teachers’ and experts’ perspectives?
- From teachers’ and experts’ perspectives, to what extent does technology need to be included in the conceptualization of assessment literacy (AL)?
Methodology
The present study was a descriptive method, a qualitative study in which the effect of the COVID-19 pandemic on TALiDE, the roles and competencies underlying TALiDE, and the status of technology in AL conceptualization were investigated through experts’ and EFL teachers’ lenses. According to Creswell and Creswell (2023), this approach requires the researchers to interact with the data, use limited frameworks to interpret the data, and categorize the information into themes.
Participants
The participants taking part in this study were recruited based on nonprobability convenience sampling (Ary et al., 2010). They comprised 12 experts (i.e., researchers who have published papers in AL and/or digital assessment and were teacher educators, except for one, teaching assessment courses as well), and 15 Iranian EFL teachers who used digital technology in classroom assessment during the COVID-19 pandemic. It should be noted that this study followed Ericsson and Smith’s (1991) definition of experts as those who have shown high-quality performance in a domain (Kazemian et al., 2020). In the present study, 12 experts participated in the study who were esteemed, published researchers in the field of AL or digital assessment worldwide. Either their publications were published in SCOPUS-indexed education-and-assessment-related journals (Estaji et al., forthcoming), or their h-indices indicated their high performance. The participants’ biographies, including their roles, gender, academic degree, and experience in digital assessment, are summarized in Table 1 and Table 2. To observe the anonymity of the participants, they are hereafter referred to as EP for expert participants and TP for teacher participants.
The experts were from different countries and nations (Iran, Norway, Turkey, New Zealand, The United States, and other nations), while all teacher participants were Iranian EFL teachers, as the researchers had direct access to them. The reason behind choosing two groups of participants (i.e., experts and teachers) was to enrich the study with different perspectives. This study continued data collection and analysis until new themes did not emerge, leading to data saturation as recommended by Ary et al. (2010).
Table 1. Descriptive Data of Expert Participants
| Name | Gender | Google Scholar (h-index) |
University Rank | The Highest Degree Completed | Role |
| P1 | Male | 10 | Assoc. Professor | PhD in ELT | Teacher Educator/Researcher |
| P2 | Female | 5 | Lecturer | PhD in ELT | Teacher Educator/Researcher/AL Course Designer |
| P3 | Male | 22 | Lecturer | PhD in ELT | Teacher Educator/Researcher |
| P4 | Male | 64 | Full Professor | PhD in Education | Teacher Educator/Researcher |
| P5 | Male | 32 | Full Professor | PhD ELT | Teacher Educator/Researcher |
| P6 | Male | 11 | Full Professor | PhD in Education | Teacher Educator/Researcher |
| P7 | Female | 15 | Assist. Professor | PhD in ELT | Teacher Educator/Researcher |
| P8 | Female | 8 | Lecturer | PhD in Education | Teacher Educator/Researcher |
| P9 | Female | 9 | – | PhD in Instructional Technology | Researcher |
| P10 | Male | 18 | Assist. Professor | PhD in ELT | Teacher Educator/Researcher |
| P11 | Male | 14 | Assist. Professor | PhD in Education | Teacher Educator/Researcher |
| P12 | Female | 26 | Assist. Professor | PhD in ELT | Teacher Educator/Researcher |
Note: a. All the teacher educators have held assessment courses for teachers. All researchers have published papers in the field of assessment literacy/digital assessment b. ELT: English Language Teaching
Table 2. Descriptive Data of Teacher Participants
| Name | Gender | The Highest Degree in ELT |
Years of Teaching Experience |
Years of Online Teaching Experience |
Context(s) of Language Teaching |
Age |
| TP1 | Female | MA | 5 | 2 | School | 26 |
| TP2 | Female | PhD | 11 | 3 | School/language institute | 32 |
| TP3 | Female | MA | 3 | 2 | Language institute | 24 |
| TP4 | Male | MA | 5 | 3 | School | 28 |
| TP5 | Female | MA | 4 | 2 | School/language institute | 26 |
| TP6 | Female | MA | 6 | 2 | School/language institute | 29 |
| TP7 | Female | PhD | 19 | 3 | School/language institute | 44 |
| TP8 | Male | MA | 7 | 3 | School/language institute | 33 |
| TP9 | Female | PhD | 6 | 3 | School/language institute | 29 |
| TP10 | Female | PhD | 15 | 2 | School/language institute | 38 |
| TP11 | Female | MA | 8 | 3 | School | 30 |
| TP12 | Female | PhDc | 8 | 3 | School | 31 |
| TP13 | Male | MA | 6 | 3 | School/language institute | 35 |
| TP14 | Male | MA | 6 | 2 | Language institute | 29 |
| TP15 | Male | MA | 5 | 3 | School | 25 |
Note: PhDc: PhD Candidate
Like many other countries, education in Iran shifted online by the COVID-19 breakout. In spite of the fact that at the time of the pandemic the teachers were provided with brief guidance on how to use learning management systems to get prepared for such an abrupt change, teacher education programs in Iran usually lack a focus on the implementation of technology in assessment. Despite the deficiencies, teachers and students have had positive attitudes toward online education, and it might be likely to continue even in the post-pandemic era (Ghanbari & Nowroozi, 2021).
Instruments
This study used two instruments, an open-ended questionnaire and semi-structured interviews, to collect data from the experts and classroom teachers, respectively. As access to experts was limited, conducting interviews with them was not possible for the researchers. Therefore, an open-ended questionnaire was designed to retrieve their perspectives. However, as suggested by Ary et al. (2010), semi-structured interviews were selected to collect more naturalistic data from classroom teachers.
Open-Ended questionnaire. After reviewing the literature concerning AL and digital assessment, the researchers developed an open-ended questionnaire and validated content-wisely by three experts in the field of educational assessment. The three experts were Ph.D. holders, one in education and the other in language teaching. Two were full professors, and the other was an associate professor. They have worked on assessment and teacher education for over a decade. One of the experts was from the same cohort of expert participants and the other two were familiar with the aims of the study. With experts’ comments, some minor modifications were made. For example, one of the experts suggested the provision of aspects of the challenges teachers face, such as cultural, personal, contextual, and administrative, for Question 11. The questionnaire contained six questions (Appendix A) responding to three main aspects reviewed in the literature regarding AL and digital assessment: (1) the use of technology in assessment (Brown, 2017; Guerrero-Roldán & Noguera, 2018), (2) the need to include technology in AL conceptualization (Coombe & Davidson, 2022), and (3) the roles and competencies of teachers in digital assessment (Eyal, 2012; Mphahlele, 2022).
Semi-Structured interviews. Regarding the interviews, as the next research instrument in this study, a greater number of questions were used based on the available models in the literature on AL and the three previously mentioned aspects in the open-ended questionnaire. Such an extension aimed to gain a deeper understanding of the teachers’ views as they are the agents of classroom assessment and can inform the study of what is needed and done in the actual technology-enhanced classroom environments. Moreover, the nature of the instrument (semi-structured interviews) provides the researchers with more latitude to dig deeper into the participants’ experiences (Ary et al., 2010). The same panel of three experts reviewed, modified and confirmed the interview questions (Appendix B).
As shown in Appendix B, the questions addressed all AL competencies, including teachers’ knowledge and skills as well as different areas of digital assessment, such as moral concerns and challenges. Therefore, the focus of questions varied; each investigating different aspects of AL or digital assessment. Questions 9 and 10 encompassed teachers’ AL about knowledge, skills, and principles (Giraldo, 2018; Pastore & Andrade, 2019; Xu & Brown, 2016). Questions 3, 11, and 13 mainly probed the teachers’ sense of confidence and comfort regarding digital assessment, larger context, and teacher learning, hence they were considered among the factors contributing to AL (e.g., Pastore & Andrade, 2019; Xu & Brown, 2016). Questions 1, 4, and 6 investigated the teachers’ uses of technology, attitudes toward technology, and experiences of digital assessment (e.g., Tang et al., 2022).
Questions 5 and 12 aimed to explore the teachers’ approaches regarding their assessment purposes, scoring, reporting, and fairness (e.g., DeLuca et al., 2016). While questions 9 and 10 focused on the teachers’ roles in digital assessment, concerns regarding the morality and ethics of digital assessment were also directly addressed by question 12 as they gain more significance in digital assessment (e.g., García-Peñalvo et al., 2021; Garg & Goel, 2022; Lee et al., 2022). Questions 2, 7, 8, and 14 explored the requirements of AL in digital assessment and its importance in teachers’ views which is partly addressed in digital assessment literature (e.g., O’Leary et al., 2018; Timmis et al., 2016). Specifically, the need to include technology in AL conceptualization (Coombe & Davidson, 2022) was investigated through questions 7 and 14.
The same concerns were more or less pursued in the open-ended questionnaire. However, the questionnaire items were more technical, such as question 1 in Appendix A asking more directly about the inclusion of technology in AL conceptualization, since the experts were quite familiar with the related concepts. However, the interviews included more questions by which the teachers could express their views and experiences freely and extensively.
Procedure
To collect data from the experts, a list of researchers in AL/digital assessment was created based on a scoping review study on TALiDE, carried out by the researchers of the current study (Estaji et al., forthcoming), and the familiarity of the authors with the experts in the fields. Several experts were contacted through email to inform them about the study and invite them to participate in the study by filling in an open-ended questionnaire. Twelve experts, most of whom were teacher educators, accepted the invitation and volunteered to participate in the study by answering the open-ended questionnaire. It is important to acknowledge that since the participants were recruited voluntarily, factors such as gender, age, and nationality were not controlled. The participants’ assessment culture can affect their conceptions of assessment (Xu & Brown, 2016). Therefore, such a variety might contaminate while parallelly enriching the findings of this study. The data were collected from August to September 2022.
Moreover, the interviews were carried out with a group of teachers who received the announcements shared among classroom teachers in Telegram groups (a social media application) whose members were teachers teaching EFL in Iran. The interviews were conducted online via Telegram in September 2022. They were audio-recorded and transcribed. Each interview lasted 45 minutes on average. Adhering to research ethics, a consent form was presented to all of the participants to inform them about the aims and ethical approval of the study.
Data Analysis
To analyze the data, a qualitative analytical approach was adopted. The transcripts were coded manually through a coding scheme (i.e., open coding, axial coding, and selective coding, Ary et al., 2010) based on the statements representing the experts’ and teacher interviewees’ accounts. To achieve a sound thematization of the data, the categorization was based on the constant comparison approach following the systematic approach (Ary et al., 2010). In this regard, the inferred codes were constantly compared with the theoretical and conceptual frameworks mentioned in the literature. Furthermore, four of the 27 transcripts were coded by a second coder familiar with the coding scheme and the purpose of this study to ensure the dependability of the results. The inter-coder reliability was found to be acceptable (Cohen’s kappa = .88). Disagreements were discussed until a consensus was reached. For example, some codes were overlapping between the confident technology user and implementer roles, but after reviewing and discussing the codes repeatedly, the coders decided to classify the codes with a major underlying focus of technology under the confident technology user role rather than the implementer role. The analysis resulted in the emergence of eight categories that were further classified under three major themes, corresponding to the three research questions of the study (Appendix C). In the following sections, these themes and categories are explained and discussed.
Findings
The analysis of the data led to the emergence of three main themes, namely the experience of COVID-19, teachers’ roles and competencies in TALiDE, and the inclusion of technology in AL. Table 3 summarizes the themes and subthemes with their frequencies of occurrence and percentages. The first theme answers the first research question, investigating the effect of the COVID-19 pandemic on TALiDE, while the second theme is related to the second research question (i.e., what TALiDE means to the participants). The last theme focuses on the third research question, discussing the need to include technology in AL conceptualization. The frequencies refer to the times each subtheme was mentioned by the participants. As each theme could be mentioned by the participants several times the frequencies are larger than the sum of experts and teachers.
Table 3. Summary of the Major Themes of the Participants’ Perspectives Concerning TALiDE
| Themes | Teacher Participants | Experts | Total | |||
| f | % | f | % | f | % | |
| Teachers’ Experience of COVID-19: Raised Awareness |
26 | 7.58% | 13 | 3.79% | 39 | 11.37% |
| New Approaches to Assessment | 9 | 2.62% | 5 | 1.46% | 14 | 4.08% |
| The Increased Value of TALiDE | 17 | 4.96% | 8 | 2.33% | 25 | 7.29% |
| Roles & Competencies of Assessment Literate Teachers |
165 | 48.10% | 53 | 15.45% | 218 | 63.56% |
| Knower | 27 | 7.87% | 16 | 4.66% | 43 | 12.54% |
| Implementer | 37 | 10.79% | 9 | 2.62% | 46 | 13.41% |
| Confident Tech User | 33 | 9.62% | 15 | 4.37% | 48 | 13.99% |
| Moral Agent | 33 | 9.62% | 3 | 0.87% | 36 | 10.50% |
| Learner | 35 | 10.20% | 10 | 2.92% | 45 | 13.12% |
| The Need for the Inclusion of Technology in AL | 52 | 15.16% | 34 | 9.91% | 86 | 25.07% |
| TALiDE in Today’s Technology-Enhanced Education | 26 | 7.58% | 15 | 4.37% | 41 | 11.95% |
| TALiDE beyond Online Classrooms | 26 | 7.58% | 19 | 5.54% | 45 | 13.12% |
| SUM | 243 | 70.85% | 100 | 29.15% | 343 | 100.00% |
| N = Documents | 15 | 12 | 27 | |||
Teachers’ Experience of COVID-19: Raised Awareness
This theme represents the importance of the pandemic in shaping and reconstructing TALiDE. As online teaching gained a seemingly unprecedented prevalence in education, the participants’ accounts of assessment in digital environments mostly revolved around their experiences during the pandemic. Therefore, such experiences appeared to be a major theme in the data. The participants addressed their perspectives in this regard in 13 and 26 experts’ and teacher participants’ (interviewees’) cases, respectively (11.37% of total data). This theme is discussed under the following subthemes: new approaches to assessment and the increased value of TALiDE.
New approaches to assessment. This subtheme, which shows the teachers’ raised awareness in attempting to become compatible with the constraints and affordances of digital environments during the COVID-19 pandemic by adopting new approaches to assessment, was reported in five experts’ and nine interviewees’ cases (4.08% of the data). The participants mentioned that at first online assessment was demanding and without any innovation prospectives. As one of the participants mentioned “most people were simply trying to shift from their current practices to online in a ‘vanilla’ fashion. Simply transfer it. They were not trying to do new things (e.g., AR, VR) in assessment” (EP4). TP1 also argued that “online assessment was at first very confusing”. Similarly, TP3 mentioned that “I only wanted to survive and get the job done” (TP3). Several terms and phrases, such as “confusing”, “vanilla fashion”, “to survive”, and “get the job done” show that teachers were not trying to use the full potential of technology, and they were aware of their habitual and uncreative assessment practices. However, there was a positive side to the experience of emergency assessment since it raised the teachers’ awareness of their assessment practices as they progressed in online assessment. In this regard, they turn to assessment practices less adopted in face-to-face classes.
For example, one of the teacher participants mentioned that “projects could reflect students’ achievements of the objectives of the course in practice” (TP9). She continued that “traditionally I did not use projects, but in online classes, I found projects as a way of making sure students have learned”. As indicated in her excerpts, TP9 started to use a new type of assessment (projects) as a form of performance-based assessment which was not used before. Another teacher participant (TP5) also referred to the “creative” reporting application provided by their language institute. TP5 mentioned that “we used a creative reporting application weekly. We used quizzes, observation, short essays, and storytelling on a range of classroom behaviors by rating them from 1 to 5”. According to TP5, teachers were required to report students’ performances every week. To do so, teachers could use a variety of assessment tools, such as “quizzes” and “observation”. To her, this way of assessment would lead to a longitudinal approach to assessing the students taking into account “a range of classroom behaviors” to evaluate, rather than relying on their mid-term and final summative scores. Therefore, although teachers were not fully creative in using advanced technology, such as “VR” (virtual reality), as earlier mentioned by EP4, the experience of the pandemic raised their awareness regarding the new assessment opportunities.
The increased value of TALiDE. This subtheme that was observed in eight experts’ and 17 interviewees’ cases (7.29 of the data) highlights the increased awareness of teachers during the pandemic as they reported enhanced values for TALiDE. EP10 mentioned that “COVID-19 acted as a critical incident that caused a move toward integrating and appreciating technology”. As EP10 highlighted, COVID-19 happened as a sudden incident that motivated teachers to appreciate the skills and knowledge provided by TALiDE. The participants also mentioned that during the pandemic teachers were “watching videos, and asking colleagues” (TP4) how to deal with digital assessment. For example, TP4 mentioned that “sadly I preferred to “take a dictation test in the same way by asking the students to type the words in the private chat. But I tried watching videos and asking colleagues to learn about technology”. He described himself as an old-fashioned person who used tests in the same traditional way as in face-to-face classes even knowing that it was not reliable, as he used the word “sadly”. However, having other teachers’ assistance, he was getting more willing to learn about technology tools and their potential for assessment. Therefore, the experience of the pandemic led to an incremented understanding of the importance of TALiDE in general. The excerpts show that teachers reconsidered their technological deficiencies in assessment and tried to compensate for them.
Roles and Competencies of Assessment Literate Teachers in Digital Environments
The participants attributed several competencies to classroom teachers as assessors in digital environments. As Table 1 represents, this theme constitutes the most frequently reported theme mentioned in 165 cases of the interviewees’ codes and 53 of the experts’, approximately encompassing 63.56% of the data. This theme incorporates five sub-themes, featuring the competencies attributed to teachers as they attempt to effectively and successfully integrate technology with assessment. These sub-themes are elaborated on in what follows.
Teacher as a knower. The participants addressed instances of knowledge related to TALiDE in 16 experts’ and 27 interviewees’ cases, roughly 12.54% of the data. The codes addressing this issue were categorized according to the TALiP Model (Xu & Brown, 2016) and the literature indicating the representation of teachers’ knowledge as a facet of AL (e.g., DeLuca et al., 2016; Giraldo, 2018). More specifically, this sub-theme represented the teachers’ pedagogical and assessment-specific knowledge. Highlighting the importance of pedagogical knowledge, one interviewee stated that “teachers are first required to be equipped with pedagogical knowledge, then, being equipped with technology” (TP2). This excerpt reflects that the teachers’ pedagogical competence is a requirement that needs to be met before other competencies.
Regarding assessment-specific knowledge, both experts and interviewees (teacher participants) referred to a range of knowledge related to classroom assessment, including the knowledge of formative assessment and feedback, score interpretation and communication, and assessment concepts such as reliability and validity. For instance, one expert referred to teachers’ knowledge “to design tasks, rubrics, and tests and interpret and report scores in digital spaces” (EP3), and others highlighted that “teachers need the knowledge to carry out good formative assessment practices” (EP8) and “it is important to know how to give different kinds of feedback” (P5). One participant teacher expressed the need to know assessment concepts by mentioning that “I need to know what reliability, validity, and washback are, then employ them in my practices” (TP7). This excerpt also indicates that an assessment-literate teacher has a role to attain assessment-specific knowledge as the participant referred to “I need to know” and then mentioned the three assessment concepts.
In line with the perspective of TP7, the data also revealed that most experts considered the knowledge of AL as a basis on which teachers rely to use technological tools effectively. For example, EP2 defined AL in digital environments as “the degree of applicability of the teacher’s repertoire of language assessment onto digital platforms” (EP2), which reflects the significance of knowledge in assessment as a fundamental requirement.
Teacher as an implementer. This sub-theme is represented in nine of the experts’ and 37 of the interviewees’ data (13.41% in total). It encompasses the roles and competencies assigned to teachers as implementers of classroom assessment, as in planning, designing, and implementing assessment tasks with technological tools while dealing with the constraints and affordances of their classroom context. One expert described this subtheme as one of the competencies teachers need to have: “Teachers need to know how to use digital assessment tools, strategies, and routines, to carry out good formative assessment practices, and to decide on the quality of classroom assessment” (EP8). Moreover, the interviewees saw their roles in designing, implementing, and evaluating assessment tools as the agents of classroom assessment. As TP8 mentioned, “I am responsible for designing and grading online quizzes”, and “I have to evaluate the applicability of different assessment tools and ensure the reliability, validity, and security of tests” (TP7). In these excerpts, using “I am responsible” and “I have to”, TP8 and TP7 referred to aspects of assessment task implementation and evaluation considering their roles as classroom assessors. However, their roles as quality evaluators and classroom assessors could be constrained by the language institutes, they were working at. For example, one teacher participant mentioned that “I have no control over the predetermined summative midterm and final exams which constituted 80% of students’ scores. They gave me a minimal role in assessment” (TP1). She indicated that such an institutional approach minimized her role in assessment. Therefore, teachers as the implementers of assessment in digital environments recognized their roles as planners, designers, enactors, and evaluators of classroom assessment for which they required assessment competence and autonomy to effectively perform them by technology.
Teacher as a confident technology user. By implementing assessment in digital environments (which was the focus of the previous sub-theme), teachers also rely on their technological competencies. However, this sub-theme pertains specifically to the teachers’ competencies and roles in digital environments in using technology to their best benefit, highlighting the significance of effective assessment over the mere use of technology.
This sub-theme, indicated by the teachers’ confidence and comfort in using technology in assessment, was addressed in 15 experts’ and 33 interviewees’ cases (13.99% of the data). The interviewees referred to different degrees of confidence and comfort in selecting and using technological tools for designing, implementing, grading, and reporting the results of assessment practices and solving technology-related problems during classroom assessment. While some interviewees reported being comfortable and confident in integrating technology with assessment and solving technical breakdowns, others expressed less confident attitudes, especially while implementing assessment tasks. For example, TP1 stated that “still when I want to take a quiz, I don’t know which way is better”.
As reflected in the subtheme of teachers’ experiences of the COVID-19 pandemic, all the interviewees unanimously mentioned that they were not confident when they started using technology in assessment. Interestingly, the expertise gained through learning and experience was reported as the main source of their confidence and comfort. The confident teachers mentioned such ideas, as “trial and error” (TP10), “watching videos on the Internet” (TP15), and “asking colleagues” (TP4), which helped them gain expertise in technology and become confident. For example, TP2 explained that her sense of confidence helped her to use technology in assessment: “As I feel confident, I find suitable resources, use multimedia in creating the tests, and use interactive assessment”. One of the experts also referred to age as another factor in teachers’ confidence stating that: “there may be an age factor for teachers who do not have the technology skills or cognitive flexibility to adapt to new technologies” (EP4).
Other experts highlighted that TALiDE “involves teachers’ knowledge and skills related to assessment in a digital learning environment as well as knowledge of emerging technology that will facilitate the practice of digital assessment” (EP9). Similarly, TP9 claimed that “it is very important that teachers have technology literacy to act efficiently in assessment”. Therefore, having competency in technology, adaptability, and cognitive flexibility were essential in TALiDE.
Teacher as a moral agent. This sub-theme indicates the teachers’ concerns about unethical practices more likely to occur in assessment in digital environments, such as cheating, plagiarism, and identity misrepresentation. It surfaced in three and 33 cases in experts’ and teachers’ data, respectively (roughly 10.50% of total data). One of the experts considered dealing with cheating as one of the competencies teachers were getting equipped with during the COVID-19 pandemic: “Treating cheating in the online environment was among the capabilities teachers have started to achieve” (P7). Another expert, similarly, stated that “due to the lack of digital assessment literacy, teachers feared plagiarism and other academic dishonesty as students wrote assessments at home without invigilation” (EP8).
The interviewees particularly underscored the differences in morality issues between face-to-face and technology-enhanced classes. Transferring from face-to-face to online classes, teachers realized that with the availability of the Internet and lack of physical observation cheating was more likely to happen, which would be a barrier to fair assessment. One interviewee argued that the physical absence of the teacher when assessing students necessitated special control: “It is essential to ensure that the students are relying on their achievements and avoid finding answers on the Internet” (TP10). The interviewees emphasized the teachers’ roles in assuring fairness and equity of assessment in digital environments, and one said: “Cheating is a concern. Using the settings, while designing the exams, to show the questions randomly for each student…is one way I prevent it. Asking them to turn on their webcams is another way.” (TP2).
The competency that the teacher participants attributed to assessment literate teachers in digital environments encompassed protecting the safety of assessment results, ensuring the students’ identities (e.g., asking them to turn on their webcams), and designing reliable assessment tasks (e.g., randomizing the distribution of test items). Although the interviewees mentioned that moral issues are more related to teachers than students, especially when they select and use assessment activities, they further mentioned that students should have equal access to the Internet, as one of the teachers mentioned that “connection breakdowns can be a source of inequality” (TP8), which causes challenges in having access to the test items. Therefore, teachers have a special role to implement fair assessment practices. They also require competency regarding fairness principles and skills to maintain them.
Teacher as a learner. The teacher participants and experts also expressed dedication to continued professional development as one of the roles of assessment-literate teachers in digital environments. This sub-theme was addressed in 10 experts’ and 35 interviewees’ cases (13.12% of total data). It incorporated the role of teachers as perpetual learners who need to learn how to integrate technology into classroom assessment through participating in training courses, workshops, and conferences as well as asking expert colleagues and experientially utilizing technological tools. One of the participants held that:
The digital assessment adds some responsibilities to teachers. For example, they have to learn how to use new technologies. They are good assessors when they keep up with technology. So, it means that they need to be both digitally literate and at the same time good assessors, of course. (TP3)
In this excerpt, TP3 considered learning as a responsibility of teachers by a reference to “adds more responsibilities” and “have to learn”. Similarly, TP4 indicated his realization of the importance of constant learning by mentioning that “when I want to make an exam, I visit different sites, examine different samples, and consult my colleagues”. He later added, “Besides being a teacher, I am a learner who should continuously learn how to use technology” (TP4). In line with the interviewees, the experts emphasized the necessity of learning by stating that “a teacher must be aware of the possible assessment applications to keep up with the new requirements of their job” (EP1), and “teachers need to receive training on how to use digital platforms, how to be interactive through digital communication, and how to give different kinds of feedback” (P5).
In these extracts, EP1 referred to the reason behind the importance of this role for assessment-literate teachers as the requirement of their job. EP5 also stressed the fields in which teachers need the training to be assessment literate in digital environments. Moreover, almost all the interviewees suggested that pre-service and in-service teacher training programs might have not remedied this oversight of technology in their assessment experiences. TP5 complained that “while learning is a must, we did not receive any courses on this subject in BA and MA education”. In this extract, TP5 saw learning-acquiring TALiDE- as a vital component to be considered in teacher education courses by mentioning that it is a “must”.
The Need for the Inclusion of Technology in AL Conceptualization
This theme answers the third research question of this study, investigating whether technology should be included in the conceptualization of AL. To gain the experts’ and teachers’ perspectives, the answers to four questions mainly fed the data (Questions 1 and 6 in Appendix A and Questions 7 and 14 in Appendix B). This theme, addressed in 25.07% of the data (34 experts’ cases and 52 interviewees’ cases) incorporates 2 subthemes, namely TALiDE in today’s technology-enhanced education and TALiDE beyond online classrooms, which are elaborated below.
TALiDE in today’s technology-enhanced education. This subtheme, found in 15 and 26 experts’ and teachers’ cases respectively (11.95% of the data), answered the question directly through the experts’ data and indirectly in those of the interviewees when they highlighted the importance of TALiDE in digital environments. The experts were directly asked whether technology needs to be added to the reconceptualization of AL. The experts expressed varied views, ranging from not necessary (EP4), to yes and no (EP2), to a strong yes (the remaining 10 experts). EP4 stated that “at most DAL will simply be a small facet of AL. The role technologies play in assessment is still not strong”. He further added:
DAL MAY [emphasis by the participant] become an important part of school-based assessment and perhaps even qualifications assessment. But I’m not convinced. It may depend on whether teachers are expected to create new materials for assessment because of the format change or if they are required by policy to prevent cheating. (P4)
EP4 indicated his doubt regarding the need for incorporating technology into the concept of AL due to the current weak role of technology in assessment and the fact that AL is grounded in the standards, set by the policy, in which any requirement for technology is missing. Therefore, he considered DAL as a “small” component of AL. EP2 attributed her moderate agreement to the view that opposes, seeing AL as a construct. She stated:
To answer this question, one needs to define assessment as a construct first. The way I would define it does not necessarily include the involvement of technology. When it comes to exemplifying innovative and more recent (more specifically post-COVID) assessment practices, however, the inclusion of technology seems inevitable. (P2)
To EP2, although as a construct AL does not have to include technology, post-COVID assessment practices would involve technology integration which might require technology integration to the concept as well. Other experts, however, were positive regarding this issue, and one stated that “AL is a broad concept covering various assessment skills and competencies that a teacher requires. Hence, being tech-savvy concerning assessment is also a competency. The issue is more highlighted in today’s education, which is affected by the COVID-19 pandemic” (EP3). In accordance with EP3, most experts believed that the post-pandemic situation may require an element of technology in AL conceptualization as technology plays a bolder role in education.
The interviewees also mentioned that without TALiDE they could not assess students effectively in the digital environment. PT10 held that “it is necessary to have skills and knowledge. Only a skilled teacher can do the assessment with technology”. TP10 highlighted that for digital assessment having “skills and knowledge” is “necessary” to enable a teacher to enact assessment practices.
TALiDE beyond online classrooms. In addition to the participants’ views concerning technology as a component of AL, the participants addressed the need to have competence in using technology in assessment even beyond online classrooms, as found in 19 experts’ and 26 interviewees’ cases (nearly 13.12% of the data). Pointing out the importance of practicality as a principle of language assessment, one expert stated:
I believe it refers not only to making the assessment procedures easy for stakeholders involved in such practices but also to the prevailing tendencies the assessment conditions might bring about. The 21st century has been calling out for digital platforms and we cannot go deaf to this call. (EP12)
In her excerpt, EP12 referred to practicality which calls for adapting to a new assessment context prevalent in the 21st century. Another expert also highlighted the general change in education due to technological advancement and COVID-19 resulting in “a growing preference for online education” (EP5) and claimed: “Since there won’t be other choices in near future…Nowadays much of learning is supposed to happen outside the classroom.”
Similarly, one of the teacher participants believed that technology was both essential and beneficial in her classroom assessment. She mentioned that “if a teacher doesn’t have enough knowledge of digital tools…then that person cannot really perform in the 21st century. It is also the same for assessment” (TP3). She later added “technology is kind of useful…because it helps in reducing the workload. For example, we do not need to correct multiple-choice questions every time. We just put them in a form and they will be auto-corrected” (TP3). To her, technology generally can be helpful in assessment as it reduced the workload.
Some other participants also believed that TALiDE may not be limited to online classrooms, but skills and competencies could be transferred to other contexts, such as face-to-face classes. In this regard, EP7 mentioned that “technology can act as a powerful supplement”. Experts also mentioned that it is “a competency useful in different aspects of a teacher’s career. It is not limited to the pandemic era. Teachers may wish to use online tests and interpret the results in some courses. Therefore, having DAL is beneficial in face-to-face education, too” (EP3), and “the collateral skills involved in designing effective digital assessments will result in skills that can easily be transferred to technology-enriched classrooms” (EP6). It can be seen in the excerpts of the three experts that TALiDE can be a supplement to teachers’ general AL and useful in non-online courses as a complement to teachers’ skills.
Discussion
This study aimed to identify the competencies and roles of TALiDE in light of the experience of the COVID-19 pandemic. For the first research question of the study, exploring the effect of COVID-19 on TALiDE, the findings showed that the experience of the COVID-19 pandemic largely affected the teachers’ awareness of TALiDE, in that the teachers gained a deeper understanding of their assessment practices and significance of TALiDE. In line with such findings, Tian et al. (2021) identified COVID-19 as an opportunity that allowed teachers to reflect on their assessment practices. This study, however, does not replicate the results of Tian et al.’s (2021), since teachers in this study reported various ways through which they gained increased awareness of TALiDE. This is while in Tian et al.’s (2021) study teachers’ reflection was the key means through which teachers moved from being unconfident to being concerned with language assessment. The findings of the present study showed that, apart from reflection, teachers rely on collegial assistance, self-experimentation, and technological means, such as the Internet. Our findings also revealed that a major aspect of raised awareness happened in the teachers’ understanding of doing classroom assessments by using technology and the value of learning how to effectively assess in digital environments, which requires sustained training (García-Peñalvo et al., 2021; Ghanbari & Nowroozi, 2021; Lee et al., 2022). The greater opportunity to use new assessment approaches or the ones which were not used in traditional assessment was highlighted by García-Peñalvo et al. (2021) as they indicated the need for competencies in going beyond merely using technology in assessment. This raised awareness can be seen in the data as teachers reported the recognition of the availability of assessment practices that emergency assessment underscored.
Furthermore, in response to the second research question, the present study found that a set of roles and competencies are attributed to assessment literate teachers in digital environments, namely, teacher as a knower, implementer, confident technology user, moral agent, and learner. These areas that are manifested and enacted in teachers’ classroom assessment conduct in a digital environment constitute their TALiDE, which is consistent with the extant literature (DeLuca et al., 2016; Giraldo, 2018; Pastore & Andrade, 2019; Xu & Brown, 2016). However, due to the intricacies of technology and the characteristics of online classrooms, they take on a relatively new nature. For example, teachers’ competencies and roles as an implementer and a learner match with TALiP’s components of teacher AL in practice and teacher learning in which the importance of teachers’ decision-making and constructing AL as an interactive process are highlighted, respectively. Nevertheless, TALiDE’s roles as an implementer and a learner underscore the effect of context, here online environment, in implementing assessment tasks and the potential use of technology in improving the assessment knowledge base.
Although TALiP reflects such components, the teacher as a learner subtheme in this study centered on the dynamics of TALiDE which builds on the increased role of feedback, more interactive opportunities, and constantly changing technological complexity given by digital platforms. The competencies of teachers as implementers in classroom assessment were detailed in DeLuca et al. (2016). However, the findings of this study revealed that in digital environments more specifically, being the conductor of classroom assessment might be more burdensome. This is because implementing assessment in digital environments needs familiarity with technological tools, and assessment processes are affected in that the security and reliability of tests might be endangered, and as a result tensions and control by institutional policies might be engendered (Holden et al., 2021).
As another instance, in Giraldo’s (2018) and Xu and Brown’s (2016) models, a component for the knowledge base in assessment is identified that was reflected in our results as well. In line with the TALiP model, our findings showed that knowledge is the necessary but insufficient component of AL as the participants of this study highlighted that teachers rely on their knowledge to perform classroom assessment in digital environments. Although many components of knowledge are represented in the mentioned models and the extant literature, such as pedagogical knowledge and feedback (e.g., Giraldo, 2018; Inbar-Lourie, 2013; Stiggins, 1991), ethics appeared to be a major subtheme going beyond the knowledge subcomponent of TALiDE. Teacher as a moral agent subtheme reflects moral concerns in digital assessment which has attracted large attention in the literature (Garg & Goel, 2022). Unlike face-to-face assessment, the physical absence of the teacher in digital environments creates some concerns for the principles of assessment fairness and ethics. Therefore, such an extension of roles and competencies in this regard is not unexpected and prevails over teachers’ knowledge of assessment principles in dealing with the influence of technology.
More specifically, this study added two more roles and competencies which become more prominent in TALiDE. These two factors, namely teacher as a moral agent and confident technology user, underscore the complexities introduced by the technology used in classroom assessment. Regarding the moral agent role, Giraldo (2018) calls attention to the principles component of AL and argues that ethical principles of AL are less attended to in the existing AL frameworks for language teachers. This study supports Giraldo’s emphasis on the importance of developing teachers’ competence to preserve fairness and equity in classroom assessment. The later subtheme, the teacher as a confident technology user, requiring competencies necessarily essential for assessment in digital environments, might not be identified in previous models of AL. Giraldo (2018) referred to technology as a skill subcomponent of teachers’ AL minimally, describing it as the ability to “use software such as Statistical Packages for the Social Sciences, run operations on Excel, and use internet resources such as online tutorials and adapt contents for particular language assessment needs” (p. 190). Rather, this study expounded on the roles and competencies of teachers as technology users in various aspects of assessment, such as designing, grading, and implementing the tasks.
The importance of technology in AL was further investigated in the third research question of the study. The findings, corresponding to the question raised by Coombe and Davidson (2022) regarding the probable importance of technology in the reconceptualization of AL, indicated that the participants mostly consider technology as a component to be included in the AL concept, as post-COVID conditions would require technology integration and the principle of practicality necessitates preparedness for new assessment situations. Moreover, they somehow agreed that TALiDE is helpful for face-to-face classes and necessary for digital environments. Therefore, this study argues for reconsidering the possible role of technology to be incorporated into the AL concept. As one of the experts pointed out, currently AL is not being called on to incorporate technology in assessment. Although this is the case in standards and requirements for AL specifically (see reviews by DeLuca et al., 2016; Gotch & French, 2012), several organizations and teacher development policies reinforce the necessity of developing teachers’ digital literacies (Knobel & Kalman, 2016).
Further, closely related to technological pedagogical content knowledge (TPACK), AL relies on all components of TPACK, as pedagogical and content knowledge, except for technological knowledge. As teachers need training and literacy regarding technological knowledge in teaching, so they do in assessment because assessment and teaching are integrated. Additionally, it can be argued that context might affect the divergence of ideas between experts and teachers regarding the inclusion of technology in AL. The experts of this study were from different countries, while all the teachers were working in the context of Iran where EFL teaching is expected to keep partly online. Therefore, the need for technology in assessment was more tangible to teachers than the experts. In line with Cooper et al. (2022), our findings indicated that the participants did not perceive the post-COVID era as similar to the pre-COVID period in which traditional assessment was prevalent, rather the changed situation is likely to affect the assessment practices forever. Accompanied by the advancement of technology in teaching, TALiDE would become an area of competence for which teachers need instruction.
Conclusion and Implications
The present study offered some insights into teachers’ AL in digital environments and identified three major themes, namely the experience of COVID-19: raised awareness, roles and competencies of assessment literate teachers in digital environments, and the need for the inclusion of technology in AL conceptualization. It can be concluded that the COVID-19 pandemic led to an increased understanding of TALiDE. Additionally, the teachers perceived their competencies and roles as classroom assessors who integrate technology in their classroom assessment and play leading parts in planning, designing, implementing, evaluating, and the like.
The overarching role of technological knowledge in teachers’ assessment practices indicates that the obtained findings can have implications for course designers and teacher educators in pre-and in-service teacher education programs. The multifaceted nature of TALiDE demands that teacher educators and educational policymakers take into account the various aspects of digital assessment alongside teachers’ assessment knowledge, implementation, agentive roles, learning, and the like to develop teachers’ TALiDE.
The findings of this study can also encourage teachers to develop the technological aspects of their AL more effectively. Given the rising need to integrate technology into education, especially language education, this study can be adopted as a basis for further research and practice that can inform technology-enhanced educational program developers and course designers. Moreover, it seems that technology has a missing space in the current standards and requirements of AL. While the findings mostly advocated the inclusion of technology in the conceptualization of AL, there were some disagreements. Reaching consensus requires the involvement of a larger number of researchers, educational policymakers, and standard developers to understand and justify the integration of technology into AL. Nevertheless, the prevalence of technology in education might have an impact on the future assessment practices of teachers, for which TALiDE would be helpful. This state gains more significance when education keeps going online.
Considering the qualitative nature of this study and its limited sample size, the role of several factors such as age, experience, academic degree, and educational context remains unexplored. The potential roles of these variables can be investigated by future researchers interested in understanding the contributing factors to AL in specific contexts. Furthermore, although the existing literature has explored some of these concepts, the complexities of TALiDE, technology integration with assessment, and AL conceptualization with current technology advancements need further investigation. The integration of AL and digital assessment, as the focus of the current study, was an initial step to explore the role of technology in assessment and its consequent roles and competencies regarding the different aspects of TALiDE. Future research may take up this line of research by examining TALiDE in other EFL contexts and involving a wider group of experts, focusing on their perspectives of TALiDE conceptualization.
About the Authors
Masoomeh Estaji is an associate professor of Applied Linguistics at Allameh Tabataba’i University (ATU), Tehran, Iran. She holds a Ph.D. in Applied Linguistics from Allameh Tabataba’i University. She earned the Top Researcher Award at ATU in 2018, 2020, 2022. She has presented and published numerous papers on methodology, testing, and second language acquisition (SLA) in various national and international journals such as International Journal of Testing, Educational Assessment, Language Learning in Higher Education& Reading Psychology. Her research interests include language testing and assessment, ESP, and teacher education. http://orcid.org/0000-0002-8014-9491
Zahra Banitalebi is a Ph.D. candidate at Allameh Tabataba’i University, Tehran, Iran. She is a lecturer at Allameh Tabataba’i University, Tehran, Iran. Her research interests lie mainly in the areas of teacher education, assessment, and research methods. She has published papers in AisaTEFL, International Journal of Testing, International Journal of Language Testing, TEFLIN, Measurement: Interdisciplinary Research and Perspectives, and Language Related Research. http://orcid.org/0000-0001-7137-9669
To Cite this Article
Estaji, M., & Banitalebi, Z. (2023). The effects of EFL learners’ attitude on participation and learning during collaborative writing. Teaching English as a Second Language Electronic Journal (TESL-EJ), 27 (2). https://doi.org/10.55593/ej.27106a10
References
Alavi, S. M., Dashtestani, R., & Mellati, M. (2022). Crisis and changes in learning behaviours: Technology-enhanced assessment in language learning contexts. Journal of Further and Higher Education, 46(4), 461-474. https://doi.org/10.1080/0309877X.2021.1985977
Ary, D., Jacob, L. C., Sorensen, C., & Razavieh, A. (2010). Introduction to research in education. Thomson & Wadsworth.
Broadfoot, P., & Black, P. (2004). Redefining assessment? The first ten years of assessment in education. Assessment in Education: Principles, Policy & Practice, 11(1), 7-26. http://doi.org/10.1080/0969594042000208976
Brookhart, S. M. (2011). Educational assessment knowledge and skills for teachers. Educational Measurement: issues and practice, 30(1), 3-12. https://doi.org/10.1111/j.1745-3992.2010.00195.x
Brown, G. T. L. (2017). The future of assessment as a human and social endeavor: Addressing the inconvenient truth of error. Frontiers in Education, 2(3), 1-4. https://doi.org/10.3389/feduc.2017.00003
Coombe, C., & Davidson, P. (2022). Language assessment literacy. In H. Mohebbi & C. Coombe (Eds.), Research questions in language education and applied linguistics: A reference guide (pp. 343-347). Springer.
Coombs, A., & DeLuca, C. (2022). Mapping the constellation of assessment discourses: a scoping review study on assessment competence, literacy, capability, and identity. Educational Assessment, Evaluation and Accountability, 1-23. https://doi.org/10.1007/s11092-022-09389-9
Cooper, A., DeLuca, C., Holden, M., & MacGregor, S. (2022). Emergency assessment: Rethinking classroom practices and priorities amid remote teaching. Assessment in Education: Principles, Policy & Practice. https://doi.org/10.1080/0969594X.2022.2069084
Creswell, J. W., & Creswell, J. D. (2023). Research design: Qualitative, quantitative, and mixed methods approaches. Sage publications.
DeLuca, C., LaPointe-McEwan, D., & Luhanga, U. (2016). Approaches to classroom assessment inventory: A new instrument to support teacher assessment literacy. Educational Assessment, 21(4), 248-266. https://doi.org/10.1080/10627197.2016.1236677
DeLuca, C., Schneider, C., Coombs, A., Pozas, M., & Rasooli, A. (2020). A cross-cultural comparison of German and Canadian student teachers’ assessment competence. Assessment in Education: Principles, Policy & Practice, 27(1), 26-45. https://doi.org/10.1080/0969594X.2019.1703171
Ericsson, K. A., & Smith, J. (1991). Prospects and limits of the empirical study of expertise: An introduction. In K. A. Ericsson & J. Smith (Eds.), Toward a general theory of expertise: Prospects and limits (pp. 1-38). Cambridge University Press.
Estaji, M., Brown, G. T. L., & Banitalebi, Z. (Forthcoming). The key competencies and components of teacher assessment literacy in digital environments: A scoping review.
Eyal, L. (2012). Digital assessment literacy: The core role of the teacher in a digital environment. Journal of Educational Technology & Society, 15(2), 37-49.
Farhady, H. (2019). A cross-contextual perspective on EFL teachers’ assessment knowledge. Edu 7: Yeditepe Üniversitesi Eğitim Fakültesi Dergisi, 8(10), 1-19. https://dergipark.org.tr/en/pub/edu7/issue/51553/665542
Farhady, H. (2022). Language testing and assessment in COVID-19 pandemic crisis. In K. Sadeghi (Ed.), Technology-Assisted language assessment in diverse contexts (pp. 54-68). Routledge.
García-Peñalvo, F. J., Corell, A., Abella-García, V., & Grande-de-Prado, M. (2021). Recommendations for mandatory online assessment in higher education during the COVID-19 pandemic. In D. Burgos, A. Tlili, A. Tabacco (Eds.), Radical solutions for education in a Crisis context. COVID-19 as an opportunity for global learning (85-98). Springer Nature. https://doi.org/10.1007/978-981-15-7869-4_6
Garg, M., & Goel, A. (2022). A systematic literature review on online assessment security: Current challenges and integrity strategies. Computers & Security, 113, 102544. https://doi.org/10.1016/j.cose.2021.102544
Ghanbari, N., & Nowroozi, S. (2021). The practice of online assessment in an EFL context amidst COVID-19 pandemic: Views from teachers. Language Testing in Asia, 11, 27. https://doi.org/10.1186/s40468-021-00143-4
Giraldo, F. (2018). Language assessment literacy: Implications for language teachers. Profile Issues in Teachers Professional Development, 20(1), 179-195. https://doi.org/10.15446/profile.v20n1.62089
Gotch, C. M., & French, B. F. (2014). A systematic review of assessment literacy measures. Educational Measurement: Issues and Practice, 33(2), 14-18. https://doi.org/10.1111/emip.12030
Guerrero-Roldán, A.-E., & Noguera, I. (2018). A model for aligning assessment with competences and learning activities in online courses. The Internet and Higher Education, 38, 36-46. https://doi.org/10.1016/j.iheduc.2018.04.005
Holden, O. L., Norris, M. E., & Kuhlmeier, V. A. (2021). Academic integrity in online assessment: A research review. Frontiers in Education, 6, 639814. https://doi.org/10.3389/feduc.2021.639814
Ibna Seraj, P. M., Chakraborty, R., Mehdi, T., & Roshid, M. M. (2022). A systematic review on pedagogical trends and assessment practices during the COVID-19 pandemic: Teachers’ and students’ perspectives. Education Research International, 1534018. https://doi.org/10.1155/2022/1534018
Inbar-Lourie, O. (2013). Guest editorial to the special issue on language assessment literacy. Language Testing, 30(3), 301-307. https://doi.org/10.1177/0265532213480126
Kazemian, M., Khodareza, M. R., Khonamri, F., & Rahimy, R. (2022). Problematizing language testing and assessment syllabi through intercultural competence assessment perspective in an EFL context. Journal of Asia TEFL, 19(1), 309-316. https://doi.org/10.18823/asiatefl.2022.19.1.23.309
Knobel, M., & Kalman, J. (2016). Teacher learning, digital technologies and new literacies. In M. Knobel, & J. Kalman (Eds.), New literacies and teacher learning professional development and the digital turn (pp. 1-20). NY: Peter Lang.
Lee, V. W. Y., Lam, P. L. C., Lo, J. T. S., Lee, J. L. F., Li, J. T. S., & Pang, N. S.-K. (2022). Rethinking online assessment from university students’ perspective in COVID-19 pandemic. Cogent Education, 9(1), 2082079. https://doi.org/10.1080/2331186x.2022.2082079
Mishra, P., & Koehler, M. J. (2006). Technological pedagogical content knowledge: A framework for teacher knowledge. Teachers College Record, 108(6), 1017-1054. https://doi.org/10.1111/j.1467-9620.2006.00684.x
Mphahlele, R. S. (2022). Digital assessment literacy in online courses (formative/summative): Rethinking assessment strategies in the open distance and e-learning institutions. In G. Durak and S. Çankaya (Eds.), Handbook of Research on Managing and Designing Online Courses in Synchronous and Asynchronous Environments, (pp. 404-417). IGI Global.
O’Leary, M., Scully, D., Karakolidis, A., & Pitsia, V. (2018). The state-of-the-art in digital technology-based assessment. European Journal of Education, 53(2), 160-175. https://doi.org/10.1111/ejed.12271
Pastore, S., & Andrade, H. L. (2019). Teacher assessment literacy: A three-dimensional model. Teaching and Teacher Education, 84, 128-138. https://doi.org/10.1016/j.tate.2019.05.003 0742-051X
Pu, S., & Xu, H. (2021). Examining changing assessment practices in online teaching: A multiple-case study of EFL school teachers in China. The Asia-Pacific Education Researcher, 30(6), 553-561. https://doi.org/10.1007/s40299-021-00605-6
Stiggins, R. J. (1991). Assessment literacy. The Phi Delta Kappan, 72(7), 534-539.
Tang, T. T., Nguyen, T. N., & Tran, H. T. T. (2022). Vietnamese teachers’ acceptance to use e-assessment tools in teaching: An empirical study using PLS-SEM. Contemporary Educational Technology, 14(3), ep375. https://doi.org/10.30935/cedtech/12106
Tian, W., Louw, S., & Khan, M. K. (2021). Covid-19 as a critical incident: Reflection on language assessment literacy and the need for radical changes. System, 103, 102682.
Timmis, S., Broadfoot, P., Sutherland, R., & Oldfield, A. (2016). Rethinking assessment in a digital age: Opportunities, challenges and risks. British Educational Research Journal, 42(3), 454-476. https://doi.org/10.1002/berj.3215
Williams, J. C. (2015). “Assessing Without Levels”: Preliminary research on assessment literacy in one primary school. Educational Studies, 41(3), 341–346. https://doi.org/10.1080/03055698.2015.1007926
Willis, J., Adie, L., & Klenowski, V. (2013). Conceptualising teachers’ assessment literacies in an era of curriculum and assessment reform. The Australian Educational Researcher, 40(2), 241–256. https://doi.org/10.1007/s13384-013-0089-9
Xu, Y., & Brown, G. T. L. (2016). Teacher assessment literacy in practice: A reconceptualization. Teaching and Teacher Education, 58(2), 149-162. https://doi.org/10.1016/j.tate.2016.05.010
Zhang, C., Yan, X., & Wang, J. (2021). EFL teachers’ online assessment practices during the COVID-19 pandemic: Changes and mediating factors. The Asia-Pacific Education Researcher, 30(6), 499-507. https://doi.org/10.1007/s40299-021-00589-3
Appendices
Appendix A
Appendix B
Appendix C
A Summary of the Hierarchical Code-Subcode Model
[back to article]
| Copyright of articles rests with the authors. Please cite TESL-EJ appropriately. Editor’s Note: The HTML version contains no page numbers. Please use the PDF version of this article for citations. |