Zia Tajeddin<\/strong> Zari Saeedi<\/strong> Hamideh Mozaffari<\/strong> Various features of teacher instruction underpin the criteria used for the evaluation of teacher quality. The current study sought to explore whether nativeness\/non-nativeness affects the criteria teachers consider for teacher quality evaluation. To this end, the participants were provided with five video clips of teaching, each presenting a 10-min lesson taught in a real classroom environment. They were requested to rate the quality of the teachers and to point out and describe the criteria they used to rate the teachers. Content analysis of the data indicated that preparation, caring, classroom management, and instruction constituted the general criteria the native and non-native teachers employed to evaluate teacher quality. Considerable differences, however, were observed between the two groups regarding a few of the criteria. The native teachers valued teachers\u2019 efficient use of learners\u2019 L1 more than the non-native teachers, while teachers\u2019 linguistic accuracy and fluency of speech were highlighted by more non-native teachers. Besides, issues related to caring, management, and instruction grabbed the attention of both native and non-native teachers, while preparation received substantially less attention. It can be concluded that the use of video-mediated peer observation can provide a platform to uncover the implicit beliefs teachers hold toward teacher quality.<\/p>\n Keywords<\/em><\/strong>: teacher quality evaluation, evaluation criteria, native English-speaking teachers, non-native English-speaking teachers<\/p>\n Teacher quality has been long regarded as one of the most significant contributing factors to student learning (Ingersoll, 2020). It is, then, incumbent on those in the educational sphere to be able to evaluate teacher quality most reliably and validly. There has been mounting debate in the literature about whether the quality of a teacher should be measured based on (1) teacher input such as credentials, (2) teaching process, i.e., teacher practices often measured through classroom observations, (3) teaching output, i.e., the impact of teaching on student achievement, or a combination of these elements (Stronge et al., 2011). Although teacher evaluation systems in several educational contexts still rely upon teacher input and output, it is teaching performance that is now regarded as the most significant indicator of teacher quality (Gergen & Gill, 2020; Olsen, 2021). This, in turn, has resulted in a serious question: What teacher attributes and practices can best define quality in teachers\u2019 performance and hence should be considered in teacher evaluation? Some attempts have been made by recent scholars to introduce such features (Danielson, 2013; Marzano, 2013; Stronge, 2018). However, it is also necessary to know how teachers, given their recent inclusion and recognition as plausible evaluators in various contexts, evaluate teacher quality.<\/p>\n Against this backdrop, it seems crucial to explore what criteria teachers employ to evaluate teacher quality. Prior research has addressed the impact that variables such as teaching experience and discipline have on the criteria that teachers apply for evaluating teachers (e.g., Chand Dayal & Alpana, 2020; McCoy & Lynam, 2021; Nguyen & Pham, 2020; Torres et al., 2017). Torres et al. (2017), for instance, investigated the criteria teachers from different educational disciplines employ for evaluating their peers. The results indicated no major differences between the teachers from various disciplinary fields. The teachers from all fields consistently emphasized communication and interaction with students and students\u2019 engagement in the class. Scant research in L2 education, however, has examined how teacher nativeness\/non-nativeness might influence teacher evaluation.<\/p>\n The debate over native versus non-native English-speaking teachers has long dominated the English language teaching profession. Non-native speaker teachers are often defined against native speaker teachers, implying a privileged status bestowed on teachers for whom English is the native language. This privilege rests on the argument that a native speaker teacher can best model the fluent and appropriate use of language, is well aware of the cultural connotations of language, and can best judge the acceptability of any piece of language. Phillipson (1992, p.195), however, called this \u201cthe native speaker fallacy\u201d and highlighted that none of these abilities are beyond the capability of a non-native speaker of a language and can be well developed through teacher education. Such criticisms along with a sudden increase in the number of non-native speakers of English worldwide and the need for effective intercultural interaction led ELT scholars, almost from the early 1980s, to question the adequacy of the century-old native speaker model. This is aligned with the rise of English as an International Language, English as a Lingua Franca, and World Englishes, which emphasize the importance of non-native varieties of English, the significance of intelligibility in recently prevalent NNS-NNS communication, and the non-exclusive ownership of English. These new strands challenged the raciolinguistic ideologies pervading the ELT industry, that is the idea that competent teachers are white native speakers of English (Ramjattan, 2019; Rosa & Flores, 2017). Despite this acknowledgment in both professional and academic literature, non-native speaker teachers of English continue to be disprivileged in several educational contexts (Kubota, 2022; Rosa, 2016; Sekaja et al., 2022). Given the significance of this issue and the fact that teacher evaluation criteria, as primary indicators of teacher quality, have not been examined from native and non-native teachers\u2019 perspectives, the study aimed to explore the criteria native and non-native English language teachers apply for teacher quality evaluation.<\/p>\n Teacher quality evaluation refers to a process employed to measure teacher effectiveness which primarily attempts at enhancing student learning (Danielson, 2001). The method of conducting this seemingly easy process, however, has undergone major shifts over time. Measures of student learning, particularly student achievement scores, had been traditionally relied on by various stakeholders to decide upon the quality of teachers in several educational contexts. However, this method, which is commonly referred to as value-added modeling (VAM), has been the target of several criticisms as to its reliability and validity (Leon & Thomas, 2015). Theoretical and empirical evidence suggests that VAM scores suffer from inconsistency, are influenced by the characteristics of students, and do not show any significant relationship with other more accepted methods of teacher evaluation (Amrein-Beardsley & Geiger, 2020; Briggs & Domingue, 2011; Collins, 2014).<\/p>\n In view of these criticisms, researchers began to reconsider the century-old question of how teachers\u2019 quality can be reliably and validly evaluated. The most plausible response, according to the literature (Gergen & Gill, 2020), lies primarily in the very performance of teachers. Instead of employing student achievement to judge teacher quality, evaluation systems should rely on teachers’ practices so as to foster student learning. The method of using teachers\u2019 performance as the criterion for examining the quality of their work has been called the performance-based\/standards-based evaluation model. Although there exists no one best method to evaluate teachers, literature offers positive evidence as to the validity and reliability of employing teacher performance for evaluating teacher quality (e.g., Coady et al., 2020; Patrick et al., 2020).<\/p>\n Any teacher evaluation system, irrespective of the educational level, attempts to achieve two main goals: to exhibit accountability to various stakeholders and to foster teacher professional development (Lillejord & Bort, 2020). Not all evaluation programs, however, manage to achieve these purposes. An effective teacher evaluation requires careful planning of the whole process, namely the frequency, nature, and use of evaluation, as well as the composition and qualifications of evaluators (Marzano et al., 2020). The evaluators, therefore, play a vital role in the success of any teacher quality evaluation system. Given skepticisms about the validity of principals’\/administrators\u2019 evaluation of teachers and concerns about the reliability of relying on judgments from a single evaluator, it is recommended in the literature that multiple evaluators inform the evaluation of teacher quality, with experienced teachers as a key evaluator (Darling-Hammond, 2015; Gordon, 2020). Although top-down approaches toward teacher quality evaluation still dominate ELT in several contexts (where evaluation is done by administrators, policy makers, etc.), teachers have been recently acknowledged as plausible evaluators either in the form of self-evaluation or peer evaluation (Borg & Edmett, 2018).<\/p>\n Whatever their advantages and disadvantages, teacher self-evaluation and peer observation are currently employed in several educational programs worldwide. Teachers are often provided with a pre-specified set of criteria based on which they should evaluate their own or their peers\u2019 teaching (O’Leary, 2020). Probably due to the popularity of such methods, we know little about the criteria that teachers themselves utilize if required to evaluate teaching practices. Meanwhile, recent studies have mostly turned toward exploring the benefits, challenges, and obstacles involved in the process of self-\/peer-evaluation (e.g., Hendry et al., 2021; Visone, 2022). It should be noted that the majority of the studies in this area, as the following review reveals, have been conducted in general education contexts and in very few cases in ELT. Nonetheless, teachers, according to the available evidence, often use three kinds of criteria for evaluating their or other teachers\u2019 practices: (a) learners’ classroom behavior and experience, (b) teachers’ classroom practices, and (c) the output of teaching; that is, student learning. Among these, student learning experience has proved to be a key criterion that teachers employ for evaluation. In his study, Madsen (2005) relied on interviews to identify the criteria that novice teachers typically employ for evaluating their practices. Student classroom participation, interest, and autonomy have been reported as the major criteria that the teachers use to evaluate themselves. Nevertheless, teachers\u2019 actual self-evaluation practices need a more valid picture of the issue. To develop a self-evaluation questionnaire, more recently, Torres et al. (2017) addressed the issue from the perspective of lecturers from different educational disciplines. Student-related behavior such as participation recurred in the data. Torres et al., however, did not specify the teaching experience of the participants to ensure whether their perspectives were valid enough to inform a questionnaire. The extent to which the students showed interest in the lesson was further considered for self-evaluation by the primary student teachers in McCoy and Lynam\u2019s (2021) study.<\/p>\n Student learning has further emerged, albeit in a few studies, as a major criterion for self-evaluation among both pre-service and in-service teachers. The inexperienced and award-winning faculty members (Dunkin, 1991), the student elementary teachers (Dunkin et al., 1996), and both novice and expert primary teachers (Madsen, 2005) have referred to student immediate learning of a particular lesson, the ability to apply the learning in a later lesson, and long-term learning for evaluating themselves. Besides, many pre-service teachers in Olson et al.\u2019s (2004) study used student learning measured through achievement scores to judge the quality of their teaching. Success in formative and summative tests was also reported as a self-evaluation criterion by a few teachers in Kremer and Ben\u2010Peretz (2006) who used interviews to examine the issue among 90 teachers from various levels and schools in the northern district of Israel. However, this criterion has received scant attention in more recent studies addressing the issue, probably due to the recent recognition that student learning is affected by several factors other than teachers\u2019 pedagogical knowledge and expertise.<\/p>\n In addition to student-related factors, teacher attributes and skills have recurrently been mentioned as one of the evaluation criteria in prior studies. Prospective primary teachers in Madsen\u2019s (2005) study considered their affective (e.g., patience), cognitive (e.g., content knowledge), and behavioral (e.g., classroom management) features important for evaluating their practices. However, the author, as mentioned earlier, employed interviews rather than a more revealing method such as examining teachers\u2019 actual self-evaluation practices. This methodological limitation was improved by subsequent studies that used video-based reflection\/observation to investigate the issue. Nonetheless, except for Gr\u00f6schner et al.\u2019s (2018) study which did not specify the participants\u2019 teaching experience, the teachers inquired were restricted to pre-service ones. Although not directly exploring teacher self-evaluation criteria, Ibrahim et al.\u2019s (2012) study of teachers\u2019 reflective practices revealed that content knowledge, management of time and student behavior, and instructional issues (e.g., clarity of explanations) entailed the main features the chemistry student teachers considered in reflecting on the quality of their practices. In Gr\u00f6schner et al.\u2019s (2018) study, the ability to engage students in learning was the major issue on which the teachers relied for self-evaluation. More recently, using a video-mediated approach, the secondary pre-service teachers in Chand Dayal and Alpana\u2019s (2020) study employed teaching-related issues such as planning and classroom management, as well as teacher personal characteristics such as confidence and enthusiasm, to evaluate their teaching. Similarly, the primary student teachers in McCoy and Lynam\u2019s (2021) study valued both affective and behavioral aspects of their practices. The key evaluation criteria included managing time, maintaining discipline, instructional clarity, monitoring learning, creating an interactive environment, engaging students, and encouraging.<\/p>\n These criteria also emerged in peer observations, with studies incorporating experienced in-service teachers. Hammersley\u2010Fletcher and Orsmond (2005), for instance, examined the observation experience of teachers with a range of experience across two schools during 2002-2003 and found that the evaluation concentrated on mechanics of teaching (e.g., lesson structure) and student experience of learning (the degree to which they enjoyed learning). Promoting and encouraging student participation and engagement through activities such as group discussion entailed the key criterion the student teachers in Harford and MacRuairc\u2019s (2008) and the in-service teachers in Shortland\u2019s (2010) and Hendry et al.\u2019s (2014) research employed to evaluate their peers. Teachers\u2019 pedagogical practices (e.g., monitoring) further recurred in Torres et al.\u2019s (2017) investigation of the issue across different educational disciplines. To develop a peer observation instrument, Dillon et al. (2020) further explored criteria STEM faculty members from a university in Portland employed to evaluate peers. The use of group work and student engagement were the most important areas for the faculty. Similarly, in Hendry et al.\u2019s (2021) more recent study, although focusing on the advantages of peer observation, teachers\u2019 skills in engaging students and creating a motivating learning environment were the major quality indicators experienced physics academics used in peer observation.<\/p>\n Although not directly addressing the criteria teachers apply for teacher evaluation, the findings from a few studies suggest that ESL teachers draw on some common attributes\/skills to judge their own\/their peers\u2019 practices. To investigate peer observation opportunities for professional learning, Sarfraz (2019), for instance, examined experienced non-native ESL academics\u2019 observation practices. Teachers\u2019 content knowledge and teaching practices, as the observation notes revealed, constituted two general criteria the participants highly valued, although they emphasized teachers\u2019 mastery of subject matter over their pedagogical practices arguing that it will foster learner enthusiasm in the classroom. In a similar vein, Nguyen and Pham\u2019s (2020) exploration of the problems 10 English lecturers encountered in conducting observation of their colleagues in a university in Vietnam indicated that teachers\u2019 knowledge of the subject and their pedagogical methods were used as two significant criteria for teacher evaluation. More recently, Yuan et al. (2022) explored the extent to which pre-service non-native language teachers\u2019 reflective practices can contribute to their professional learning. Instructional strategies, teacher language, classroom interaction, and catering for student diversity entailed the major instructional criteria used in their reflections. Although these studies are illuminating, none of them investigated the issue in relation to native speaker teachers.<\/p>\n Other frequent criteria that teachers, according to the literature, use to evaluate their teaching, include judgments and comments of others such as students, colleagues, supervisors, and parents gained either through formal evaluation instruments or through some informal feedback. Approximately half of the pre-service and experienced teachers in Macleod (1988), several novice teachers and a few award-winning professors in Dunkin (1991), all the student teachers in Dunkin et al. (1996), and several faculty members in Wood and Su (2017) used feedback from external sources including supervisors, principals, colleagues, and student parents in evaluating themselves. A recent meta-review by Harrison et al. (2022) further reported student feedback as a key source of some teacher evaluations.<\/p>\n Notwithstanding this body of research that has addressed the criteria that teachers with differing educational backgrounds, teaching experience, and from varying disciplines employ in teacher evaluation, scant research in L2 education has problematized nativeness in teacher quality evaluation. Given that the criteria teachers employ in evaluation reflect their perceptions of what constitutes teacher quality and that teacher beliefs, whether explicitly or implicitly held, influence their practices, an exploration of native and non-native teachers\u2019 evaluation criteria can contribute to a better understanding of the debate over native\/non-native English speaker teachers. By demonstrating the extent to which native and non-native teachers\u2019 criteria converge\/diverge, the results can provide some evidence to support\/refute the traditional dichotomy between native and non-native teachers. The study, therefore, aimed to explore criteria native and non-native English language teachers apply for teacher quality evaluation. To fulfill the purpose of this research, the study addressed the following research question: What criteria do native and non-native English language teachers apply for teacher quality evaluation?<\/p>\n Two groups of teachers took part in the study: native teachers (i.e., teachers who speak English as their first language\/mother tongue) and non-native teachers (i.e., teachers for whom English is the second\/foreign language). In total, 50 native (male = 26, female = 24) and 50 non-native teachers (male = 21, female = 29) participated in the video-mediated peer observation on a voluntary basis and conducted the observation. They were selected based on convenience sampling, that is, according to their availability and willingness to cooperate. The non-native participants included Persian-speaking EFL teachers who taught at private English institutes across the country where English is learned as a foreign language. The native participants were ESL teachers teaching English at language institutes in a range of contexts including the United States, the United Kingdom, and Australia.<\/p>\n The participants had different educational backgrounds, ranging from B.A. to Ph.D., in English-related fields including Teaching English as a Foreign\/Second Language (native = 20, non-native = 25), English Literature (native = 4, non-native = 10), Linguistics (native = 6, non-native = 1), Translation (native = 1, non-native = 9), and Foreign Language Studies (native = 1). In a few cases, their education was in non-English fields such as Philosophy (native = 3), Education (native = 6), Psychology (native = 1), Engineering (non-native = 2), Architecture (non-native = 1), Biology (non-native = 1), Physics (non-native = 1), Finance (native = 1), Theology (native = 1), Business (native = 1), Tourism (native = 1), Law (native = 1), Distance Learning (native = 1), Criminal Justice (native = 1), and Psycholinguistics (native = 1). A number of participants (N = 7, NN = 15) further held CELTA. Moreover, they had varying teaching experience ranging from 1 to 33 years. Following Gatbonton (2008), those teachers who had less than two years of experience were regarded as novice and those having at least five years as experienced. Table 1 displays participants\u2019 demographic information.<\/p>\n Table 1. Participants\u2019 Demographic Information<\/strong><\/p>\n
\nTarbiat Modares University
\n<tajeddinz![\"atmark\"](\"http:\/\/www.tesl-ej.org\/atmark.png\")
modares.ac.ir><\/p>\n
\nAllameh Tabataba\u2019i University
\n<saeedi.zagmail.com><\/p>\n
\nAllameh Tabataba\u2019i University
\n<hamideh_mozaffarigmail.com><\/p>\nAbstract<\/h3>\n
Literature Review<\/h3>\n
Teacher Quality Evaluation<\/h4>\n
Teacher Criteria for Teacher Quality Evaluation<\/h4>\n
Method<\/h3>\n
Participants<\/h4>\n