Evaluating Language Assessments<\/h2>\n<\/td>\n<\/tr>\n\nAuthor:<\/strong><\/td>\nAntony John Kunnan (2018)<\/td>\n ![\"book](\"http:\/\/tesl-ej.org\/ej94\/rpix\/r3.jpg\")
<\/td>\n<\/tr>\n\nPublisher:<\/strong><\/td>\nNew York and London: Routledge<\/td>\n<\/tr>\n \nPages<\/th>\n ISBN<\/th>\n Price<\/th>\n<\/tr>\n \nPp. 1 – 273<\/td>\n 978-0-415-89777-8 (paper)<\/td>\n $41.95 U.S.<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\nEvaluating Language Assessments,<\/em> written by Antony Kunnan, is a reference book about language assessment. This book combines past frameworks and Kunnan\u2019s recent framework named the \u201cethics-based approach\u201d (p. 64). The book is divided into ten chapters: The Need for Evaluation (Chapter 1), Past Frameworks and Evaluations (Chapter 2), Ethics-Based Approach to Assessment Evaluation (Chapter 3), Building the Fairness and Justice Argument (Chapter 4), Opportunity to Learn (Chapter 5), Meaningfulness (Chapter 6), Absence of Bias (Chapter 7), Washback and Consequences (Chapter 8), Advancing Fairness and Justice (Chapter 9), and Applications and Implications (Chapter 10). This book is useful for educators, teachers, and students who want to know more about how to achieve justice and fairness in assessing language learners, a concept that Kunnan emphasizes with \u201cassessment agencies need to chart a development and research agenda that puts fairness and justice in the center of their evaluation frameworks\u201d (p. xi).<\/p>\nIn Chapter 1, Kunnan mentions the importance of assessment in education, work environments, and high-stakes contexts (e.g., citizenship tests). Since assessment can significantly impact a person\u2019s life, it is vital to create valid and fair assessments. Kunnan reviews assessment frameworks in Chapter 2. This review provides English as a Second Language\/Foreign Language (ESL\/EFL) teachers with a greater understanding of different ways to judge whether a test is fair, in addition to an understanding of the reasoning behind using these judgements. One way to address test fairness is to set up standards of tests, which are the requirements necessary for an impartial procedure of conducting a test. For example, standards set by the American Psychological Association (APA), the American Educational Research Association (AERA), and the National Council on Measurement in Education (NCME) claim that tests cannot contain language that is offensive to people of different races or genders, and that all test takers need to be treated fairly during each stage of the testing process. Apart from standards, using arguments is a method to evaluate if a test is valid or not. Specifically, to prove the validity of a test, researchers have to make claims and\/or sub-claims backed by evidence. For instance, if the claim is that the English test for naturalization in the U.S. is fair, there must be proof that test takers have enough opportunities to familiarize themselves with the test format before taking it.<\/p>\n
Since past assessment frameworks lack a philosophical foundation, Kunnan proposes an ethics-based approach to language assessment in Chapter 3. One key aspect of his approach is transparency. All test takers need to know the format of the test, how it is rated\/scored, and how decisions of passing and failing are made. His ethics-based approach also focuses on equity. Test takers who have the same scores in a certain test need to be treated equally regardless of their race, background, gender, or personal attributes. Furthermore, Kunnan proposes principles of fairness and justice at the core of his ethics-based approach. The principle of fairness demands that every test taker be treated the same; the principle of justice prescribes that the assessment benefit society and advocate morality. According to Kunnan, the principle of fairness has to be satisfied first, with the principle of justice examined later because if a test does not treat all test-takers equally first, it cannot benefit and bring justice to the society.<\/p>\n
In Chapter 4, Kunnan explains the connection among standard-based, argument-based, and ethics-based assessment frameworks. For instance, he says that ethics, or the principles of fairness and justice, are embedded in the standards set by the APA, AERA and NCME. To judge if these principles are achieved, an argument-based approach can be utilized. Chapters 5, 6, and 7 detail how arguments are built to investigate whether the APA, AERA and NCME standards are fair and just. Chapters 8, 9 and 10 discuss issues related to language assessment, including the influence of assessment on learning and teaching languages, how to improve the principles of fairness and justice among stakeholders, and how to apply an ethics-based approach in languages other than English.<\/p>\n
While Kunnan\u2019s ethics-based approach can serve as a philosophical framework for pedagogy, it has its challenges in language assessment research. A test has to satisfy all sub-principles (those mentioned above) to be considered fair and just; however, it may take a significant amount of time for a researcher to provide evidence to support all of the sub-principles. Although Kunnan supports his ethics-based approach in assessment research, many studies or projects he cites in the book do not specify how the approach is operationalized. For example, Kunnan discusses the FairTest (a non-profit organization that creates fair assessments) and the International Language Testing Association (and its development of a Code of Ethics), but there is no information regarding how these two organizations can support his ethics-based approach in their assessments.<\/p>\n
Despite the weaknesses, Evaluating Language Assessments <\/em>has many strengths. First, the chapters are organized coherently, and they are easy to follow. Kunnan starts by explaining the importance of evaluation and previous assessment frameworks, then details his own framework, the ethics-based approach. He also provides research-based evidence for whether fairness and justice are achieved in various English tests, such as the Certificate in Advanced English (a popular test administered by the University of Cambridge). Moreover, the book is useful for ESL\/EFL teachers as it provides extensive information about fairness and justice in language testing, including research ideas for assessment and a rationale for these factors in assessment. Additionally, the book includes specific suggestions for how to maintain fairness and justice in assessing students\u2019 language ability, such as testing what has been taught in class, making sure students understand the prompts of the test, and giving students with disabilities extra time to finish the test.<\/p>\nReference<\/h3>\n
Kunnan, A. J. (2018). Evaluating language assessments<\/em>. New York, NY: Routledge.<\/p>\nReviewed by<\/strong>
\nDung (Davy) Tran
\nUniversity of Texas at San Antonio
\n<dung.tran2<\/span>![\"atmark\"](\"http:\/\/www.tesl-ej.org\/atmark.png\")
utsa.edu<\/span>><\/p>\n
<\/td>\n<\/tr>\nEvaluating Language Assessments,<\/em> written by Antony Kunnan, is a reference book about language assessment. This book combines past frameworks and Kunnan\u2019s recent framework named the \u201cethics-based approach\u201d (p. 64). The book is divided into ten chapters: The Need for Evaluation (Chapter 1), Past Frameworks and Evaluations (Chapter 2), Ethics-Based Approach to Assessment Evaluation (Chapter 3), Building the Fairness and Justice Argument (Chapter 4), Opportunity to Learn (Chapter 5), Meaningfulness (Chapter 6), Absence of Bias (Chapter 7), Washback and Consequences (Chapter 8), Advancing Fairness and Justice (Chapter 9), and Applications and Implications (Chapter 10). This book is useful for educators, teachers, and students who want to know more about how to achieve justice and fairness in assessing language learners, a concept that Kunnan emphasizes with \u201cassessment agencies need to chart a development and research agenda that puts fairness and justice in the center of their evaluation frameworks\u201d (p. xi).<\/p>\n In Chapter 1, Kunnan mentions the importance of assessment in education, work environments, and high-stakes contexts (e.g., citizenship tests). Since assessment can significantly impact a person\u2019s life, it is vital to create valid and fair assessments. Kunnan reviews assessment frameworks in Chapter 2. This review provides English as a Second Language\/Foreign Language (ESL\/EFL) teachers with a greater understanding of different ways to judge whether a test is fair, in addition to an understanding of the reasoning behind using these judgements. One way to address test fairness is to set up standards of tests, which are the requirements necessary for an impartial procedure of conducting a test. For example, standards set by the American Psychological Association (APA), the American Educational Research Association (AERA), and the National Council on Measurement in Education (NCME) claim that tests cannot contain language that is offensive to people of different races or genders, and that all test takers need to be treated fairly during each stage of the testing process. Apart from standards, using arguments is a method to evaluate if a test is valid or not. Specifically, to prove the validity of a test, researchers have to make claims and\/or sub-claims backed by evidence. For instance, if the claim is that the English test for naturalization in the U.S. is fair, there must be proof that test takers have enough opportunities to familiarize themselves with the test format before taking it.<\/p>\n Since past assessment frameworks lack a philosophical foundation, Kunnan proposes an ethics-based approach to language assessment in Chapter 3. One key aspect of his approach is transparency. All test takers need to know the format of the test, how it is rated\/scored, and how decisions of passing and failing are made. His ethics-based approach also focuses on equity. Test takers who have the same scores in a certain test need to be treated equally regardless of their race, background, gender, or personal attributes. Furthermore, Kunnan proposes principles of fairness and justice at the core of his ethics-based approach. The principle of fairness demands that every test taker be treated the same; the principle of justice prescribes that the assessment benefit society and advocate morality. According to Kunnan, the principle of fairness has to be satisfied first, with the principle of justice examined later because if a test does not treat all test-takers equally first, it cannot benefit and bring justice to the society.<\/p>\n In Chapter 4, Kunnan explains the connection among standard-based, argument-based, and ethics-based assessment frameworks. For instance, he says that ethics, or the principles of fairness and justice, are embedded in the standards set by the APA, AERA and NCME. To judge if these principles are achieved, an argument-based approach can be utilized. Chapters 5, 6, and 7 detail how arguments are built to investigate whether the APA, AERA and NCME standards are fair and just. Chapters 8, 9 and 10 discuss issues related to language assessment, including the influence of assessment on learning and teaching languages, how to improve the principles of fairness and justice among stakeholders, and how to apply an ethics-based approach in languages other than English.<\/p>\n While Kunnan\u2019s ethics-based approach can serve as a philosophical framework for pedagogy, it has its challenges in language assessment research. A test has to satisfy all sub-principles (those mentioned above) to be considered fair and just; however, it may take a significant amount of time for a researcher to provide evidence to support all of the sub-principles. Although Kunnan supports his ethics-based approach in assessment research, many studies or projects he cites in the book do not specify how the approach is operationalized. For example, Kunnan discusses the FairTest (a non-profit organization that creates fair assessments) and the International Language Testing Association (and its development of a Code of Ethics), but there is no information regarding how these two organizations can support his ethics-based approach in their assessments.<\/p>\n Despite the weaknesses, Evaluating Language Assessments <\/em>has many strengths. First, the chapters are organized coherently, and they are easy to follow. Kunnan starts by explaining the importance of evaluation and previous assessment frameworks, then details his own framework, the ethics-based approach. He also provides research-based evidence for whether fairness and justice are achieved in various English tests, such as the Certificate in Advanced English (a popular test administered by the University of Cambridge). Moreover, the book is useful for ESL\/EFL teachers as it provides extensive information about fairness and justice in language testing, including research ideas for assessment and a rationale for these factors in assessment. Additionally, the book includes specific suggestions for how to maintain fairness and justice in assessing students\u2019 language ability, such as testing what has been taught in class, making sure students understand the prompts of the test, and giving students with disabilities extra time to finish the test.<\/p>\n Kunnan, A. J. (2018). Evaluating language assessments<\/em>. New York, NY: Routledge.<\/p>\n Reviewed by<\/strong>Reference<\/h3>\n
\nDung (Davy) Tran
\nUniversity of Texas at San Antonio
\n<dung.tran2<\/span>utsa.edu<\/span>><\/p>\n