Huu Thanh Minh Nguyen<\/strong> Nguyen Van Anh Le<\/strong> Comparing language tests and test preparation materials holds important implications for the latter\u2019s validity and reliability. However, not enough studies compare such materials across a wide range of indices. Therefore, this study investigated the text complexity of IELTS academic reading tests (IRT) and IELTS reading practice tests (IRPrT). Fine-grained quantitative analyses were undertaken to delineate measures of lexical, syntactic, and discourse complexity across a corpus of 108 IRT and 108 IRPrT published by Pearson, Macmillan, and Cengage Learning. The results suggest little difference between IRT and IRPrT at the lexical level; however, there were significant differences in some measures of syntactic and discourse level complexity. The findings bear implications for stakeholders including learners as test takers, instructors, material developers, and language testing researchers. We interpret this to mean that while IRPrT materials are lexically conducive to the practice for IRT and can provide a similar experience, the IRPrT do show some differences in the amount of subordination and idea repetition at the discourse level. Therefore, instructors and learners may seek to supplement practice with these structures when preparing for IRT , and the designers of such practice materials should consider aligning these factors in the future.<\/p>\n Keywords<\/em><\/strong>: Language testing, Reading tests, Reading practice test materials, Lexical text complexity, Syntactic text complexity, Discourse text complexity<\/p>\n Readability research and text-leveling schemes have emphasized the need to delineate text complexity to interpret the interaction between the reader and the text (Mesmer et al., 2012). Text complexity refers to the \u201ctext elements that can be analyzed, studied or manipulated\u201d (Mesmer et al., 2012, p. 236). According to Snow\u2019s (2002) RAND model of reading comprehension, text elements including vocabulary, syntax, and discourse affect how readers construct different representations of a text, including the surface code (i.e., the exact wording), the text base (i.e., idea units representing textual meaning), and the mental models (i.e., the way of processing textual information). For this reason, understanding text complexity at lexical, syntactic and discourse levels is necessary for research on L2 reading to better understand the relationship between texts and readers (Alderson, 2000; Guthrie et al., 2013).<\/p>\n High-stakes English language proficiency tests (ELPT) are important to test takers as gate-keeping tools. One such test is the International English Language Testing System (IELTS), which is often used for selection in academic contexts (Pearson, 2019). Because of its importance, preparation for the IELTS has become important, and increasing attention has been given to practice test materials, which are generally considered helpful to test takers (Kirby, 2016; O\u2019Sullivan et al., 2019). This has given rise to an international industry of practice materials for test preparation, but since such materials are not created by the test designers themselves, there are questions about how comparable the IELTS academic reading practice tests (IRPrT) are to the actual IELTS academic reading tests (IRT) (Bachman et al., 1996; Kunnan & Carr, 2017).<\/p>\n However, to our knowledge, no studies have been conducted to examine whether IRPrT have similar text complexity to IRT preparation. This is an important potential gap in the literature because incongruencies between practice test materials and the actual test could potentially negatively affect learners (Green, 2007). Therefore, this study seeks to compare the text complexity of IRT with IRPrT at the lexical, syntactic and discourse levels to uncover any significant differences so that educators can make informed decisions about the use of practice materials and so that the designers of such materials can potentially revise them if necessary.<\/p>\n L2 readability research has been viewed from two main strands. One strand is situated in traditional readability formulas that generally measure the number of words per sentence (i.e., sentence difficulty index), the number of syllables per words (i.e., word difficulty index), and word frequency (Brown, 1998; Greenfield, 1999). However, traditional readability formulas are criticized for being restrained to measures at the word level, given that L2 readability formulas are sensitive to other elements such as syntax and rhetorical organization (Carrell, 1987). Because of this limitation, the other strand has been motivated by more recent formulas that transcend the word level to demonstrate the relationships between textual elements. According to Crossley et al. (2008), the readability formula is not only constructed of word frequency but comprises such additional measures as syntax across sentences, and cohesion at the discourse level.<\/p>\n Text complexity shows a negative correlation with reading comprehension (Yang et al., 2021), although it is potential to support learner engagement with a text (Fulmer et al., 2015). Several empirical studies have revealed the relationships between lexical, syntactic, and discourse features of texts and reading comprehension. In terms of the lexical level, lexical sophistication, diversity and density are the three most widely examined properties (Read, 2000). Lexical sophistication is often considered a measure of what percentage of words the learner is likely to know (Laufer & Nation, 1995; Nation, 2006). Several works have suggested that readers need to be able to understand somewhere between 95 and 98% of the words or word families in a text in order to understand it (Laufer & Ravenhorst-Kalovski, 2010; Nation, 2006). Aside from lexical sophistication, higher lexical diversity, i.e., the amount of different words in the text, and density, i.e., the amount of content words, contributes to increasing difficulty in reading comprehension (Read, 2000).<\/p>\n Other studies have shown that the syntactic features of a text also impact its complexity (e.g., Perfetti & Stafura, 2014). For example, Shiotsu and Weir (2017) suggests that readers\u2019 syntactic knowledge accounts for more variance in the L2 reading comprehension level than lexical knowledge. This is because limited syntactic knowledge may impede the ability to understand a text despite the ability to understand meaning of single words within it (Tong et al., 2024). In addition to this, more complex sentences and grammatical structures make the sentences more difficult to be parsed, thereby possibly increasing text complexity beyond comprehension (Mesmer et al., 2012; Kyle, 2016).<\/p>\n Cohesion has also been suggested as the most prominent discourse level feature to impact text complexity in L2 readability research. In general, more cohesive texts are easier to be comprehended (Gernsbacher, 2013; Graesser et al., 2004), because complex mental representations and recall are not required (Ehrlich, 1991). The use of cohesive devices is conducive to establishing textual coherence (Goldman & Rakestraw, 2000);therefore, texts that are coherent at both sentence and global levels aid readers\u2019 memory of text information, increasing comprehension (Koda, 2005). However, increased text cohesion is generally accompanied by the inclusion of more information (Beck et al., 1991), which is associated with \u201cincreased text length, density, and complexity,\u2019 requiring readers to \u2018process larger amounts of text-based information” (Ozuru et al., 2009, p. 229).<\/p>\n Examining the comparability between reading tests and practice test materials is significant due to washback, i.e., \u201cthe effects of tests on the teaching and learning directed toward them\u201d (Green, 2006, p. 334). According to Green\u2019s (2007) model of washback direction, washback is positive if there is a consistency \u201cbetween test design and skills developed by a curriculum or required in a target language use domain\u201d (Green, 2006, p. 339). However, if there is a large gap between the test design in terms of format, content, and complexity and what is learned during test preparation, negative washback, such as construct-irrelevant variance, may occur (Messick, 1989). Therefore, examining the comparability of texts in authentic and practice tests can provide insights into how accurately they can reflect future test performance, and whether or not they are providing sufficiently realistic reading materials in terms of text complexity.<\/p>\n To our knowledge, there has been no investigation into IRPrT, except for Everett and Colman (2003). They examined the appropriateness of content in the listening and reading components of commercially produced IELTS practice tests dating from 1996 to 1998 and simply found that the vocabulary used in the IRPrT in their study varied from being unfamiliar and difficult to familiar while sentence structures ranged from complicated to uncomplicated. However, Everett and Colman (2003) only examined lexical and syntactic levels and did not look at cohesion or other discourse level factors. Furthermore, their study has become somewhat dated as it was conducted on much older materials and was limited to the technology available at the time, i.e., since such time many advances have been made in natural language processing that allow for more text features to be calculated automatically.<\/p>\n Given the lack of recent research in this area, we believe it is important to reexamine the text complexity of the reading passages of test preparation materials and actual reading test passages to investigate how valid more current materials are when viewed from a wider lense of text complexity. Accordingly, this study addresses the following research questions:<\/p>\n This study employed two corpora of IRT<\/a> and IRPrT<\/a>. The IRT were obtained from Cambridge IELTS series 9-17 published by Cambridge English Assessment. The IRPrT were extracted from the test preparation materials of three publishers:<\/p>\n The reading texts in both IRT and IRPrT vary in genres. Each corpus comprises 108 texts, with the average number of tokens in each text and the total number of tokens in total being relatively comparable (Table 1). It can therefore be argued that the compilation of both corpora was balanced and representative (McEnery, 2006).<\/p>\n Table 1. IRT and IRPrT corpora<\/strong><\/p>\n
\nUniversity of Foreign Language Studies, The University of Danang
\n<nhtminh![\"atmark\"](\"http:\/\/www.tesl-ej.org\/atmark.png\")
ufl.udn.vn><\/p>\n
\nUniversity of Foreign Language Studies, The University of Danang
\n<lnvanhufl.udn.vn><\/p>\nAbstract<\/h3>\n
Literature Review<\/h3>\n
Text Complexity and L2 Reading Comprehension<\/h4>\n
Text Comparability between IRT and IRPrT<\/h4>\n
\n
Methodology<\/h3>\n
Research Design<\/h4>\n
\n