May 2025 – Volume 29, Number 1
https://doi.org/10.55593/ej.29113a2
Humairah Fauziah
Universitas Trunojoyo Madura, Bangkalan, Indonesia
<humairah.fauziah
trunojoyo.ac.id>
Yazid Basthomi
Universitas Negeri Malang, Malang, Indonesia
<ybasthomium.ac.id>
Nurenzia Yannuar
Universitas Negeri Malang, Malang, Indonesia
<nurenzia.yannuar.fsum.ac.id>
Abstract
Concordancer tools have been used to help students learn aspects of written English by engaging them to access, examine, and analyze language examples in a corpus. Nonetheless, their efficacy remains uncertain when compared to other tools. This study compared a concordancer tool to Grammarly, QuillBot, Google search, online dictionaries, and students’ own knowledge to carry out language exercises requiring the students to choose correct forms. It involved 59 EFL undergraduate students enrolled in English writing classes. The students were presented with a sentence completion exercise addressing four areas: subject-verb agreement, tenses, prepositions, and correct word usage. They were given the option of using a concordancer or other tools to address the sentence completion exercise. Data came from worksheets and an open-ended questionnaire. The results demonstrated that the concordancer had varying degrees of effectiveness in helping the students identify correct language forms and retain the language they learned across the four assessed areas. The questionnaire revealed that the students found the concordancer challenging and preferred other tools. The paper also offers recommendations for addressing the students’ difficulties when using the concordancer tool.
Keywords: Corpus-based learning, Data-Driven Learning, English language accuracy English language retention, Grammarly, QuillBot
One of our EFL students once inquired about the correct use of “prefer than” versus “prefer to” in a sentence. In response, we directed him to consult a grammar book for the answer. When he could not find the information in the book, we provided him with the correct answer. Carter and McCarthy (1988) argued that providing the students with the correct answer directly without encouraging them to figure it out on their own is less communicative and has a short-term memory impact. They contended that this method tends to keep the students from actively engaging with the language and hinders their ability to draw connections between the language form and context, leading to the possibility that they will forget things easily. To increase the durability of learning, teachers should promote students’ awareness of language forms that indirectly contribute to language acquisition by providing them with more interactive and communicative language learning activities that could promote the students’ participation and enable them to improve their language skills more naturally and practically (Cook, 2008).
Data-Driven Learning (henceforth, DDL) provides an excellent opportunity for language learners to engage with the language more interactively and communicatively. DDL in ELT involves the use of corpora to explore patterns of language use in authentic contexts. Some researchers have found that DDL could enhance students’ language skills, as well as develop their high-order thinking skills and language awareness. A growing number of DDL studies have recognized the use of corpora to help students solve linguistic problems and carry out error correction (see Bridle, 2019; Cheng, 2021; Crosthwaite, 2017; Quinn, 2015). However, these studies notably focused on using a concordancer tool (software used to access, retrieve, and analyze language data from a corpus) and did not take into account the potential of other tools in students’ self-correction practices. Using a concordancer tool that is unfamiliar to them and rarely employed by students in their typical learning activities, as noted by Liou (2019), could lead to reluctance among students to engage with concordancing for self-correction (Liou, 2019). In other words, if students are asked to utilize a tool outside their usual practice, there is a chance they might feel overwhelmed, lack motivation, or be uninterested in adopting the new tool. Besides, as Cheng (2021) mentioned, concordancer tools might not be appropriate for all students. Therefore, there is a need to use other, possibly more accessible tools for helping students find correct English language forms. Yoon and Hirvela (2004) believed that concordancer tools and dictionaries can complement each other, depending on the tasks involved.
Cheng (2021) and Crosthwaite (2017) compared concordancer tool with other tools, allowing students to use their preferred tools for correcting their errors, especially in grammar. However, little is known regarding the effectiveness of concordancer tools compared to other tools in the extent to which the students can recall the correct forms later on. There remains a considerable gap in our understanding of its impact on students’ ability to retain language forms over time. The gap highlights the need for further investigation into the effectiveness of concordancer tools in comparison to other methods for promoting students’ long-term retention of language after using these tools to find the correct English language forms.
Literature Review
Corpus Integration in English Language Teaching and Learning
Initially, corpora were employed as primary data sources in linguistic research (Lin, 2021) including for examining linguistic features (see Basthomi et al., 2015; Kirana et al., 2018; Yannuar et al., 2014) or developing language teaching materials (Ahsanuddin et al., 2022). As noted above, the integration of corpora in English Language Teaching (ELT) classrooms is sometimes referred to as DDL (Data Driven Learning), a term that was first coined by Tim Johns (Johns, 1991). In DDL, students are engaged in examining and analyzing the target language in the corpus and then formulating a conclusion based on their observations. Investigating the target language in a corpus offers students valuable insights into its contextual usage, as corpora encompass authentic discourse, including written texts and spoken language transcripts, from diverse settings (Bernardini, 2000). This awareness assists students in cultivating a more precise and authentic grasp of the target language, reducing their first language interference, and promoting enhanced language acquisition. DDL entails a recursive process that iteratively persists until the validity of the information is established (Boulton, 2010; Johns, 1991). It starts with the identification of a particular language issue that students are going to work on. In order to find answers, they examine the target language in the corpus and find relevant patterns associated with the language problems. Subsequently, they form hypotheses and establish rules about grammar, meaning, or language usage. These formulated rules may be either accurate or erroneous; thus, students must verify their hypotheses by referring to their grammar textbook or other credible sources. If their hypotheses prove correct, they can apply the acquired knowledge to new cases or contexts; otherwise, they must re-evaluate the target language in the corpus until they can confidently affirm the validity of their findings (see Figure 1).
Figure 1. The Steps of Corpus-Based DDL Classroom Activities (based on Boulton, 2010 and Johns, 1991)
In various research studies, two common forms of DDL activities for grammar instruction are mentioned, namely concordance analysis (Bridle, 2019; Corino & Onesti, 2019; Lin, 2021) and error correction (Bridle, 2019; Chen et al., 2019; Cheng, 2021; Crosthwaite, 2017; Dolgova & Mueller, 2019). In concordance analysis, students examine concordance lines for a specific word or phrase to identify its collocates and determine common usage patterns, while in the error correction activity, students use the corpus as a reference tool to correct their language errors that have been marked by the teachers. Lin (2021), for example, carried out a comparative study to examine the effect of DDL on students’ grammar proficiency across various grammar proficiency levels and found that students made a significant improvement in their grammar performance after engaging with DDL. In Lin’s study, the experimental group outperformed their counterparts in the control group. This indicates that the DDL approach has the potential to be a beneficial pedagogical tool for enhancing grammar learning, especially among those learning English as a foreign language.
Even though DDL has several benefits for language learning, some studies have indicated that it may not be able to address all language issues. For instance, Crosthwaite (2017) investigates the use of a corpus tool to correct errors in students’ L2 writing. In his study, teachers provided feedback on students’ writing composition and the students then made corrections and highlighted them to indicate which were made with and which without the help of corpora. The results indicated that students corrected problems in word choice, word form, collocations, and phrasing by using corpora. However, they were less inclined to use corpora for addressing deletion errors (errors where a word needs to be deleted to make the text correct) or morphosyntax errors. In addition, Dolgova and Mueller (2019) found that participants used corpus tools to correct local lexico-grammatical errors in more than half of their revisions, but these corrections were frequently found to be inappropriate, whereas students’ corrections of register errors were more often correct. The fact that DDL could not equally address all types of errors highlights the importance for teachers implementing DDL to consider the specific error types that align with their students’ needs (Boulton & Tyne, 2013), allowing students to utilize the knowledge acquired through DDL in their own written or spoken language.
Concordancing in Corpus-Based Data-Driven Learning
In DDL, the core process is called concordancing. This involves accessing, retrieving, and analyzing language data from corpora using a concordancer tool. This process requires entering a specific word or phrase, referred to as the Key Word in Context (KWIC), into the concordancer tool. The tool then retrieves all instances from the corpus, known as concordance lines, that include the target KWIC. For example, when students search the corpus for the term “teach,” they will see multiple concordance lines displaying forms of the word “teach” in various contexts (see Figure 2).
Figure 2. Concordance Lines from KWIC “teach” in the BAWE Corpus Using the Sketch Engine Concordancer
The concordancer tool for querying corpora in DDL should be designed to be user-friendly and easily accessible for individuals who have limited familiarity with working with corpora (Lee et al., 2019). Some popular programs to access corpora include AntConc (Anthony, 2019), SkELL (https://skell.sketchengine.eu/) and Sketch Engine (https://app.sketchengine.eu/) (Kilgarriff et al., 2015), Word Smith Tools (Scott, 2020) and BNClab (Brezina et al., 2018). The software frequently offers an array of functions like sorting, filtering, and frequency analysis, enhancing students’ ability to manipulate and explore corpus data more effectively (Lee et al., 2015).
In the literature, concordancing is claimed to have several advantages in ELT contexts. First, it empowers students with the element of autonomous learning, since they have to analyze the language patterns for themselves. Yoon and Jo (2014) reported that, by using the concordancer tool, students became aware of their difficulties in writing genre-specific texts in English. Second, it encourages students to be engaged in exploratory learning; Hafner and Candlin (2007) found that concordancing gave students motivation for lifelong exploratory learning because it satisfied their personal and professional curiosity about new words. Besides, students could use the concordancer tool as a problem-solving device to notice collocations, conventions, and connections between grammar and vocabulary in professional texts. This may also allow them to transfer word knowledge and usage patterns to their pleasure reading and academic writing (Lin, 2021).
Besides its benefits, concordancing also offers several challenges for ELT due to its limitations. The first is dealing with students’ ability to analyze the language. As stated by Yoon and Hirvela (2004), some students do not analyze the concordance lines effectively, suggesting that DDL requires students to have a certain level of language analysis skills that not all learners possess. The second issue concerns the format of concordance lines, which often appear as fragmented or incomplete sentences. This causes students to experience difficulty in analyzing them, as the lack of full context makes it harder to grasp the intended meaning and usage of the language. Lai (2015) points out that students often face difficulties in working with lists of concordance lines; this can be attributed to their unfamiliarity with the format or structure of these lists, which can appear overwhelming or complex. The last is about the students’ technical ability to operate the concordancer tool. According to Boulton (2019), students occasionally lack the skills essential to use the concordancer tool effectively and retrieve the information they require. Given the challenges students face while working with concordance lines, teachers should offer guidance and support during the implementation of DDL, while also taking into account the students’ current skills and familiarity with linguistic analysis tools.
Some scholars have taken a broader perspective on the effectiveness of concordancer tools compared to other tools in helping students correct their errors. For instance, Bridle (2019) compared the use of a concordancer as a reference tool to access the British National Corpus (BNC) compared to dictionaries and students’ own knowledge to correct their writing errors in four short essays. After students had made four revisions of their text, the use of the concordancer tool was found to rank second following the dictionary in aiding students to make accurate corrections consistently for over 80% of errors. It was seemingly most effective for correcting “Wrong Words” and “Formal/Informal” errors, but less effective for “Articles”, “Grammar”, “Incomprehensible” parts, and “Missing Words” errors. Similarly, Cheng (2021) compared concordancer tools to other tools such as dictionaries, Google, Google Translate, or students’ own knowledge and found that students predominantly relied on their own knowledge for correcting errors, while the use of corpus consultation remained constrained. In Fauzan et al.’s study (2022), a comparison was made between the impact of utilizing a concordancer tool to access online corpora and the impact of utilizing online dictionaries in enhancing EFL students’ grammar proficiency. The statistical analysis of grammar test outcomes indicated that there was no significant distinction in students’ grammar mastery following their exposure to either the online corpus or the online dictionary for learning grammar.
Direct and Indirect Use of Corpora
In DDL, students are permitted to engage with corpora both directly and indirectly, as highlighted by Crosthwaite (2019), Godwin-Jones (2017), and Yoon and Jo (2014). The distinction between direct and indirect use of corpora in how learners interact with and access language data within the corpus is illustrated in Table 1. Direct corpus use entails teachers granting students direct access to the corpus, enabling them to explore target language patterns independently through specific software (Sun & Hu, 2020). The role of teachers in direct corpus use is to verify and validate students’ findings in the corpus, as there is a potential for misinterpretation of the research results. On the other hand, indirect corpus use, also referred to as hands-on concordancing (Boulton, 2010) and using corpus printouts (Stevens, 1991) involves students not having direct access to the corpus; instead, teachers furnish them with corpus printouts containing various examples from the corpus. In this scenario, teachers not only validate students’ hypotheses but also curate suitable examples while excluding instances that might be too challenging for the learners.
Table 1. Direct and Indirect Use of Corpora in DDL
| No | Direct Use of Corpora | Indirect Use of Corpora |
| 1 | Introduce the Language Topic: Teacher identifies and introduces the specific language topic or feature to be explored using the corpus. This could be grammar aspects, vocabulary usage, collocations, or any other language aspect. | |
| 2 | Access the Corpus: Students access the corpora directly, either through online platforms or software. The teacher familiarizes them with the tools and functions available for searching and retrieving language data. | Teacher’s Pre-selection: Teacher pre-selects and compiles a set of relevant language examples or concordance lines from the corpus that illustrate the target language feature. These examples should be representative and varied in their usage |
| 3 | Retrieve Examples: Students search for specific keywords or phrases related to the language to retrieve relevant examples or concordance lines that illustrate the target language feature. | Provide Examples: Teacher presents the pre-selected examples to the students and invites students to discuss the patterns, collocations, or grammatical structures present in each example and how they relate to the language topic. |
| 4 | Analyze Examples: Students analyze the retrieved language data, observe patterns, identify collocations, examine grammatical structures, and make observations about the frequency, context, and usage of the language feature. | Analyze Print-out Examples: Students analyze the provided examples, observe patterns, identify collocations, and examine grammatical structures. |
| 5 | Formulate Hypotheses: Based on their analysis, students formulate hypotheses or create rules to explain the observed language patterns. | |
| 6 | Validate Hypotheses: Under the teacher’s guidance, students validate their hypotheses using external sources such as grammar reference works or textbooks and compare their findings with established explanations to confirm or adjust their understanding of the language feature. | |
| 7 | Reflect and Discuss: Students discuss and share their observations, interpretations, and insights with their counterparts. This collaborative learning environment encourages students to learn from one another’s perspectives. | |
| 8 | Apply and Practice: Students apply the discovered language patterns or rules in other contexts by engaging in communicative activities, writing exercises, or undertaking tasks that require the use of the identified language feature. | |
Godwin-Jones (2017) suggests that teachers encourage students to use the corpus directly by exploring the corpus to identify the frequency of words and observe how phrase patterns and language rules are applied. Otherwise, the teachers could also use printed handouts compiling some search results from a corpus and ask the students to answer some questions and do some exercises based on the pre-prepared concordance lines provided to them. The choice between direct and indirect use of a corpus depends on various factors, including the learners’ proficiency levels, access to resources, and instructional goals. Direct use of corpora offers more autonomy (Boulton & Cobb, 2017; Chambers, 2007) and exploration opportunities, promoting independent learning and critical thinking skills. To effectively explore and interpret the language forms while accessing the corpus directly, students should possess certain technical skills and language competency (Sun & Hu, 2020), while indirect use of corpora requires teachers to provide students with greater support and scaffolding, enabling a more organized and directed learning process.
While the studies mentioned earlier have delved into various aspects of using concordancer tools in DDL, there is a notable gap in the research when it comes to investigating the impact of concordancer tools on students’ retention of English language skills in comparison to other tools. Despite there being a variety of studies examining various aspects of corpus-based learning, little is known about the specific question of how effectively the use of concordancer tools in corpus-based learning affects language knowledge retention over a period of time. This discrepancy is noteworthy since language retention is a significant indicator of the extent to which a language learning strategy is effective. One of the most important aspects of language acquisition is the ability of students to remember language patterns for future use, in addition to being able to identify an appropriate form in the short term. Assessing whether corpus-based learning can improve retention compared to other approaches should shed light on how long-lasting and useful concordancing is as a language learning tool.
Bridging this gap could result in a greater understanding of the benefits of corpus-based learning in promoting the students’ retention of language usage Therefore, this study seeks to answer the following research questions:
- What methods do students prefer for selecting correct language forms in a sentence completion exercise?
- How accurately does a concordancer tool help students discover the correct forms compared to other tools?
- To what extent do students retain their knowledge after using a concordancer toolcompar ed to using other tools for finding correct forms?
- What are students’ perceptions of using a concordancer tool to assist them in discovering the correct form in a sentence completion exercise?
Method
Participants
We conducted this study in a non-English department at a university in Indonesia and recruited 59 students, who were enrolled in an English writing class. This writing class was an elective course that the students could take during their sixth semester; it was their second English course at the university, and they had taken an English for General Purposes course in their third semester. The main objective of the writing class was to help the students develop their basic English writing skills.
Before conducting this research, one of the researchers (acting as the classroom instructor) informed the students about the course description, objectives, learning materials, and the activities they were going to participate in during the semester. We also emphasized our intention to observe their learning activities throughout the semester and provided them with a consent form to sign. We assured them that their data would not be included in our research findings if they preferred not to participate in the study. We also requested the students to complete a survey aimed at assessing their knowledge of concordancer tools and their preferences for English learning and self-correction tools. From the survey, we found that the students had no prior familiarity with corpora or concordancer tools. They often used automated grammar tools such as Grammarly and QuillBot to assist them in correcting errors while writing in English. Additionally, they depended on Google searches and online dictionaries as reference tools for their English learning needs.
Research Procedures
This study was part of a larger research initiative investigating the application of DDL in an English writing classroom. Spanning one semester and consisting of 16 meetings, the broader project aimed to explore how DDL could enhance students’ language learning experiences. Within this context, the present study focused specifically on the implementation of Sketch Engine as a concordancer tool to support grammar learning. Table 2 provides a comprehensive outline of the research procedures employed throughout the study.
Table 2. The Research Procedures
| Meeting | DDL Classroom Activities |
| 1 – 5 | Teaching fundamental English writing concepts, such as parts of speech, tenses, and sentence structures (utilizing both direct and indirect corpus use). |
| 6 | Corpus training: Introducing the BAWE Corpus and Sketch Engine as a concordancer tool, including instructions on selecting suitable KWIC for analysis (direct corpus use). |
| 7 | Corpus training: Practicing grammar through subject-verb agreement exercises using the Sketch Engine (direct corpus use). |
| 8 | Conducting the first sentence completion exercise. |
| 9-15 | Enhancing students’ English writing skills by focusing on outlining, drafting, and revising, following a DDL approach (indirect corpus use). |
| 16 | Conducting the second sentence completion exercise and administering a questionnaire. |
This study focused on evaluating the use of Sketch Engine, a concordancer tool introduced during the sixth and seventh classroom activities, and comparing its effectiveness with other instructional tools, conducted through a sentence completion exercise administered in the eighth and sixteenth meetings.
The Corpus Training
In the sixth and seventh meetings, the students were involved in a corpus training process, in which the students were introduced to the British Academic Written English (BAWE) corpus and guided on how to use Sketch Engine to explore specific English language patterns in the BAWE corpus. The primary goal of this training was to familiarize students with the concordancer tool, enabling them to identify key terms within context (KWIC) accurately. Additionally, the training aimed to enhance their ability to analyze concordance lines, allowing them to extract meaningful insights and effectively apply observed language patterns to complete the exercise tasks. As depicted in Figure 3, the students were tasked with subject-verb agreement exercises. In these exercises, they were required to identify and underline a KWIC, guiding them in the selection of the correct verb for each sentence.
Figure 3. Subject-verb Agreement Exercise
For instance, in question 1, the students were required to put the keyword “one of” in the concordancer tool. They were then directed to search for comparable patterns between the question and the concordance lines (see Figure 4), in which the students deduced the correct verb agreement for the subject “one of + plural form,” was “is”.
Figure 4. The Concordance Lines for the KWIC “one of”
The issue was that some students struggled to identify the correct KWIC, resulting in the retrieval of concordance lines that were not relevant to the sentence being analyzed. For example, in sentence 9, which stated “There (is/are) several kinds of flowers in the bouquet,” some students with limited knowledge of English structure selected the word “there” as the KWIC. This case confused the students, as the concordance lines displayed both “is” and “are” following “there,” as shown in Figure 5.
Figure 5. The Concordance Lines for the KWIC “there”
The correct KWIC to use was “several,” which helped the students retrieve concordance lines more relevant to the question’s context. As shown in Figure 6, when students entered “several” as the KWIC, Sketch Engine provided concordance lines that better matched the sentence’s context.
Figure 6. The Concordance Lines for the KWIC “several”
Another issue was that some KWIC terms did not appear in the corpus because the corpus did not contain those specific words. For instance, in sentence number 10, when students mistakenly used “firecrackers” as the KWIC instead of “noise,” they could not find any concordance lines with the word “firecrackers.” The corpus does not include every possible search term, as shown in Figure 7.
Figure 7. The Page View for the KWIC “firecrackers”
The cases presented above suggest that accurately identifying the correct KWIC is closely related to the students’ level of English language proficiency. Those with a stronger background in English tended to be more successful in both selecting the appropriate KWIC and analyzing language patterns within the concordance lines, compared to their peers with lower proficiency. This observation is consistent with Lin’s (2021) research, which highlights that a strong understanding of English not only facilitates the identification of relevant KWIC but also enhances students’ effectiveness in utilizing concordancer tools to explore and interpret language patterns.
Data Collection
The data collection tools for this study comprised a sentence completion exercise and a questionnaire. A detailed explanation of each data collection method is provided in the following section.
The Sentence Completion Exercise. The first data collection procedure used was the sentence completion exercise. This was set once the students had become familiar with the concordancing activity for a grammar exercise. We assigned the students a binary choice version of a sentence completion exercise to be answered using their preferred tools (see Appendix 1). The exercise items were designed based on common errors found in essays written by the students at the beginning of the semester, and they were categorized into four groups with five items in each (see Table 3): (1) subject-verb agreement; (2) tenses; (3) prepositions; and (4) correct word usage.
Table 3. Target Language Usage for the Sentence Completion Exercise
| Language Usage | Example of Exercise Items |
| Subject-verb agreement | Everyone (want/wants) to know the name of the newborn baby. |
| Tenses | You (ate/have eaten) three ice-creams since morning. |
| Prepositions | The bank is close (to/with) the main train station. |
| Correct word usage | She runs (fast/fastly) because the dog keeps chasing her. |
The initial sentence completion exercise was set during our eighth week, where students employed a range of tools, including a concordancer tool, Grammarly, QuillBot, Google, and online dictionaries, to choose the correct language forms to complete the sentences. Following the initial running of the sentence completion exercise, the students were neither given the correct answers nor engaged in any discussion about whether their responses were accurate or incorrect. Instead, they participated in indirect DDL activities that emphasized developing their comprehension of the writing process, which aimed at guiding the students in constructing paragraphs by applying the DDL approach, allowing them to explore patterns and structures to understand how to write effectively. Later, in the sixteenth week, we repeated the same exercise with a modification, wherein the students received the same exercise but with the items in a different sequence than before. During the second iteration of the activity, we introduced a change in the approach. The students were prohibited from using any tools. The primary aim of this second sentence completion activity was to discern which tool, among those employed initially, had a more lasting effect on the students’ language retention. By prohibiting the use of external tools in the second iteration, we aimed to investigate the impact of the initial exposure and practice on the student’s ability to retain and apply grammar corrections independently over time.
During the first sentence completion exercise, the students had access to several tools: a concordancer tool, Grammarly/QuillBot, Google Search, an online dictionary or textbook, and their own prior knowledge. In the following section, we focus specifically on the concordancer tool and Grammarly/QuillBot, as these may be less familiar to readers and offer unique functionalities that set them apart from traditional resources like dictionaries or general search engines.
Concordancer Tool. The concordancer tool available to students in the sentence completion exercise was Sketch Engine, a web-based concordancer and corpus analysis tool (www.SketchEngine.eu), developed by British corpus linguist Adam Kilgarriff (Kilgarriff et al., 2015). This tool was selected primarily due to its compatibility with both mobile phones and laptops, along with its user-friendly interface, as noted by Bridle (2019), making it accessible to all students. It has eight features (see Figure 8) among which the concordance feature was the only one used during the corpus training and the sentence completion exercise.
Figure 8. Sketch Engine and its Features
The Sketch Engine simplifies the process of finding suitable concordance lines for students by offering a filter menu that identifies parts of speech appearing before or after the KWIC. Students can adjust the filter’s position within a range of 1-10 words on the left, right, or both sides. For instance, students were provided with the sentence, “There is/are several kinds of flowers in the bouquet,” where they needed to decide whether “is” or “are” correctly aligns with “several kinds.” To assist with this, they entered the word “there” as the KWIC and applied a “noun” filter on the right side within a range of five words (see Figure 9).
Figure 9. Filter Menu in Sketch Engine
When “there” was used as the KWIC with “noun” set as the filter, the concordance lines in the corpus highlighted both terms in red, providing a visual aid that helped students identify the appropriate article usage. As shown in Figure 10, the concordance lines displayed “there” followed by either the singular “is” or the plural “are,” depending on the accompanying noun. By analyzing these lines, students determined that the correct choice for the sentence “There is/are several kinds of flowers in the bouquet” was “are,” similar to the structure in the example sentence “there are several reasons for this decline.”
Figure 10. Searching for “There” Using Filter “Noun” on the Left Side
Grammarly. The second tool available in the sentence completion exercise was Grammarly. It is a digital tool providing computer-mediated corrective feedback for students’ writing. It identifies errors in grammar, vocabulary, mechanics, and language style, proving valuable during the revision and editing stages of writing work. Grammarly offers real-time feedback by underlining errors and inappropriate language use, using color-coded underlines for specific issues. On the right side of the editing textbox, as depicted in Figure 11, Grammarly provides corrections with corresponding explanations and an overall text score (1 to 100), where a higher score indicates fewer suggestions for corrections and thus a more accurate text (Barrot, 2020).
Figure 11. Grammarly Editing Textbox
QuillBot. QuillBot was also available for use in the exercise. QuillBot is primarily an AI-powered paraphrasing tool designed to rephrase and rewrite text while retaining the original meaning. Additionally, it features a grammar checker that offers direct feedback on correct forms, categorizing mistakes into grammar or fluency issues. The tool swiftly reviews and highlights errors, providing suggestions for correction. As illustrated in Figure 12, QuillBot also allows users the option to correct all mistakes simultaneously, saving time for those who prefer not to review individual error explanations (Malik et al., 2023).
Figure 12. QuillBot Editing Textbox
In the sentence completion exercise, Grammarly and QuillBot were integrated into the same set of tools rather than being used separately, because they provide similar functions in grammar checking. Both tools highlight incorrect words by underlining them and suggest appropriate corrections.
Questionnaire
The second data collection instrument was a questionnaire, which comprised eight open-ended questions, each carefully formulated to elicit specific information and written in Indonesian as all the students speak that language to facilitate their understanding of the questions (see Appendix 2 for the English translation). The initial question aimed to gain insights into the students’ preferred tools for addressing grammar correction activities. Participants were encouraged to provide detailed explanations and rationale for their choice of a specific tool. In the subsequent four items (questions 2, 3, 4, and 5), the participants were prompted to share their thoughts on the practicality of the concordancer tool, emphasizing both its advantages and disadvantages based on their experiences during the study. Finally, the last three items (questions 6, 7, and 8) inquired about the students’ preferences for future language-learning tools, enabling participants to articulate their preferences for tools they would like to use in future language-learning activities.
Data Analysis
Two sets of data were analyzed: data from the sentence completion exercise and data from the questionnaire. The results of the sentence completion exercise were scrutinized to assess the accuracy rates of each method, and the students’ language retention after using each method. To evaluate the students’ language retention after using each tool, we first identified correct responses in the initial exercise, and the tools that had been used to produce these correct responses. We then identified whether or not the students had again given the correct responses to these items in the second administration of the exercise. Finally, we calculated the percentage of repeated correct responses produced by each of the tools that the students used in the first administration of the exercise.
In analyzing the questionnaire data, we employed thematic analysis, which is intended to identify, analyze, and report the prevalent themes (Daniel & Harland, 2018). This involved systematically identifying recurring themes and patterns in the responses. We began by assigning brief labels to the main ideas, refining them to capture details in the data. Then we organized the data according to these themes for a more structured analysis. In the interpretation phase, each theme was explored in detail, supported by relevant quotes, to provide a comprehensive understanding of students’ varied perspectives on and interactions with concordancing in language learning.
Findings
The findings were organized into four topics corresponding to the research questions. The first topic addresses which tools students preferred to use for a sentence completion exercise, as related to the first research question. Table 4 shows the number of times each tool was chosen to answer exercise items in each category in the first exercise. Thus, for example, students chose to use a concordancer tool on a total of 50 occasions to answer subject/verb agreement exercise items. It can be seen that most students favored using Grammarly or QuillBot as their chosen tool for completing exercise items across all language usage categories. It is noteworthy that a considerable proportion of the students relied on their own knowledge to choose the correct responses, indicating a level of confidence in their English language skills. The concordancer tool emerged as the third most preferred tool, while Google Search and online dictionaries/textbooks were less favored as tools for aiding the students in selecting the correct form.
Table 4. Students Chosen Tools for the First Sentence Completion Exercise
| Question category | Concordancer Tool | Grammarly/ QuillBot | Google Search | Online Dictionary/ Textbook | Students’ own Knowledge | No Answer | Total |
| Subject-verb agreement | 50 | 133 | 25 | 5 | 81 | 1 | 295 |
| Tenses | 32 | 140 | 34 | 9 | 80 | 0 | 295 |
| Prepositions | 58 | 157 | 11 | 14 | 54 | 1 | 295 |
| Incorrect word usage | 33 | 154 | 17 | 10 | 80 | 1 | 295 |
| Total | 173 | 584 | 87 | 38 | 295 | 3 | 1180 |
| (%) | 14.7% | 49.5% | 7.4% | 3.2% | 25.0% | 0.3% | 100% |
The second topic focuses on how effectively each tool helped students find the correct forms in the first sentence completion exercise, corresponding to the second research question. Table 5 presents the proportion of the students’ accurate responses for each tool used, calculated as the number of correct answers divided by the total of items answered with each tool. This outcome shows the effectiveness of each tool in assisting the students to correctly answer the exercise items, representing an accuracy rate of the tools in each category. The results indicate that online Dictionary/Textbook and Grammarly/QuillBot proved to be the most effective tools, offering high accuracy rates and reliable assistance in identifying the correct form. Students’ own language knowledge ranked third – students’ confidence in their own knowledge was not always justified – while Google Search and the concordancer tool showed slightly lower accuracy rates in the exercise.
Table 5. The Accuracy Rates for All Tools Used for the First Sentence Completion Exercise
| Question Category | Concordancer Tool | Grammarly/ QuillBot | Google Search | Online Dictionary/ Textbook | Students’ Own Knowledge | No Answer | ||||||||||
| a | b | % AR | a | b | % AR | A | b | % AR | a | b | % AR | a | b | % AR | ||
| Subject-verb Agreement | 50 | 38 | 78 | 133 | 123 | 95 | 25 | 19 | 86 | 5 | 5 | 100 | 81 | 68 | 86 | 1 |
| Tenses | 32 | 24 | 83 | 140 | 124 | 99 | 34 | 30 | 88 | 9 | 9 | 100 | 80 | 69 | 85 | 0 |
| Prepositions | 58 | 41 | 69 | 157 | 149 | 100 | 11 | 11 | 86 | 14 | 10 | 91 | 54 | 40 | 75 | 1 |
| Correct word usage | 33 | 15 | 45 | 154 | 138 | 90 | 17 | 11 | 65 | 10 | 10 | 100 | 80 | 68 | 86 | 1 |
| Total | 173 | 118 | 68.2 | 584 | 534 | 91.4 | 87 | 71 | 81.6 | 38 | 34 | 89.5 | 295 | 245 | 83.1 | 3 |
a = Total number of items that the students answered after applying the tool; b = The number of correct responses out of all the items the students answered after applying the tool; AR = Accuracy rate (measured as the percentage of correct answers achieved by the students when applying the tool)
The third finding addresses the third research question, which examines how effective each tool is in helping students consistently repeat correct answers from their first exercise to their second exercise. This consistency is referred to as the retention rate. Table 6 displays the retention rates for each tool. Only students who gave correct answers in the first exercise were included in this calculation. The retention rate is then calculated by dividing the number of students who gave correct answers in the second exercise (without the use of a tool) by the number of students who gave correct answers in the first exercise. The results give an indication of the effectiveness of each tool across different language categories in helping the students retain the knowledge gained in the first instance.
Table 6. Students’ Retention Rates after Using Each Tool
| Question category | Concordancer Tool | Grammarly/ QuillBot | Google Search | Online Dictionary/ Textbook | Students’ Own Knowledge | ||||||||||
| b1 | b2 | % RR | b1 | b2 | % RR | b1 | b2 | % RR | b1 | b2 | % RR | b1 | b2 | % RR | |
| Subject-verb agreement | 38 | 22 | 57.9 | 123 | 55 | 44.7 | 19 | 9 | 47.4 | 5 | 3 | 60.0 | 68 | 44 | 64.7 |
| Tenses | 24 | 23 | 95.8 | 124 | 77 | 62.1 | 30 | 20 | 66.7 | 9 | 5 | 55.6 | 69 | 56 | 81.2 |
| Prepositions | 41 | 39 | 95.1 | 149 | 71 | 47.7 | 11 | 6 | 54.5 | 10 | 8 | 80.0 | 40 | 26 | 65.0 |
| Correct word usage | 15 | 15 | 100 | 138 | 76 | 55.1 | 11 | 9 | 81.8 | 10 | 9 | 90.0 | 68 | 47 | 69.1 |
| Total | 118 | 99 | 83.9 | 534 | 279 | 52.2 | 71 | 44 | 62.0 | 34 | 25 | 73.5 | 245 | 173 | 70.6 |
b1 = The number of accurate responses out of all the items the students answered using the tool in the first activity; b2 = The number of repeated correct responses out of all the items answered by the same students in the second activity (without applying any tools); RR = Retention rates (the percentage of repeated correct answers in the second test without using the tools)
Table 7 revealed that among the various methods tested, the concordancer tool had the highest retention rate (83.9%). This indicates that this tool is particularly beneficial for aiding the students in retaining correct language forms over an extended period. On the other hand, Grammarly and QuillBot showed the lowest retention rates (52.2%), suggesting that although these tools may provide immediate accuracy, they do not substantially contribute to the students’ long-term retention of correct answers. It is noteworthy that the students who relied solely on their existing knowledge during the first test retained 70.6% of their correct answers in the second test, indicating that much of their dependence on their own language knowledge was justified and that they did know the answers to the questions without resorting to the use of a tool. However, the 29.4% of correct answers not retained suggests that some correct answers may have been guesses or based on a weaker grasp of certain concepts. If the language knowledge had been fully internalized, retention would likely have been closer to 100%.
Additionally, the level of retention varied depending on the specific language aspect being addressed by the tools. For instance, the concordancer tool was highly effective in helping the students retain correct word usage, with a retention rate of 100%. However, its success was notably lower in aiding the retention of subject-verb agreement, with only 57.9% retention. This suggests that while the concordancer tool is a powerful tool for some aspects of language learning, its impact may be less pronounced for others, depending on the linguistic challenge
We conducted a series of paired sample t-tests to analyze the differences among the overall retention rates of three tools: the concordancer tool, Grammarly/QuillBot, and Google Search. The online dictionary and students’ own prior knowledge were excluded from this analysis. The online dictionary was rarely used—only 38 times overall—making its impact too limited for meaningful analysis. Students’ prior knowledge was excluded as it is not a tool in the conventional sense.
The analysis, therefore, focused on three paired comparisons: Concordancing vs. Grammarly/QuillBot, Concordancing vs. Google Search, and Grammarly/QuillBot vs. Google Search, using the scores for the sub-tests as reported in Table 5 as the data for the comparisons. Table 6 shows that the concordancer tool significantly outperformed both Grammarly/QuillBot (p = 0.021) and Google Search (p = 0.033) in helping the students retain correct answers across exercises. The difference between Grammarly/QuillBot and Google Search was not statistically significant at the p< .05 level. Thus, the data suggest that the concordancer tool was more effective for retention than the automated tools.
Table 7. Paired Samples t-Tests Comparing Three Tools
| Paired Variables | Paired Differences | t | df | Sig. (2-tailed) | |||||
| N | Mean | Std. Deviation | Std. Error Mean | 95% Confidence Interval | |||||
| Lower | Upper | ||||||||
| Concordancer – QuillBot | 4 | 34.8 | 15.584 | 7.792 | 10.003 | 59.5969 | 4.466 | 3 | .021 |
| Concordancer – Google | 4 | 24.6 | 13.115 | 6.558 | 3.731 | 45.4691 | 3.751 | 3 | .033 |
| QuillBot – Google | 4 | -10.2 | 11.127 | 5.563 | -27.905 | 7.5053 | -1.833 | 3 | .164 |
The open-ended questionnaire delivered additional perspectives that enhanced the quantitative findings presented above, revolving around three main themes: the students’ preferred tools for help with language exercises, the students’ perceptions of the concordancer tool, and the students’ preferences for future use of language learning tools. In the first question, as illustrated in Table 8, the students were asked about their preferred tools for completing the initial sentence exercise. They were allowed to select multiple tools, as it was recognized that a single student might favor more than one tool for this task.
Table 8. Students’ Preferred Tools for the First Sentence Completion Exercise
| No | Questions | Students’ Responses | Number of Responses | Percentage |
| 1. | Which tool did you prefer for answering the questions? Kindly share your rationale | a. QuillBot | 32 | 37.6% |
| b. Grammarly | 26 | 30.6% | ||
| c. Concordancer tool | 17 | 20.0% | ||
| d. Google search | 8 | 9.4% | ||
| e. Online dictionary | 2 | 2.4% | ||
| Total no. of responses | 85* | 100 % | ||
| 2. | What motivated you to utilize the concordancer tool for answering the questions? | Easy to use | 7 | 33.3% |
| Easy to find the requested words when you already know the appropriate keywords | 7 | 33.3% | ||
| Develops critical thinking | 6 | 28.6% | ||
| Full-featured | 1 | 4.8% | ||
| Total no. of responses | 21 |
*Some students provided more than one response.
Table 8 highlights a clear preference for Grammarly/ QuillBot. Students expressed their preference for Grammarly/ QuillBot as in Excerpts 1 and 2:
Excerpt 1:
“I prefer Grammarly because it highlights errors and offers suggestions for improvement.”
Excerpt 2:
“I like using QuillBot better than the concordancer tool because it gives me more direct corrections, making it easy to identify the correct answer automatically.”
From these excerpts, we can infer that the students prefer tools like Grammarly and QuillBot because they provide direct, automated feedback, making it easier to identify and correct errors without much effort.
In examining why 17 students preferred using the concordancer tool for the sentence completion exercise, we found that they regarded it as easy to use. In Excerpt 3, one student stated that the tool was easy to use as long as they could identify the right keywords. The ease of use in this context relates to their ability to efficiently retrieve relevant concordance lines by using the correct KWIC. Meanwhile, another student highlighted the technical benefits of the concordancer tool, noting its accessibility and speed, as reflected in Excerpt 4. Additionally, as expressed in Excerpt 5, students believed that the process of identifying keywords enhanced their critical thinking skills.
Excerpt 3:
“The concordancer tool is easy to use and I could find the examples in the tool easily when I could identify the correct keywords.”
Excerpt 4:
“The tool is easy to access and it is fast to retrieve the concordance lines. Identifying the correct concordance lines is easier when we understand parts of speech and apply the filter menu effectively.”
Excerpt 5:
“I have to think more deeply to identify the keywords, but I like it because I think it can improve my critical thinking skills.”
Table 9 reveals the students’ perspectives on the concordancer tool, with 55.9% of responses indicating the tool was challenging, 23.7% finding it easy, and 8.5% considering it quite easy. The main difficulties the students faced included identifying the right keywords (31.7%) and confusion in locating answers (19%). However, 23.8% noted that the concordancer tool helped develop their critical thinking. Interestingly, while 23.8% appreciated the detailed and comprehensive answers provided by the concordancer tool, 19% still found the process too complicated and confusing.
Table 9. Students’ Perceptions of the Concordancer Tool
| No | Questions | Students’ Responses | Number of Responses | Percentage |
| 3. | How did you perceive the use of the concordancer tool in addressing the questions? (Was it easy or challenging?) | Difficult to find the answer | 33 | 55.9% |
| Easy | 14 | 23.7% | ||
| Quite easy | 5 | 8.5% | ||
| Sometimes easy, sometimes difficult. | 4 | 6.8% | ||
| I did not use it | 3 | 5.1% | ||
| Total no. of responses | 59 | |||
| 4. | According to your perspective, what were the strengths of the concordancer tool in comparison to others? | Provides more detailed and comprehensive answers | 15 | 23.8% |
| Develops critical thinking | 15 | 23.8% | ||
| Easy to use | 10 | 15.9% | ||
| Has complete features | 8 | 12.7% | ||
| Assists in searching for correct grammar usage. | 7 | 11.1% | ||
| It is free | 2 | 3.17% | ||
| No idea | 6 | 9.5% | ||
| Total no. of responses | 63* | |||
| 5. | From your viewpoint, what were the limitations of the concordancer tool compared to the other tools? | Difficult to determine keywords | 20 | 31.7% |
| The usage is too complicated. | 12 | 19.0% | ||
| Confusion in finding answers. | 12 | 19.0% | ||
| Too many examples. | 5 | 7.9% | ||
| Time-consuming. | 5 | 7.9% | ||
| Unable to find answers when using specific keywords. | 3 | 4.8% | ||
| Error occurs | 2 | 3.2% | ||
| Not really familiar with the tool | 2 | 3.2% | ||
| No idea | 2 | 3.2% | ||
| Total no. of responses | 63* |
*Some students provided more than one response.
Two students shared their difficulties with using the concordancer tool, as illustrated in Excerpts 6 and 7.
Excerpt 6:
“Finding the keyword is very hard and sometimes I guess the wrong keyword.”
Excerpt 7:
“Using the concordancer tool takes longer than Grammarly/QuillBot and it’s confusing to see so many examples in the tool.”
On the other hand, some of the responses revealed that the concordancer tool was easy to use when the students could identify the most appropriate KWIC, as one student explained in Excerpt 8:
Excerpt 8:
“Once I have identified the KWIC, it is easy to find the examples in the corpus.”
We also asked about the benefits and drawbacks of the concordancer tool. The findings showed that most responses highlighted that the tool offered more detailed and comprehensive answers, enabling the students to see a variety of examples from the corpus in diverse contexts. For example, one student noted in Excerpt 9:
Excerpt 9:
“Using the concordancer tool enhances my knowledge about the use of the keyword in various sentences because it gives me a lot of examples.”
Another advantage of the concordancer tool was that it encouraged the students to think critically by analyzing numerous examples to derive the correct answers, as illustrated in Excerpt 10:
Excerpt 10:
“Even though the concordancer tool gives a lot of examples that sometimes make me confused, it helps me develop my critical thinking because it forces me to analyze the examples and think more deeply.”
The concordancer tool occasionally failed to generate concordance lines for certain KWIC entries because those terms were not in the corpus, making it difficult for students to explore language patterns. In Excerpt 11, a student expressed frustration over not finding expected examples, while Excerpt 12 highlighted how misidentifying the KWIC led to irrelevant results.
Excerpt 11:
“I do not understand why the concordancer tool sometimes fails to provide the examples I am looking for, even when I have entered the keyword.”
Excerpt 12:
“Sometimes I felt confused when searching for answers because I misidentified the KWIC, which caused the concordancer tool to provide incorrect examples.”
In conclusion, the students’ experiences with the concordancer tool were mixed. Some struggled with identifying the right keywords and found the tool time-consuming and confusing, while others found it easier to use when the correct keywords were known. The concordancer tool was particularly valuable for providing detailed examples and fostering critical thinking, though occasional frustration arose when relevant examples were missing. Despite these challenges, it offered significant opportunities for language learning and deeper analysis.
Even though most students found the Sketch Engine challenging, Table 10 shows that many are still eager to use it in the future. Their reasons include understanding word usage (33.93%), practicing TOEFL and grammar (28.57%), expanding vocabulary (16.07%), completing assignments (10.71%), confirming Grammarly results (3.57%), and fostering critical thinking (3.57%).
Table 10. Students’ Preferences for Future Language Learning Tools
| No | Questions | Students’ Responses | Total Responses | Percentage |
| 6. | Which tool do you plan to use in the future? | QuillBot | 35 | 34.31% |
| Grammarly | 26 | 25.49% | ||
| Concordancer tool | 24 | 23.53% | ||
| 10 | 9.80% | |||
| Online dictionary | 7 | 6.86% | ||
| Total no. of responses | 105* | |||
| 7. | Are you considering using the concordancer tool in the future? | Yes | 45 | 76.27% |
| Maybe | 11 | 18.64% | ||
| No | 3 | 5.08% | ||
| Total no. of responses | 59 | |||
| 8. | For what purpose do you anticipate using the concordancer tool in the future? | To understand the correct usage of words in sentences | 19 | 33.93% |
| To work on TOEFL and grammar | 16 | 28.57% | ||
| To expand vocabulary | 9 | 16.07% | ||
| To complete assignments | 6 | 10.71% | ||
| To confirm the accuracy of answers from Grammarly results | 4 | 6.78% | ||
| To develop critical thinking | 2 | 3.57% | ||
| No idea | 3 | 5.08% | ||
| Total no. of responses | 59 |
*Some students indicated more than one tool that they planned to use in the future.
Table 10 indicates that most respondents favored QuillBot and Grammarly, with the concordancer tool ranking just behind these tools as a preferred method for learning English in the future. Some students indicated that QuillBot and Grammarly were easier to use, as one student mentioned in Excerpt 13:
Excerpt 13:
“I think I will use QuillBot or Grammarly in the future because they are easy to use and they can correct my errors directly.”
Despite the preference for QuillBot and Grammarly, many students also expressed a willingness to use the concordancer tool for learning English, particularly for understanding vocabulary in various contexts. One student commented on Excerpt 14:
Excerpt 14:
“I will use the concordancer tool because it helps me understand that a single word can have different meanings in different contexts by observing many examples provided in the tool.”
In short, while Table 10 shows a clear preference for QuillBot and Grammarly due to their ease of use and direct error correction, many students are also open to using the concordancer tool for future English learning. The concordancer tool is valued for its ability to provide a variety of examples, which helps the students grasp the nuanced meanings of words in different contexts. This dual preference highlights the potential for integrating both types of tools to support diverse aspects of language learning.
Discussion
The use of a concordancer tool to help the students identify correct language forms and retain learned material offered both advantages and challenges. The findings indicate that, overall, the concordancer tool had the lowest accuracy rate when compared to other tools in addressing subject-verb agreement, tenses, prepositions, and incorrect word usage. The lower accuracy rates in this study contrast with the higher rates reported by Cheng (2021) and Crosthwaite (2017), who achieved greater success with concordancer tools. Cheng emphasized the benefits of repeated use of the concordancer tool across multiple essays, enabling students to become more proficient with the tool. In Crosthwaite’s study, students benefited from an ESL context where the students had a strong foundation in English grammar and vocabulary, and access to well-resourced computer labs, which facilitated effective tool usage.
In contrast, this study encountered several challenges that may have led to its lower accuracy rates, particularly the varying levels of English proficiency among the students. Those with higher proficiency demonstrated greater success in identifying the KWIC and retrieving relevant concordance lines to complete the sentence exercises effectively. Their advanced skills also enabled them to analyze and understand complex grammatical structures more efficiently. In contrast, students with lower proficiency levels struggled with these tasks. They found it difficult to correctly identify the KWIC, leading to irrelevant concordance line retrieval, and they had trouble analyzing the language patterns presented. This aligns with Lin’s (2021) findings that students’ performance in DDL is heavily influenced by their proficiency and familiarity with corpus analysis. Additionally, the limited exposure to the use of the concordancer tools may have hindered students’ ability to fully utilize the resources provided, as noted by Sun and Hu (2023), who highlighted the necessity of sufficient training to enhance accuracy and efficiency in corpus-based language learning. Furthermore, technological limitations also played a critical role in the students’ use of concordancing; issues such as unstable internet connections and restricted access to computers during exercises made it difficult for them to interact consistently and effectively with the tools.
Although the concordancer tool faced challenges in that it achieved the lowest accuracy rates when compared to other tools, it demonstrated superior retention rates. The tool enabled students to analyze language patterns within a corpus, which not only facilitated their understanding of grammatical structures but also helped them recall answers from previous exercises. Supporting these findings, studies by Al-mahbashi et al. (2015) and Hong (2010) also indicated that consulting the corpus through concordancing outperformed dictionary use in developing students’ receptive vocabulary, with the effects persisting over time, as evidenced in delayed post-tests. Furthermore, the high retention rates associated with the concordancer tool confirmed its effectiveness in helping students retain language more efficiently compared to AI-driven tools like Grammarly or QuillBot. These AI tools tend to offer immediate corrections without encouraging deep engagement with the rules of language. In contrast, concordancing promotes active discovery and exploration of linguistic patterns, which likely strengthens memory retention and fosters a deeper comprehension of the language. The increased cognitive effort required by this approach, as opposed to the passive nature of AI-generated feedback, seems to contribute to a more enduring mastery of the language.
The study highlighted a notable contrast between the students’ preferences for the concordancer tool and its actual effectiveness in language retention. Despite students favoring the tool for subject-verb agreement, it showed the lowest level of effectiveness in helping them retain this grammatical concept, which challenges Cheng’s (2021) findings. On the other hand, the tool proved more effective for prepositions, with students achieving better retention in this area. This difference may be due to the students’ linguistic background—Indonesian lacks grammatical subject-verb agreement, making it difficult for students to fully grasp and retain these English rules (Lin, 2021). In contrast, prepositions may align more closely with their native language, leading to better retention. These findings point to the need for additional instructional support when students face unfamiliar grammatical concepts, as emphasized by Sun and Hu (2023), highlighting the importance of targeted strategies to help students bridge gaps between their native language and English.
The open-ended questionnaire results align closely with the findings on accuracy and retention, offering further insights into the students’ preferences and perceptions of various language learning tools. The students who preferred Grammarly and QuillBot emphasized the ability of these tools to provide direct, automated feedback, which not only helped them improve their accuracy but also allowed for faster error correction, aligning with Barrot’s findings (2020). This preference for tools offering immediate, user-friendly assistance supports the earlier quantitative results, where Grammarly and QuillBot outperformed other tools in terms of improving sentence accuracy. These tools were particularly favored by the students seeking quick and straightforward solutions to language problems, highlighting their contribution to short-term retention of language rules and patterns, though potentially with less engagement in deeper learning processes. In contrast, although the concordancer tool was used by fewer students, those students recognized its ability to strengthen language retention through critical thinking and analytical skills. This observation is consistent with findings by Boulton and Cobb (2017), who found that concordancer tools supported lasting language retention by requiring the students to analyze language patterns independently. The open-ended responses revealed that students who used the concordancer tool believed it promoted deeper engagement by requiring them to identify and analyze language patterns using keyword searches. This deeper cognitive engagement aligns with the retention data, where students who actively engaged with the concordancer tool showed better long-term retention of language structures compared to those who relied on more automated tools like Grammarly or QuillBot.
Moreover, responses from the questionnaires revealed that some students encountered challenges when using the concordancer tool, such as difficulty in identifying the right keywords and confusion with multiple examples. These challenges reflect the findings from the earlier sentence completion exercise, which showed lower initial accuracy but higher retention over time. While the tool required more effort and time to use effectively, it encouraged the students to think critically and analyze multiple contexts, fostering a deeper understanding of language usage. The willingness of many students to continue using the concordancer tool in the future, despite these challenges, underscores its value for long-term language learning, particularly in developing nuanced vocabulary understanding and enhancing retention of complex language patterns. To help students address challenges with identifying keywords in concordancing, several strategies have been proposed by researchers. Lin (2021) and Sun and Hu (2020) emphasize that providing clear instructions and teacher guidance is crucial for creating a more structured and focused learning experience in DDL. Additionally, Yoon and Jo (2014) and Quinn (2015) recommend that students analyze printed concordance lines before accessing the corpus directly, as this can help them become familiar with KWIC identification. Dolgova and Mueller (2019) suggest that narrowing the focus to specific language patterns or a limited set of errors can enhance learning outcomes.
In conclusion, while concordancer tools effectively support long-term language retention, they tend to result in lower accuracy compared to AI tools like Grammarly and QuillBot. This study revealed that the students using the concordancer tool had better retention but struggled with accuracy, likely due to their lower English proficiency and the effort required for corpus analysis. Boulton (2012) suggests that combining concordancing with Google-driven learning could help students verify their findings using more familiar search tools. Similarly, Crosthwaite (2023) recommends integrating Generative AI with DDL to simplify complex corpus queries. In this approach, Generative AI simplifies the process of querying language patterns by replacing traditional concordancer tools. Instead of using complex corpus queries like KWIC, students can directly ask GenAI for examples, such as sentences containing the preposition “from.” This makes concordancing easier and more accessible, allowing the students to focus on language analysis without needing technical expertise in corpus analysis. By streamlining the process, GenAI helps students engage with language patterns more efficiently and effectively. Crosthwaite and Boulton (2023) argue that incorporating DDL principles into GenAI tools can make them more practical for students with limited corpus linguistics experience, ultimately enhancing both engagement and accuracy in language learning.
While this study offers insights into the effectiveness of the concordancer tool, it also presents several limitations that should be considered. Firstly, its focus on a specific student group may not fully represent all language learners. Broadening the range of participants studied would enhance the applicability of the findings. Secondly, the sample size was relatively small, and a larger sample could yield more robust outcomes. One significant constraint of this study about the evaluation of tool efficacy was the low number of students who used certain tools. For instance, very few students chose to access online dictionaries, and concordance use for tenses occurred only 29 times. All these limitations restrict the degree to which definitive conclusions regarding tool efficacy can be drawn from our study. Another limitation of the study was the use of binary choices in the sentence completion exercises, which made it easier for the students to resort to guessing. Some students, especially those who lacked the motivation to carefully analyze the questions, tended to choose an option at random rather than thoughtfully considering their answers. This guessing behavior could have impacted the accuracy of the results, as it did not necessarily indicate the students’ actual comprehension or retention. In future research, it would be beneficial to incorporate more answer options in the exercises to obtain a more accurate measure of the students’ understanding. By offering a wider range of choices, the likelihood of guessing is reduced, encouraging the students to engage more deeply with the material.
Another limitation worth addressing is that, while this study found that the concordancer tool appeared to support better language retention compared to other tools, this conclusion does not stem from a direct comparison involving the same students. Instead, each student had the freedom to choose from multiple tools during the sentence completion exercise. As a result, the findings may not fully account for individual differences in tool usage or the possibility that certain students were naturally more adept at retaining language, regardless of the tool they selected. A more controlled approach, where all students use each tool under the same conditions, could provide more robust evidence of the concordancer tool’s impact on language retention.
Moreover, the students involved in this study had no previous experience or familiarity with the concordancer tool before their participation. This lack of prior knowledge may have influenced their performance and understanding during the exercises. The students with more experience using the concordancer tool would likely have demonstrated different, potentially more favorable, results. Familiarity with the tool could lead to improved accuracy, better navigation of its features, and a deeper engagement with the learning process, suggesting that the outcomes observed in this study might differ with a more experienced group.
Conclusion
This study offers insights into the advantages and disadvantages of utilizing a concordancer tool to help students select accurate answers during sentence completion exercises and retain those answers as they progress from the first to the second exercise. The student feedback highlighted challenges in using the tool, such as difficulties in identifying KWIC, a lack of direct answers, an abundance of example sentences, and time-consuming processes. These challenges may have caused the students to prefer tools like Grammarly and QuillBot, which provide quicker and more efficient feedback on language usage and error correction. Concordancing, on the other hand, showed lower accuracy rates for language choices. This was due to factors such as the students’ inability to select the most appropriate KWIC, their lack of skill in analyzing concordance lines, and the limitations of the corpus, which sometimes lacked examples for the keywords they were searching for. Despite these challenges, concordancing demonstrated the highest retention rate among all the tools tested, indicating its potential to enhance long-term language retention. The students’ active involvement in the analysis process empowers the students in their language learning journey, enhancing their understanding of grammar rules and language usage for a longer period. To overcome the limitations of concordancing, teachers can employ five strategies: providing clear instructions and guidance, training students to analyze KWIC through printed concordance lines, focusing on specific language patterns, allowing students to use Google-driven learning to confirm their hypotheses and integrating GenAI-based DDL.
About the Authors
Humairah Fauziah earned her doctorate from Universitas Negeri Malang where she specialized in English language education. Currently, she serves as a lecturer at Universitas Trunojoyo Madura, where she actively engages in teaching and research. Her research focuses on Teaching English to Foreign Students (TEFL), integrating technology in language instruction, and improving English writing pedagogy. ORCID ID: 0000-0003-3524-9992
Yazid Basthomi is a Professor of Applied Linguistics at the Department of English, Faculty of Letters, Universitas Negeri Malang, Indonesia. During his doctoral career, he spent a stint of research, as a Fulbright scholar, at the ELI, University of Michigan. Having interests in genre analysis, intercultural education, and digital culture, he is currently the coordinator of the publication division of TEFLIN. ORCID ID: 0000-0003-3314-3334
Nurenzia Yannuar received her PhD from Leiden University Centre for Linguistics, the Netherlands. She was a post-doc fellow at KITLV/ Royal Netherlands Institute of Southeast Asian and Caribbean Studies, exploring the sociolinguistics of plurilingualism in colonial Indonesia through a novel. She currently teaches at Universitas Negeri Malang. Her research interests include youth languages, colloquial languages, and linguistics landscape. ORCID ID: 0000-0002-5974-6072
To Cite this Article
Fauziah, H., Basthomi, Y. & Yannuar, N. (2025). Concordancers vs. other tools: Comparing their roles in students’ English language retention. Teaching English as a Second Language Electronic Journal (TESL-EJ), 29(1). https://doi.org/10.55593/ej.29113a2
References
Ahsanuddin, M., Hanafi, Y., Basthomi, Y., Taufiqurrahman, F., Bukhori, H. A., Samodra, J., Widiati, U., & Wijayati, P. H. (2022). Building a corpus-based academic vocabulary list of four languages. Pegem Journal of Education and Instruction, 12(1), 159-167. https://doi.org/10.47750/pegegog.12.01.15
Al-mahbashi, A., Noorizah Mohd Noor, & Zaini Amir. (2015). The effect of data driven learning on receptive vocabulary knowledge of Yemeni University learners. 3L: The Southeast Asian Journal of English Language Studies, 21(3), 13–24. Retrieved from https://ejournal.ukm.my/3l/article/view/9511
Anthony, L. (2019). AntConc (Version 3.5.8) [Computer Software]. Tokyo, Japan: Waseda University. Retrieved from http://www.laurenceanthony.net/software
Barrot, J.S. (2020). Integrating technology into ESL/EFL writing through Grammarly. RELC Journal, 1-5. https://doi.org/10.1177/0033688220966632
Basthomi, Y., Yannuar, N., & Widiati, U. (2015). C-Smile, COCA, and BNC: A focus on amplifiers and adjective collocations. 3L: The Southeast Asian Journal of English Language Studies, 21(2), 73-87. https://doi.org/10.17576/3L-2015-2102-06
Bernardini, S. (2000). Systematising serendipity: Proposals for concordancing large corpora with language learners. In L. Burnard, T. McEnery (Eds.), Rethinking language pedagogy from a corpus perspective (pp. 225-234). Peter Lang.
Boulton, A. (2010). Data-driven learning: On paper, in practice. In T. Harris, & M. Jaén (Eds.), Corpus linguistics in language teaching (pp.17–52). Peter Lang.
Boulton, A. (2012). What data for data-driven learning? The EuroCALL Review, 20(1), 36. https://doi.org/10.4995/eurocall.2012.16038
Boulton, A., & Tyne, H. (2013). Corpus linguistics and data-driven learning: A critical overview. Bulletin Suisse de Linguistique Appliquée, 97, 97-118. https://hal.archives-ouvertes.fr/hal-01208263
Boulton, A., & Cobb, T. (2017). Corpus use in language learning: A meta-analysis. Language Learning, 67(2), 348-393. https://doi.org/10.1111/lang.12224
Boulton, A. (2019). Data-driven learning for younger learners: Obstacles and optimism. In P. Crosthwaite (Ed.), Data-driven learning for the next generation: Corpora and DDL for pre-tertiary learners (pp.14-20). Routledge.
Brezina, V., Gablasova, D., and Reichelt, S. (2018). BNClab. http://corpora.lancs.ac.uk/bnclab[electronic resource], Lancaster University.
Bridle, M. (2019). Learner use of a corpus as a reference tool in error correction: Factors influencing consultation and success. Journal of English for Academic Purposes, 37, 52-69. https://doi.org/10.1016/j.jeap.2018.11.003
Carter, R., & McCarthy, M. (1988). Vocabulary and language teaching. Longman
Chambers, A. (2007). Popularising corpus consultation by language learners and teachers. In E. Hidalgo, L. Quereda, & J. Santana (Eds.), Corpora in the foreign language classroom (pp. 3–16). Rodopi.
Chen, M., Flowerdew, J., & Anthony, L. (2019). Introducing in-service English language teachers to data-driven learning for academic writing. System, 87, 102148. https://doi.org/10.1016/j.system.2019.102148
Cheng, Y.H. (2021). EFL college students’ concordancing for error correction. English Teaching & Learning, 45, 431–460. https://doi.org/10.1007/s42321-021-00075-5
Cook, V. (2008). Second language learning and language teaching. Hodder Education.
Corino, E., & Onesti, C. (2019). Data-driven learning: A scaffolding methodology for CLIL and LSP teaching and learning. Frontiers in Education, 4(7), 1-12. https://doi.org/10.3389/feduc.2019.00007
Crosthwaite, P. (2017). Retesting the limits of data-driven learning: Feedback and error correction. Computer Assisted Language Learning, 30(6), 447–473. https://doi.org/10.1080/09588221.2017.1312462
Crosthwaite, P. (2019). Data-driven learning and younger learners: Introduction to the volume. In P. Crosthwaite (Ed.), Data-driven learning for the next generation: Corpora and DDL forpre-tertiary learners (pp. 1-10). Routledge. https://doi.org/10.4324/9780429425899-1
Crosthwaite, P., & Boulton, A. (2023). DDL is dead? Long live DDL! Expanding the boundaries of data-driven learning. In: P´erez-Paredes, P., Tyne, H. (Eds.), Discovering Language: Learning and Affordance in press. Retrieved from https://hal.univ-lorraine.fr/ATILF/hal-03506624v1
Daniel, B.K., & Harland, T. (2018). Higher education research methodology. A step-by-step guide to the research process. Taylor & Francis
Dolgova, N., & Mueller, C. (2019). How useful are corpus tools for error correction? Insights from learner data. Journal of English for Academic Purposes, 39, 97-108. https://doi.org/10.1016/j.jeap.2019.03.007
Fauzan, A., Basthomi, Y., & Ivone, F. M. (2022). Effects of using online corpus and online dictionary as data-driven learning on students’ grammar mastery. LEARN Journal: Language Education and Acquisition Research Network, 15(2), 679–704. Retrieved from https://so04.tci-thaijo.org/index.php/LEARN/article/view/259945
Godwin-Jones, R. (2017). Data-informed language learning. Language Learning & Technology, 21(3), 9–27. https://doi.org/10125/44629
Hafner, C. A.,& Candlin, C. N. (2007). Corpus tools as an affordance to learning in professional legal education. Journal of English for Academic Purposes, 6(4), 303-318. https://doi.org/10.1016/j.jeap.2007.09.005
Hong, S. C. (2010). EFL learners’ consciousness-raising through a corpus-based approach. English Teaching, 65(1), 57–86. https://doi.org/10.15858/engtea.65.1.201003.57
Johns, T. F. (1991). Should you be persuaded: Two examples of data-driven learning materials. English Language Research Journal, 4, 1-16. Retrieved from https://lexically.net/wordsmith/corpus_linguistics_links/Tim%20Johns%20and%20DDL.pdf
Kilgarriff, A., Marcowitz, F., Smith, S., & Thomas, J. (2015). Corpora and language learning with the Sketch Engine and SkELL. Revue Française de Linguistique Appliquée, 10(1), 61-80. https://doi.org/10.3917/rfla.201.0061
Kirana, D. P., Basthomi, Y., & Isnawati, U. M. (2018). A corpus-based study of vocabulary as input in EFL text-book: A case in an Indonesian Islamic college. Journal of Applied Linguistics and Language Research, 5(5), 93-104. Retrieved from https://www.jallr.com/index.php/JALLR/article/view/891/pdf891
Lee, H., Warschauer, M., & Lee, J. H. (2019). The effects of corpus use on second language vocabulary learning: A multilevel meta-analysis. Applied Linguistics, 40(5): 721-53. https://doi.org/10.1093/applin/amy012
Lee, J. H., Lee, H., & Sert, C. (2015). A corpus approach for autonomous teachers and learners: Implementing an on-line concordancing on teachers’ laptops. Language Learning & Technology, 19(2), 1–15. https://doi.org/10125/44411
Lin, M.H. (2021). Effects of data-driven learning on college students of different grammar proficiencies: A preliminary empirical assessment in EFL classes. SAGE Open, 11(3), 1-15. https://doi.org/10.1177/21582440211029936
Liou, H.C. (2019). Learner concordancing for EFL college writing accuracy. English Teaching & Learning, 43, 165–188. https://doi.org/10.1007/s42321-019-00022-5
Quinn, C. (2015). Training L2 writers to reference corpora as a self-correction tool. ELT Journal, 69(2), 165–177. https://doi.org/10.1093/elt/ccu062
Malik, A.R., Pratiwi, Y., Andajani, K., Numertayasa, I., Suharti, S., Darwis, A., & Marzuki. (2023). Axploring artificial intelligence in academic essay: Higher education student’s perspective. International Journal of Educational Research Open, 5(2),100296. https://doi.org/10.1016/j.ijedro.2023.100296
Scott, M. (2020). WordSmith tools version 8. Stroud: Lexical Analysis Software. https://www.lexically.net/publications/publications.htm#gsc.tab=0
Stevens, V. (1991). Classroom concordancing: Vocabulary materials derived from relevant authentic text. English for Specific Purposes, 10(1), 35–46. https://doi.org/10.1016/0889-4906(91)90014-N
Sun, X., & Hu, G. (2020). Direct and indirect data-driven learning: An experimental study of hedging in an EFL writing class. Language Teaching Research, 27(3), 1-29. https://doi.org/10.1177/1362168820954459
Yannuar, N., Shitadevi, I. A., Basthomi, Y., & Widiati, U. (2014). Active and passive voice constructions by Indonesian student writers. Theory and Practice in Language Studies, 4(7), 1400-1408. https://doi.org/10.4304/tpls.4.7.1400-1408
Yoon, H., & Hirvela, A. (2004). ESL student attitudes toward corpus use in L2 writing. Journal of Second Language Writing, 13(4), 257-283. https://doi.org/10.1016/j.jslw.2004.06.002
Yoon, H., & Jo, J. W. (2014). Direct and indirect access to corpora: An exploratory case study comparing students’ error correction and learning strategy use in L2 writing. Language Learning & Technology, 18(1), 96–117. https://doi.org/10125/44356
Appendix 1: Sentence Completion Exercise
In this exercise, students were directed to select one method for addressing each item. To indicate which method they used for each answer, they were required to mark the table beneath each method.
| No | Exercise Items | The Methods | ||||
| Concerdancer tool | Grammarly/ Quilbot | Google Search | Online dictionary | Your knowledge | ||
| 1 | Everyone (want/wants) to know the name of the new-born baby. | |||||
| 2 | People (like/likes) it when you listen to them and not only talk about yourself. | |||||
| 3 | The news (is/are) bad, I’m afraid. We won’t be coming. | |||||
| 4 | We want to know why so much of the money (is/are) missing. Where is it? | |||||
| 5 | Many cattle (was/were) killed by the leopards. | |||||
| 6 | You (ate/have eaten) three ice-creams since morning. | |||||
| 7 | His grandfather (died/had died) last month. | |||||
| 8 | The boy told the girl that he (likes/liked) her hairstyle. | |||||
| 9 | He says that he (wants/wanted) to be an astronaut. | |||||
| 10 | The mother asked the child (changing/to change) her wet clothes. | |||||
| 11 | The bank is close (to/with) the main train station. | |||||
| 12 | Are you good (in/at) remembering large quantities of information? | |||||
| 13 | You have to be crazy (with/about) a language to be able to learn it well. | |||||
| 14 | Do you ever dream (to/of) becoming a successful sports star? | |||||
| 15 | My opinion tends to be different (with/from) that of my wife. | |||||
| 16 | I want to (tell/talk) to my boss about my salary. | |||||
| 17 | She runs (fast/fastly) because the dog keeps chasing her. | |||||
| 18 | In the last few months, competition has become (tougher/more tough). | |||||
| 19 | He is a famous architect (who/whose) designs won an international award last year. | |||||
| 20 | Please be (quite/quiet)! The baby is sleeping. | |||||
Appendix 2: Post-course Questionnaire on Students’ Perception of Concordancing
We are interested in understanding your views on the concordancer tool in comparison to other correction tools you used. Kindly share your thoughts on the ease and challenges you experienced when employing the concordancer tool in the language exercise.
- Which tool did you prefer for answering the questions? Kindly share your rationale.
- What motivated you to utilize the concordancer tool for answering the questions?
- How did you perceive the use of the concordancer tool in addressing the questions? (Was it easy or challenging?)
- According to your perspective, what were the strengths of the concordancer tool in comparison to others?
- From your viewpoint, what were the limitations of the concordancer tool compared to the other tools?
- Which tool do you plan to use in the future?
- Are you considering using the concordancer tool in the future?
- For what purpose do you anticipate using the concordancer tool in the future?
| Copyright of articles rests with the authors. Please cite TESL-EJ appropriately. Editor’s Note: The HTML version contains no page numbers. Please use the PDF version of this article for citations. |