Vol. 3. No. 3 A-1 September 1998
Return to Table of Contents Return to Main Page

Data-collecting on Reading-writing Strategies: A Comparison of Instruments: A Case Study

Adina Levine and Thea Reves
Bar-Ilan University, Israel



The paper reports on an attempt to compare two data- collecting instruments, Think-aloud Protocols and Strategies Questionnaires, applied to capture reading/writing strategies.

The subjects were immigrant speakers of linguistically different heritage languages, Russian and Amharic (L1), and thus represented two ethnic groups, who went through the process of both acquiring the official language of their new homeland (L2 = Hebrew) and learning the language of international communication (FL = English). The two data-collecting instruments, Think-aloud Protocols and Strategies Questionnaires, were compared with regard to the way in which the subjects reflected on their reading/writing performance and were thus validated against each other. Background and Attitudes Questionnaires were administered to gather information on the subjects' linguistic, cultural and educational backgrounds.

The study addressed the following questions:

  1. To what extent do Think-aloud Protocols and Strategies Questionnaires provide similar or different information about reading and writing strategies?
  2. To what extent do specific subject-related factors (language proficiency, educational and cultural background, language attitudes and motivation,) account for the similarities and the differences between the information provided by the two instruments?

    The study was set up as a case study; by virtue of its ethnographic character it offers some interesting insights into the differential representation of individual strategies employed in the reading/writing processes. Its limited scope, however, allows only for tentative suggestions and not for generalizable conclusions.



The present study reports on an attempt to compare two instruments, Think-aloud Protocols and Strategies Questionnaires, which were applied to elicit information on the subjects' reading/writing strategies. The question emerged within the framework of a more comprehensive study dealing with the issues of reading/writing relationships in three languages, first language (L1), second language (L2) and foreign language (FL), across languages and across modalities.

Both reading and writing are complex cognitive activities requiring a set of processes and strategies. Strategies are a deliberate cognitive action (Carrell, 1991); when elicited for a conscious report, they can be of interest for what they reveal about the way readers process written texts and the way writers produce texts. Readers and writers who are aware of the strategies they use can also distinguish between strategies that are appropriate or inappropriate for specific reading/writing situations, and are thus able to monitor their reading and writing (Paris, Lipson & Wixson, 1983). This conscious knowledge and control of cognitive processes constitutes the reader's and writer's metacognition (Baker & Brown, 1984). The use and regulation of strategies can be viewed as a function of individual reader characteristics (Flavell, 1979).

In order to describe reading-writing processes as well as to obtain information on individual variation in the use of strategies, there seems to be obvious need to gain insight into the learners' thoughts, i.e., their cognition and metacognition. These involve thinking about the process, planning for it, monitoring it, and self-evaluating after the completion of the process.(O'Malley, 1985). To capture learners' cognitive and metacognitive strategies, different report data (interviews and questionnaires), as well as mentalistic data-collecting techniques (immediate retrospection reports and think-aloud protocols) were employed.

Mentalistic data-collecting techniques seem to enjoy rather wide support on the part of cognitive psychologists as well as language researchers (Ericsson & Simon, 1980; Cavalcanti, 1982; Cohen, 1986, 1987; Hayes & Flower, 1983; Poulisse, N. et al., 1986; & Smagorinsky, 1989), who use them to obtain direct evidence about processes which are otherwise inaccessible. The application of such instruments, has, however, raised concerns related mainly to two aspects:

  1. the subjects' ability to reflect on their cognitive behavior (Cavanaugh & Perlmutter, 1982) and
  2. the truth value of the reports (Nisbett & Wilson, 1977).


Regarding the first concern, the subjects might use strategies they fail to report, while regarding the second one, subjects may report using strategies they did not really employ. In other words, the issue that remains problematic is whether verbal reports are genuine descriptions of the actual processes the learners are involved in, or whether they are intelligent guesses based on the product. Additionally, they may be attempts to provide information that would satisfy the researcher. Thus, although verbal report data emerge as a useful research tool, their application seems to require certain caution (see also Garner, 1982). Therefore, while some researchers call for more studies on the use of think-aloud protocols as a method of capturing the learners' mental processes (Alderson 1984; Cohen, 1987), some others recommend the use of multiple research methods for identifying and validating language learning strategies (Oxford & Crookall, 1989).

The aim of the present case study was to compare two of the techniques commonly employed by researchers, namely Think-aloud Protocols and Strategies Questionnaires. This comparison was made in order to determine if, and to what extent, the information collected from one of them is corroborated by information elicited from the other, as well as to provide possible explanations for the similarities and/or differences between them. The questions were:

  1. To what extent do the Think-aloud Protocol and the Strategies Questionnaire provide similar or different information about reading and writing strategies?
  2. To what extent do specific subject-related factors (language proficiency, educational and cultural background, language attitudes and motivation) account for the similarities and the differences between the information provided by the two instruments?

The Study


The subjects of the case study were two educated professionals, a Russian and an Ethiopian immigrant, living in Israel for a period of about eighteen months. They were immigrant speakers of linguistically distant heritage languages--Russian and Amharic--and thus represented two different ethnic groups. Both went through the processes of both acquiring the official language of their new homeland--Hebrew--and using the language of wider communication-- English.

The subjects had received their high-school and part of their higher education in their L1s, in their countries of origin. In both cases, [-3-] instruction in English as a Foreign language (FL) in the L1 educational environment had concentrated on grammar-centered, form- oriented, controlled reading and writing, that is, no instruction in authentic communication in FL had been provided. However, while the Russian immigrant had not been exposed to the FL as a spoken language outside the school setting, the Ethiopian immigrant had had some exposure to English as a spoken language in his country of origin. They acquired their knowledge in Hebrew (L2) mainly informally, through contact with native speakers of Hebrew as well as through exposure to the Hebrew-language environment in their new home country. The instruction they received in Hebrew at the time of immigration was limited to oral "survival language," with very elementary reading-writing components.


The subjects' level of proficiency in FL and in L2 was established with the help of an adapted version of the Michigan Test of English Language Proficiency (MTELP) and by the Hebrew Ulpan Exit Test. [1] The following results were obtained: The Russian subject's (further referred to as Tina) proficiency in Hebrew is intermediate (67%) and in English, intermediate/advanced (78%). The Ethiopian subject's (further referred to as Anunu) Hebrew proficiency is intermediate/advanced (75%), while his English proficiency is advanced (90%).

In order to test the subjects' reading abilities in each of the three languages, L1 (Russian, Amharic), L2 (Hebrew) and FL (English), the following reading tasks were administered: the subjects were asked to read excerpts (about 250 words) from autobiographies of eminent personalities written in Russian or Amharic (L1), Hebrew (L2) and English (FL). Each text was followed by five multiple choice comprehension questions, testing the subject's understanding of the main point, supporting details, logical relationships, author's intention, and conclusion.

In order to test the subjects' writing abilities in each of the three languages, they were asked to write short essays in L1, L2 and FL. The topic areas for the essays were selected with the intent of being maximally close to the subject of reading. For example the reading assignment in Hebrew was an extract from the Autobiography of Golda Meir; the topic of the essay in Hebrew was "The Role of Women in Political Life." The evaluation of the written assignment was based on the Educational Testing Service's Test of Written English Scoring Guidelines [2].

The two subjects' performance in reading and writing in L1, L2, and FL showed noticeable differences across the three languages as well as across modalities. Tina scored 95% on her L1 reading assignment, and 80% on her L1 written composition. In L2, both her reading and writing score were 45%; the score on her FL reading was 65%, on her [-4-] FL writing, 75%. Anunu scored 100% on both his L1 reading and writing assignments. His L2 reading score was 75%, and his L2 writing score was 66%. He scored 90% on his FL reading and 80% on his FL writing.

The instruments employed to capture the subjects' reading/writing strategies were Think-aloud Protocols and Reading/Writing Strategies Questionnaires. The following categories of reading strategies served as the basis for constructing the Questionnaire as well as for analyzing the Think-aloud protocols: skimming, scanning, predicting, contextual guessing, exploiting text surface structure, making inferences, activating background knowledge (linguistic, content and formal schemata) (see Block, 1986; Carrell, 1989) .

The writing strategies pertaining to the writers' problem-solving and decision-making about focus, audience, language use and composing processes were categorized according to Cumming's (1990), "Coding Scheme for Student Think-aloud" as: gist, procedures, intention, language, discourse organization.

These categories were employed for both the construction of the Questionnaire and the analysis of Think-aloud protocols. The subjects were introduced to the Think-aloud procedure and were given an opportunity to practice it during an orientation session. The Think-aloud statements (comments) were audio-recorded, transcribed and submitted to analysis. In order to avoid a possible transfer effect, the subjects were asked to fill out the Strategies Questionnaires after the Think-aloud data were collected.

Data on the subjects' demographic background as well as information concerning their previous educational experiences, specifically with regard to language (reading-writing) instruction, ethno-cultural differences, and so on, were elicited with the help of Background Questionnaires. The Background Questionnaires provided the following information:

Tina was born and educated in Moscow, Russia. At the time of the study she was completing her bachelor of arts' studies in social work. She had learned English as a FL in school and at the university for ten years; she reads extensively in English (FL), yet very little in Hebrew (L2). She often writes personal letters in FL, but almost never in L2. In addition, she can read and write also in French.

Anunu was born and educated in Addis Ababa, Ethiopia. At the time of the study, he was studying towards his doctoral degree. He had learned English as a FL in high school and at the university for eight years. He reads and writes in English extensively, while his [-5-] reading and writing in Hebrew are limited to materials relevant to his studies.

Information about the subjects' attitudes and motivation was obtained with the help of Attitudes Questionnaires, which contains questions referring to the subjects' motivation in learning languages in general, their views about the country of immigration, its people, culture and education, and so forth. The subjects were allowed to do their Think-aloud as well as to answer questions on the Questionnaires in any language they chose (see Block, 1986; Devine, 1987). The evaluation and scoring of all the materials produced by the subjects were done by independent raters.

The Attitudes Questionnaires showed that Tina is proud of her native language and its cultural values. The English language represents for her the culture of the free world, as well as a language of high instrumental value. She considers the knowledge of Hebrew important for integration into the Israeli society, although in general, she does not think Hebrew is indispensable for her in her studies or elsewhere. Tina does not seem to be very enthusiastic about the social, cultural, and educational values of her new home country. Anunu expressed high esteem for his native language and culture. He values English as the language of international communication. He thinks Hebrew is a beautiful language, which has some resemblance to Amharic. He thinks highly of the family life traditions as well as of the opportunities offered to young people in his new home country, in comparison to his country of origin.

Data On Reading And Writing Strategies

The variables analyzed in the study are presented according to Subjects (Tina and Anunu); Languages and Modalities (L1 reading, L2 reading, FL reading; L1 writing, L2 writing, FL writing). The transcripts of the subjects Think-aloud Protocols and the Strategies Questionnaires revealed similarities as well as differences regarding the information on strategies applied to their reading and writing in each of the three languages (L1, L2, FL).

A. Data On Reading Strategies

L1 Reading

The transcripts of Tina's Think-aloud and her answers on the Strategies Questionnaire give evidence that in her L1 reading, she consciously tries to predict the content of the text as she reads. She frequently makes references to her background knowledge (content schemata) and looks for clues to meaning in the rhetorical structure of the text with the help of discourse markers (formal schemata). Anunu's Think-aloud Transcript and his answers on the Reading Strategies Questionnaire with regard to his reading in L1 corroborate each other in almost every aspect. [-6-]

Thus, he predicts the content of the text from the title. He formulates hypotheses about the content of the text with the help of his background knowledge (content schemata). He frequently refers to his general knowledge as well as to the information he possesses about the specific political situation in Ethiopia, making comparisons between the democracy in Israel and the totalitarian regime of his country of origin. In only one case is there a discrepancy between his statements on the Think-aloud protocols and on the Strategies Questionnaire. This discrepancy is related to L1 reading: his Think-aloud report shows that he tries to use the markers of rhetorical structure (formal schemata); in the answers on the Questionnaire, however, he claims not to resort to any discourse markers for logical connections.

L2 Reading

Tina's Think-aloud report during her L2 reading and her answers on the Strategies Questionnaire are contradictory.

Her Think-aloud report shows that she seems to be reading the text word by word. She tries to deduce the meaning of unknown words using her knowledge of Hebrew morphology (word-roots in Hebrew serve as the source of multiple derivatives). Tina is thereby using her linguistic schemata to a greater extent than her content schemata. She often resorts to translation into L1 or into FL, in which she feels more comfortable than in L2. In her answers on the Strategies Questionnaire, however, Tina states that she does not use the strategy of translation when she reads in a language that is not her mother tongue.

The strategies reported by Anunu during his reading in the L2 are the same as those indicated on the Strategies Questionnaire. He avoids reading word by word and relies on strategies on prediction and skimming. In his attempt to comprehend the text, he depends on his knowledge of the subject (content schemata).

FL Reading

With regard to Tina's FL reading, the analysis of her Think-aloud report reveals that she uses different strategies from the ones claimed in her Strategies Questionnaire.

Her Think-aloud report shows that she uses the strategy of prediction extensively: she formulates hypotheses with the help of the background knowledge related to the content of the text (content schemata). She makes frequent remarks as to the syntactic structure of the text (linguistic schemata) and she also relates to text organization (formal schemata). Differently from Tina's Think-aloud report, her answers on the Strategies Questionnaire make no reference to such strategies as guessing from context or using background knowledge to predict text content. [-7-]

Tina states that when reading in the FL, she tries to understand almost every word and therefore makes extensive use of the dictionary.

Regarding Anunu's FL reading, both his Think-aloud transcript and his Strategies Questionnaire show that he uses the same strategies: prediction, skimming for main ideas, and scanning for specific information. He looks up only the most important key-words in the dictionary, and reads every word only when he wants to achieve close comprehension. He does not translate word for word. He comments on the Questionnaire that translation of the words would interfere with the overall comprehension of the text. He does, however, sometimes check the meaning conveyed by the text against his own knowledge of the subject (content schemata).

Thus, there are substantial differences between the two subjects in the extent of corroboration between their Think-aloud and Questionnaire reports on Reading Strategies.

B. Data On Writing Strategies.

L1 Writing

Tina's statements on her L1 writing Strategies Questionnaire confirm her Think-aloud report. The latter shows that during her L1 writing, she plans her writing carefully in order to be able to organize her composition. She seems to be conscious of the rhetorical organization of the text, of the correct use of discourse markers, and of the length of her sentences. She considers her audience in her composition, and is aware of the stylistic factors involved in writing. She acknowledges the importance of examples and is conscious of their proportion in the text. Tina jots down and says aloud words and expressions she intends to use. She makes a special effort to interpret the meaning of the prompting title correctly, analyzing the intentions involved; she repeats the key words in the title several times, trying to determine their reference ("intention"). She relates the topic to her background knowledge about the political situation in her country of origin. She often says aloud what she intends to highlight ("gist"). She tries to justify her statements by repeating them aloud ("procedures"). She hesitates about how to formulate her ideas, reconsiders the style she will use in writing ("procedures"). Tina refers to the potential reader of her composition, wondering if she is doing what she is expected to do ("intention"). She chooses her expressions very carefully ("language"), occasionally crosses out words and looks for substitutes.

All of these strategies are corroborated by Tina's statements on the Strategies Questionnaire: Tina mentions the importance of planning and of considering the potential reader. She emphasizes the need of proportion between general statements and supporting details. She [-8-] also thinks that text editing plays an important role in preparing the final draft of the composition.

Anunu's Think-aloud Protocols of L1 writing and his answers on the Writing Strategies Questionnaire offer the same information. The Think-aloud report shows that he plans his writing carefully, and tries to word his opening statement so that it would express his main point ("gist"). He thinks about his potential reader ("intentions") and makes stylistic changes accordingly ("procedures"; "language use"). He summarizes orally what he has written in order to be sure he has expressed his message clearly ("gist"). He contemplates a number of details and settles for those that would illustrate the main point better ("procedures"); Anunu often monitors his writing with editing remarks ("procedures").

In his answers on the Questionnaire, Anunu also states that he usually plans his writing, depending mainly on the subject of the writing task. He considers the rhetorical structure of his writing as part of the planning phase. He considers the potential readers of his writing, and consciously uses discourse markers in order to highlight important ideas. He also illustrates his statements with examples. Anunu claims that he usually states the purpose of his writing in the opening sentences. He repeatedly acknowledges the importance of editing and summarizing.

L2 Writing

Tina's Think-aloud during her L2 writing indicates that only part of it is confirmed by her answers on the Writing Strategies Questionnaire. Her Think-aloud report shows that she seems not to be able to make valid statements about what she is doing, she is too absorbed in the act of writing itself. Only her planning, that is, the thinking aloud before she starts writing, reveals some self- observation ("gist"), while during the rest of her writing activity she is trying hard to find appropriate words and hesitates over their spelling, and is even worried about the mechanics of the Hebrew script ("language"). In her Think-aloud she goes back and forth from L2 to L1, and all her hesitations, while writing, refer to the most elementary mechanics of writing ("language").

In her answers on the Strategies Questionnaire, however, Tina claims that in writing in the L2 she always plans her writing very carefully. She states that she carefully considers the words she is going to use and notes that she likes to edit what she has written, mostly for grammar and style.

With regard to Anunu's L2 writing, there is a significant discrepancy between his Think-aloud protocol and his answers on the Strategies Questionnaire. His thinking aloud is done neither in the language of his writing, Hebrew (L2), nor in Amharic (L1), but in English (FL). He sometimes does, however, jot down some statements [-9-]

in L1 which he later translates into L2 ("procedures"). He does not refer to general knowledge or to the stylistic problems involved. He searches for the right words ("language use") and hardly ever consi- ders using discourse markers to emphasize his point ("procedures").

In his answers on the Questionnaire, however, Anunu claims that he thinks only in L2 when writing in L2. He does not see any difference between the way he writes (i.e., the strategies he uses) in L1 or in L2.

FL Writing

Tina's thinking aloud during her FL writing and her answers on the Strategies Questionnaire reveal a rather sophisticated process. There are, however, clear differences between the information obtained from the two instruments. Tina's Think-aloud transcripts reveal that during her FL writing, all her thinking aloud is done in L1. Her intentions are very clearly stated, she makes frequent references to the potential reader of her composition ("intention"), all done in L1. She hesitates in L1 as to the words she intends to choose in FL, looking for clearer and richer expressions ("language use"). She meditates in L1 over her choice of the discourse markers she will use in FL, in order to get a more logical rhetorical structure and thus highlight the gist of her writing ("discourse organization").

However, referring to her FL writing in her answers on the Strategies Questionnaire, Tina claims she does not resort to the L1 when writing in the FL. She says she plans her writing in the language in which she has to write.

Both Anunu's thinking aloud at the time of writing in FL and his answers on the Strategies Questionnaire make reference to similar strategies. The Think-aloud reports show that he thinks in the FL when writing in the FL; he addresses the main topic in the introductory part of his composition ("gist") .He makes statements referring to the connections between paragraphs ("discourse organization"); he wonders how his readers will react to the statements he makes ("intentions"); he checks his statement against the prompt as well as against what he has already written ("procedures"). Anunu hesitates over the words he is going to use trying to express himself more clearly ("language use").

Similarly, according to the answers on the Strategies Questionnaire, Anunu does not resort to his L1 when writing in the FL. He plans his writing carefully, focuses on the main topic, provides reasons and arguments to support his point. He claims he finishes his writing with a clear, conclusive statement and uses discourse markers to attract the reader's attention. Although he does not write draft copies, he considers editing an important step in writing. [-10-]

Thus, the comparison of writing strategies of both subjects, as reflected on the Strategies Questionnaires and the Think-aloud Transcripts, shows that the two instruments do not always verify each other.


The findings of the case study reported here allow us to answer the research questions by making the following tentative suggestions:

  1. The two instruments, Think-aloud Protocols and Strategies Questionnaires, reveal certain similarities as well as certain differences between the two ways of reporting about reading and writing strategies. Thus the similarities and differences found in the reports may be considered not only a function of the two different instruments but also a function of the type of strategy applied. The study demonstrated that strategies of more general character, such as reliance on background knowledge, editing the style of the written expression, etc., were similarly reported in both Think-aloud Protocols and Questionnaires. On the other hand, more specific reading/writing strategies, such as relying on L1, skimming for the main idea, using emphasizers, etc., are reported on differently in the two instruments.
  2. The observed similarities and differences between the report in the two data-collecting instruments may also be related to a number of specific subject-related factors:
    1. Language Proficiency
    2. Most of the similarities between the two sources were observed when the tasks had to be performed in a language in which the subject's proficiency was higher. A higher level of language proficiency may have made it easier for the subject to perform the task, since it required less mental effort on his/her part. The subject could thus direct his/her effort to the Think-aloud report. It was in this case that the expression of the strategies actually employed in the reading/writing tasks were similar to those indicated on the Questionnaire, which is a text-independent data-collecting instrument answered with conscious deliberation. A lower level of proficiency, on the other hand, may have made the task to be performed more difficult, and thus it required a greater mental effort on the part of the subject. As a result, the subjects' statements on the Think-aloud were different from their answers on the Questionnaire; moreover, the statements were shorter and the volume of reporting decreased.

      In other words, it seems that the easier the task of reading or writing is, the freer the subjects are to contemplate and report on their own mental activities simultaneously to their reading and writing. In this case, the strategies reported on the Think-aloud [-11-] Protocols and on the text-independent Strategies Questionnaire appear to be similar. When, however, the reading or writing task increases in difficulty, the subjects' thinking-aloud reports on strategy use become scarcer, and the differences between the reports in the two instruments increase. This finding is in line with the reports made by previous researchers. Thus, Olshavsky, (1978) noted a decline in strategy use relatively to the increase in text difficulty.

    3. Educational and Cultural Background
    4. Both our subjects were high-school graduates, with sophisticated literacy skills acquired in academic settings in their countries of origin. Such skills apparently include high-level language consciousness. This may be the reason that they were able to express the reading/writing strategies of which they were consciously aware (as evidenced in the text-independent Strategies Questionnaire) in the Think-aloud process. This was mostly noticeable in the subjects' performance in the FL, the language they learned via formal instruction: the differences found between the Strategies Questionnaires and the Think-aloud were smaller in the case of the FL tasks than in those of the L2 tasks.

      In Tina's (the Russian immigrant's) case, there is a large difference between the reports on the two instruments. Thus, on the one hand, Tina reports more strategies on the Questionnaire than she seems to be using according to the Think-aloud; on the other hand, she is constantly speculating in the Think-aloud whether she is doing what she is expected to do. She may be hesitant in filling out questionnaires or reporting on anything that could be interpreted as weakness. However, the desire to make a favorable impression may be very important to her.

    5. Attitudes and Motivation
    6. A well-known position found in the literature on language attitudes is that a higher appreciation for a language and the culture it represents entails a more positive motivation towards the tasks to be performed in that language (Gardner & Lambert, 1972). Accordingly, Anunu's enthusiasm for both L2 (Hebrew) and FL (English) probably entails a high motivation for reading and writing in those languages. This is revealed in the readiness to comment extensively on his activities during his thinking aloud. He also seems to be interested in self-observation per se, and this may be the reason for his detailed report. His Think-aloud Protocol is, in most cases, corroborated by his answers on the text-independent Strategies Questionnaire.

      On the other hand, Tina's thinking aloud on the Hebrew (L2) task allows us to speculate that when the subject does not feel attracted to the acquired language--either because the subject does not view [-12-] it as indispensable, or for some other aesthetic or integrative reason. (That is, when the subject's motivation is low, the self- expression may become forced, tongue-tied, not sufficiently revealing). The subject's Think-aloud statements will be different from the answers on the text-independent Questionnaire, which are responses to guiding questions and which relate to the processes of reading and writing in general independent of the activity itself.

    We should emphasize that in addition to the above factors, the discrepancies observed between the two data-collecting instruments may also be due to other variables, such as objective personality factors (age, sex) or personal idiosyncrasies. A case study obviously does not allow for a generalizable statement as to the relative contribution of such factors to the agreement and disagreement found between the two instruments.


The two data-collecting instruments used in our study complemented each other by providing partly similar and partly different data. The similarities and the differences observed between the two devices can mainly be ascribed to factors such as subject-related language proficiency, educational and cultural background, as well as attitudes and motivation.

The Think-aloud data consist of a stream of spontaneously provided descriptions. The subjects' comments on their activities are probably more authentic and less structured than their answers on a formal Questionnaire, which is by nature a guiding instrument. On the other hand, not every subject may be aware of the whole range of the strategic activities available, and thus may not make any reference to certain strategies while thinking aloud. An independently administered Strategies Questionnaire may serve as a source of more distanced, generalized self-observation: the subject may realize that she/he actually uses a certain strategy in the process of reading and writing.

Our findings suggest that in order to identify reading and writing strategies, there is an obvious need for the use of multiple data- collecting devices. To obtain more specific answers to questions regarding the agreement and disagreement between the two data- collecting devices (i.e., which one is more reliable), more systematic, empirical studies with a representative number of subjects would be required.


[1] Ulpan test: Hebrew Ulpan studies provide instruction in listening and reading skills as well as in elements of cohesive writing. The Hebrew Ulpan exit test consists of three parts (equally weighted): reading comprehension (300-400 word- [-13-] long text followed by comprehension questions), grammar (morphology and syntax ) and a short essay (300-400 words). The test is written by the Hebrew Ulpan staff at each university.

[2]. For more information on the Test of Written English, see http://www.toefl.org/abttwe.html


Alderson, J.C. (1984). Reading in a foreign language: A reading problem or a learning problem? In J.C.Alderson & H. Urquhart (Eds.), Reading in a foreign language (pp. 1-24). New York: Longman.

Baker, L. & Brown, A.L. (1984). Metacognitive skills and reading. In P.D. Pearson (Ed.), Handbook of Reading Research (pp. 353- 394). NY: Longman.

Block, E. (1986). The comprehension strategies of second language readers. TESOL Quarterly 20, pp. 463-494.

Carrell, P.L. (1989). Metacognitive awareness and second language reading. The Modern Language Journal 73, 121-134.

Carrell, P.L. (1991). Strategic reading. Unpublished manuscript.

Carrell, P. & U. Connor. (1991). Reading and writing descriptive and persuasive texts. The Modern Language Journal 75, 3, 314- 328.

Cavalcanti, M. (1982). Using the unorthodox, unreasonable verbal protocol technique: Qualitative data in foreign language reading research. In S. Dingwall, S. Mann & F. Katamba (Eds.), Methods and Problems in Doing Applied Linguistics Research(pp. 72-85). University of Lancaster,

Cavanaugh, J.C. & Perlmutter, M. (1982). Metamemory: A critical examination. Child Development 53, 11-28.

Cohen, A. (1986). Mentalistic measures in reading strategy research: Some recent findings. English for Specific Purposes 5, 131- 145.

Cohen, A. (1987). Studying learner strategies: How we get information? In A. Wenden & J. Rubin (Eds.), Learner strategies in language learning(pp. 31-39). Englewood Cliffs, NJ: Prentice-Hall.

Cumming, A. (1990). Expertise in evaluating second language compositions. Paper presented at the 24th Annual TESOL Convention, San Francisco. [-14-]

Devine, J. (1984). ESL readers' internalized models of the reading process. In J. Handscombe, R. Orem & B. Taylor (Eds.), On TESOL '83 (pp. 95-108). Washington, DC: TESOL.

Ericsson, K. & H. Simon. (1984). Protocol analysis: verbal reports as data. Cambridge, MA: MIT Press.

Flavell, J. (1979). Metacognition and cognitive monitoring: a new area of cognitive-developmental inquiry. American Psychologist 34, 109-118.

Gardner, R. & W. Lambert. (1972). Attitudes and motivation in second language learning. Newbury House Publishers.

Garner, R. (1982). Verbal report data on reading strategies. Journal of Reading Behavior, XIV (2), 159-167.

Hayes, J. & Flower, L. (1983). Uncovering cognitive processes in writing. In P. Mosenthal, L. Tamor, and S.A. Walmsley (Eds.), Research in Writing: Principles and Methods, New York: Longman, pp. 207-220.

Nisbett, R. & Wilson, T. (1977). Telling more than we can know: Verbal reports on mental processes. Psychological Review 84, 231-259.

Olshavsky, J. (1978). Comprehension profiles of good and poor readers across materials of increasing difficulty. In P. Pearson & J. Hansen (Eds.), Reading: disciplined inquiry in process and practice. Twenty-eighth Yearbook of the National Reading Conference(pp. 73-76), Clemson, SC.

O'Malley, J., Chamot, A., Stewner-Manzares, G., Kupper, L.& Russo, P. (1985). Learning strategies used by beginning and intermediate ESL students. Language Learning 35, 1, 21-46.

Oxford, R. & D. Crookall (1988). "Learning strategies". You can take it with you: helping students to maintain second language skills. Englewood Cliffs, NJ: Prentice-Hall.

Paris, S., Lipson, M., & Wixson, K. (1983). Becoming a strategic reader. Contemporary Educational Psychology 8, 293-316.

Poulisse, N., Bongaerts, T. & Kellerman, E. (1986). The use of retrospective verbal reports in the analysis of compensatory strategies. In C. Faerch & G. Kasper (Eds.). Introspection in second language research (pp. 213-229). Clevedon, U.K.: Multilingual Matters,

Smagorinsky, P. (1991). The reliability and validity of protocol analysis. Written Communication 6, 463-479. [-15-]

About the Authors

Adina Levine holds a Ph.D. in linguistics from the Hebrew University, Jerusalem. She is Director of the EFL Program at Bar- Ilan University and author and co-author of a number of articles on various aspects of English reading comprehension. She is co- author of two books on reading comprehension for university students published by Collier-Macmillan.

Thea Reves holds a Ph.D. from the Hebrew University, Jerusalem. She is senior Lecturer of English at Bar-Ilan University and Supervisor Emeritus of TEFL, Ministry of Education, Israel. Dr.Reves is the co-author of an oral proficiency test-battery introduced as the national school-leaving exam. She is author and co-author of numerous articles on various aspects of language education.

Return to Table of Contents Return to Top Return to Main Page
© Copyright rests with authors. Please cite TESL-EJ appropriately.

Editor's Note: Dashed numbers in square brackets indicate the end of each page in the paginated ASCII version of this article, which is the definitive edition. Please use these page numbers when citing this work.