Evaluation of Learning Objectives in Iranian High-School and Pre-University English Textbooks Using Bloom’s Taxonomy

March 2010 – Volume 13, Number 4

Evaluation of Learning Objectives in Iranian High-School and Pre-University English Textbooks Using Bloom’s Taxonomy

A. Mehdi Riazi
Macquarie University, Australia
Mehdi.riazimq.edu.au

Narjes Mosalanejad
Shiraz University, Iran
n_mosallayahoo.com

Abstract

This paper reports a study that investigated the types of learning objectives represented in Iranian senior high school and pre-university English textbooks using Bloom’s taxonomy of learning objectives. Three high-school textbooks and the sole pre-university textbook were included in the analysis. To codify the learning objectives, a coding scheme was developed based on Bloom’s (1956) taxonomy of learning objectives. The exercises and tasks of the textbooks were codified and the frequencies and percentages of occurrence of different learning objectives were calculated. Results of the study indicate that in all grades lower-order cognitive skills were more prevalent than higher-order ones. Furthermore, the difference between the senior high school and the pre-university textbooks in terms of the levels of the taxonomy were significant insofar as the pre-university textbook used some degrees of higher-order learning objectives. Results of this study have implications both for teaching and materials development.

Introduction

Materials development and evaluation is a relatively young phenomenon in the field of language teaching. In the practical sense, it includes the production, evaluation and adaptation of materials. Tomlinson (2001, p.66) defines materials as “anything which can be used to facilitate the learning of a language.”

Although textbooks can be affected by other aspects of an instructional situation (for instance by the teacher’s use of different approaches, by students’ preferences for one part over another, or other environmental factors), the effect of these instructional materials on the other components is more significant as texts have the potential to change the objectives of the situation or alter students’ motivation. Nevertheless, it seems that all the components involved in teaching and learning situations are in constant interaction to enhance learning achievement. As Hutchinson and Torres (1994 cited in Litz, 2005) suggest:

The textbook is an almost universal element of [English language] teaching. Millions of copies are sold every year, and numerous aid projects have been set up to produce them in [various] countries…No teaching-learning situation, it seems, is complete until it has its relevant textbook. (p. 315)

There are, however, different attitudes towards textbooks. Tomlinson (2001) divides the attitudes into two groups: proponents and opponents. The former group argues that textbooks are the most convenient form of presenting materials because they give consistency, systematicity, cohesion, continuation, and progression. The latter contend that textbooks are inevitably superficial and reductionist in their coverage and are not able to satisfy the diverse and broad needs of all their users. It is implied that the first group sees textbooks as useful for general purposes while the second group considers the shortcomings of textbooks for specific purposes. In spite of this disagreement it is widely agreed that textbooks are of great value in the process of teaching and learning (See, for example, Cunningsworth, 1995; Haycroft, 1998; O’Neil, 1982; Sheldon, 1988).

Constant evaluation of textbooks to see if they are appropriate is of great importance. As Genesee (2001) stated, evaluation in TESOL settings is a process of collecting, analyzing and interpreting information. This process enables us to make informed decisions through which student achievement will increase and educational programs will be more successful. According to Sheldon (1988), there are several reasons for the evaluation of textbooks. Among these reasons, he suggests that the selection of an English language teaching textbook often demonstrates an important administrative and educational decision in which one can see considerable amount of professional, financial, or even political investment. As there are many different and diverse ELT textbooks on the market, there is a necessity for the evaluation of textbooks in order to be able to recognize the advantages of one over the others, which in turn will lead to the adoption of the textbook. Ellis (1997) suggests that material evaluation could be conducted at three stages:

1) ‘predictive’ or ‘pre-use’ evaluation that is designed to examine the future or potential performance of a textbook;

2) ‘in-use’ evaluation designed to examine material that is currently being used; and

3) ‘retrospective’ or ‘post-use’ (reflective) evaluation of a textbook that is concerned with the evaluation of textbooks after they have been used in a specific institution or situation.

While different criteria and approaches have been presented to evaluate textbooks (see, for example, Cunningworth, 1995; Harmer, 1996; Williams, 1983), taxonomies like Bloom’s taxonomy of educational objectives also prove useful in textbook evaluation studies. Aviles (2000) believes that Bloom’s taxonomy of educational objectives is a tool that can be used in the wider context of education to help both new and experienced educators to think more precisely about what it means to teach and test for critical thinking. Sultana (2001) used the taxonomy to examine the lesson plans of 67 teacher interns in Kentucky to determine the extent to which their lesson objectives were designed to develop higher-order thinking skills in their students. This analysis showed that 41.3% of the new teachers’ lesson objectives were at the “knowledge” level, the lowest cognitive category. Only 3.2% of the teachers’ lesson objectives were found to be at the highest level of “evaluation” in Bloom’s taxonomy.

Literature Review

Most of the textbook evaluation studies carried out in Iran focus on three main goals. The first group has mostly tried to develop some criteria to contribute to more successful textbook evaluation studies (see e.g., Ansary & Babaii, 2002). The second group has evaluated certain textbooks for their strength and weaknesses to find their advantages and shortcomings (see e.g., Jahangard, 2007; Riazi & Aryashokouh, 2007), and the third group has studied discourse features and the representation of discourse elements in the textbooks (see e.g., Darali, 2007; Tavakoli, 1995). Other studies have addressed issues outside of these categories as well (see e.g., Marzban, 2005; Manafi, 2005).

Ansary and Babaii (2002) presented some characteristics and criteria for a good textbook based on a close scrutiny of a corpus of 10 EFL/ESL textbook reviews plus 10 EFL/ESL textbook evaluation checklists. The features they presented were:

Dissemination of a vision (theory or approach) about the nature of language,
The nature of learning and how the theory can be put to applied use;
Stating purpose(s) and objective(s) for the total course and for individual units;
Selection and its rationale for coverage, grading, organization and sequencing;
Teacher’s satisfaction with the syllabus for providing a guide book, advice on the methodology and explaining theoretical orientations, and keys to the exercises and supplementary materials;
Learner satisfaction with the syllabus for giving piecemeal, unit-by-unit instruction and clear instructions for exercises.

Jahangard (2007) evaluated four EFL textbooks that have been used in the Iranian high schools by the Ministry of Education. He discussed the merits and demerits of the textbooks with reference to 13 common criteria extracted from different materials evaluation checklists. The criteria were as follows: explicitness of objectives, good vocabulary explanation and practice, educationally and socially acceptable approaches to the target community, periodic review and test sections, clear attractive layout, print easy to read, appropriate visual materials, interesting topics and tasks, clear instructions, clearly organized and graded content, plenty of authentic language, good grammar presentation and practice, fluency practice in all four skills, and independent learning situations. The results of the study indicated that book four had better features in comparison with the three other textbooks (which needed huge revisions and modifications).

Riazi and Aryasholouh (2007) also studied the four high school and pre-university English textbooks focusing on the consciousness-raising aspect of vocabulary exercises. They found that of all exercises in the four books, only one percent of them could be categorized as consciousness-raising. They also found that the exercises mainly concentrated on individual words (approximately 26%) with no emphasis on fixed expressions, lexical collocations (approximately 15%) and grammatical collocations (approximately 2%). They concluded that students are mainly dealing with meanings of individual words and not with how words are used with other words or in what combinations.

The third group of textbook evaluation studies in Iran has focused on discourse features such as speech acts, intertextuality and so on. Darali (2007) studied the important features of new English textbooks such as the Spectrum series to see how cultural pragmatic knowledge of the language is included in the lessons. The results of the study showed that the series provided a variety of language functions, but the most frequent ones in daily speech were not focused on as much as other functions. Although the textbooks provided valuable metalinguistic information, they lacked explanations on the use of different forms in the same situation. There was also a paucity of explicit descriptions regarding appropriateness, paralinguistic information and contextual information.

In another study, Tavakoli (1995) used Searle’s (1976) model of speech act to analyze dialogues excerpted from three English textbooks, used in Iran at the high school senior level, to see whether different forms of speech acts were correctly used and how frequently each function was used. The researcher concluded that only three of the five language functions, that is, representative, directive, and expressive, were introduced in the textbooks, while commissives and declarations were completely ignored.

In other contexts, textbook evaluation studies have also attracted researchers’ attention. For instance, Morgan (2003) evaluated IELTS preparation materials and showed that there is a need for more materials that aim beyond test-taking practice and endeavor to develop the language competencies that the candidates need for their work or study goals. Morgan stated that in the books, IELTS candidates were expected to be highly motivated and therefore, there is not any attempt to make the books emotive as visually attractive books are. This was found to be a disadvantage of the books.

Kartner (2003) wonders why students and teachers’ enthusiasm towards a new textbook tends to get less and less by the end of the school year. The answer he provides is that textbooks that are at first interesting eventually get too familiar and unexciting. He admires course books that give the reader new ideas and perspectives and “gets your creative juices flowing”
(as he puts it).

Weiten, Deguara, Rehmke, and Sewell (1999) focused on textbook pedagogical aids while they paid attention to students as the main users of textbooks. They examined students’ evaluation of textbook pedagogical aids and found that boldface technical terms, running or chapter glossaries, chapter summaries and self-tests earned the highest marks in their evaluation.

Vellenga (2004) was concerned with how pragmatics was presented in EFL/ESL textbooks. She studied eight English as a Second Language (ESL) and English as a Foreign Language (EFL) textbooks to determine the amount and quality of pragmatic information included. She focused mainly on the use of metalanguage, explicit treatment of speech acts, and metapragmatic information, including discussion(s) of register, illocutionary force, politeness, appropriacy and usage, and found that textbooks included a lack of explicit metapragmatic information, and teachers’ manuals rarely supplemented adequately. The researcher also found that teachers rarely brought outside materials related to pragmatics into the classroom and concluded that learning pragmatics from textbooks would be highly unlikely.

As this review of the related literature indicates, there is a paucity of research on the cognitive domains in textbook evaluation studies. The current study, therefore, is felt to be needed as it focuses on the components of the cognitive domain of English textbooks currently in use in Iranian high-schools and pre-university.

Objectives of the Study

The purpose of this study was to evaluate Iranian high school English textbooks with regard to their aims as manifested by the content. The evaluation took place with regard to the six levels of learning objectives in Bloom’s taxonomy.

The study intended to investigate how the content of textbooks represents Bloom’s taxonomy of educational objectives. In particular, it sought to indicate which levels of the taxonomy were more focused on in the textbooks. While evaluating textbooks in terms of content and objectives, the researchers gave some suggestions for improving the content of the textbooks.

The study, therefore, aimed to find answers to the following questions:

Which levels of Bloom’s taxonomy are more prevalent in English textbooks currently in use in Iranian high school and pre-university textbooks?
How are the books evaluated in terms of lower-order and higher-order cognitive skills?
How could the learning objectives in high school and pre-university textbooks be compared?

Regarding the context of the study, it should be stated that the English language is taught two hours per week in high schools in Iran, and teachers can mostly teach through their preferred methodology as there is no specific teacher manual available to them. However, the fact is that teaching is highly affected by the well-known University Entrance Exam (Konkoor) and teachers feel responsible for preparing students for the big test, which is a high-stakes multiple-choice exam. The focus of the exam is mostly on grammatical points, memorization of new words in isolation, and reading through some passages and answering related questions.

Methods

This study was a textbook evaluation. Using a coding scheme, all parts of senior high school and pre-university English textbooks were coded in terms of learning objectives and the frequency of each learning objective was calculated for each level and also for the whole levels. In order to determine if there was a significant pattern in the occurrence of different levels of cognitive skills in the four textbooks, Chi-square tests were performed.

Materials

The English textbooks of three senior high schools and one pre-university used in the high schools of Iran were used as the materials to be evaluated using Bloom’s taxonomy. The list of the books is as follows:

Birjandy, P., Soheili, A., Noroozi, M., & Mahmoodi, Gh. (2006). English Book 1.Tehran: Textbook Publishing Company of Iran.
Birjandy, P., Noroozi, M., & Mahmoodi, Gh. (2005). English Book 2.Tehran: Textbook Publishing Company of Iran.
Birjandy, P., Noroozi, M., & Mahmoodi, Gh. (2007). English Book 3.Tehran: Textbook Publishing Company of Iran.
Birjandy, P., Ananisarab, M.R., & Samimi, D. (2005). Learning to Read English for Pre-University Students. Tehran: Textbook Publishing Company of Iran.

Data organization and analysis

Coding Scheme

A coding scheme for classifying and evaluating the content of textbooks using Bloom’s Taxonomy was developed. Bloom’s definitions of different levels of the cognitive domain were carefully studied and the key word examples were extracted and used. The coding scheme represented the six levels of learning objectives from the simple recall or recognition of facts, as the lowest level, through increasingly more complex and abstract mental levels of synthesis and evaluation.

The coding categories were labeled as: 1) knowledge 2) comprehension 3) application 4) analysis 5) synthesis 6) evaluation. Each coding category included examples for each level, key words that represented intellectual activity on each level and sample task rubrics.

The reliability of the coding scheme was examined through inter- and intra-coder reliability. Intra-coder reliability was 0.98 and the inter-coder reliability was 0.91 (See appendix 1 for a copy of the coding scheme).

Findings and Results

Overall Features of Senior High School and Pre-University English Textbooks

Table 1 includes the number of lessons in each of the textbooks; the average text length calculated for both senior high school and pre-university English textbooks; readability score for all the passages in the books; the number of parts which are included in each lesson with some explanation about each part, and the skills and the components the textbooks have paid attention to.

Table 1: Overall features of senior high school and pre-university English textbooks

High school	Number of lessons	Average Text Length (Number of Words)	Average readability	Number of Parts in Each Lesson	Skills and Components Attended to
1st year	9	204.6	85.4	9	Reading-Writing Speaking-Vocabulary Grammar-Pronunciation
2nd year	7	238	80.6	9	Reading-Writing Speaking-Vocabulary Grammar-Pronunciation
3rd year	6	273.1	62.8	9	Reading-Writing Speaking-Vocabulary Grammar-Pronunciation
Pre-university	8	567.28	54.3	7	Reading Grammar Vocabulary
Average	7.5	320.745	70.775	8.5

Table 1 indicates that text length ranges in a logical order so that the Grade 1 textbook has the lowest number of words while the pre-university textbook has the highest number. This is logical as learners’ level is being taken into account. Also, the easiness of the contents decreases from the first year English course book to the pre-university textbook. This again seems logical when we consider the proficiency level of learners at different levels.

Learning Objectives in High School Senior English Textbooks

The frequency and percentage of learning objectives in high school senior English textbooks are presented in Table 2. These results were obtained through the codification of the whole content of all four textbooks which included the exercises. The most frequent learning objectives were comprehension and application in the first year English textbook while the least frequent objective was evaluation, with the frequency of 0.6%. Knowledge, synthesis and analysis came in between. In the second year textbook, the most frequent objective was application (37.6%) while evaluation was totally absent in the coded data. Comprehension, analysis, knowledge and synthesis came in between in this range.

Table 2: Learning objectives in high school senior English textbooks

Learning Objectives

Knowledge

Comprehension

Application

Analysis

Synthesis

Evaluation

1st year

(Total:178)

(14%)

(32.6%)

(9.5%)

(10.7%)

(0.6%)

2nd year

(Total:149)

(12.8%)

(24.1%)

(37.6%)

(18.1%)

(7.4%)

(0%)

3rd year

(Total:115)

(12.2%)

(26.1%)

(33.9%)

(14.8%)

(13%)

(0%)

Average

19.3

(13%)

41.3

(27.6%)

(34.7%)

20.3

(14.15%)

(10.35%)

0.3

(0.2%)

As in the two previous textbooks, application was the most frequent learning objective in the third year English course book. Evaluation was absent in this textbook, too. Comprehension, analysis, synthesis and knowledge were placed in between in this continuum.

If we classify the six levels of Bloom’s taxonomy into “lower” and “higher” order cognitive skills, then we can restate the information in Table 2 as presented in Table 3 below.

Table 3: Lower- and higher-order cognitive skills in senior high school English textbooks

Learning Objectives

Lower-order

cognitive skills

Higher-order

cognitive skills

1st year

141

(79.2%)

(20.8%)

2nd year

111

(74.5%)

(25.5%)

3rd year

(72.2%)

(27.8%)

Average

111.6

(75.3%)

35.6

(24.7%)

As demonstrated in Table 3, lower-order cognitive skills are the most frequent cognitive skills according to the classification of learning objectives of Bloom’s taxonomy.

Learning Objectives in Pre-university English Textbook

The following table demonstrates the frequencies and percentages of the distribution of different levels of Bloom’s taxonomy of learning objectives in pre-university English textbook.

Table 4: Learning objectives in pre-university English textbooks

Learning Objectives

Knowledge

Comprehension

Application

Analysis

Synthesis

Evaluation

Total

Frequency &

Percentage

(23.3%)

(38.4%)

(11.3%)

(3.8%)

(13.8%)

(9.4%)

159

As shown in Table 4, the most frequent learning objective in pre-university English textbook was related to comprehension, while the least represented was related to analysis. Knowledge, synthesis, application and evaluation came in between on this continuum. Again if we classify the six cognitive levels into “lower” and “higher” order cognitive skills, the information in Table 4 can be restated in the following way as presented in Table 5.

Table 5: Lower- and higher-order cognitive skills in pre-university English textbook

Learning Objectives

Lower-order cognitive skills

Higher-order cognitive skills

Frequency &

Percentage

38.66

(73%)

14.33

(27%)

According to Table 5, attention in the pre-university English textbook is mostly focused on lower-order cognitive skills. This means that knowledge, comprehension and application are the most prevalent objectives in the pre-university textbook.

In order to see how high school and pre-university textbooks could be compared in terms of the six levels of Bloom’s taxonomy, a Chi-square was carried out which gave a significant difference (X² = 48.88, df= 5, p=.00) between the learning objectives in the four textbooks.

Discussion

With regard to text length and readability accounts, it was found that a logical order is followed in the four textbooks. This means that the textbooks in lower grades are shorter and easier, in line with learners’ levels of proficiency in their foreign language. As it was shown in Table 1, the difference between the average text length in senior high school textbooks and pre-university textbook is very great; which can be related to the changes this book has recently gone through and the recently included longer texts.

Regarding the cognitive levels in the textbooks, lower-order components were dominant in the high school textbooks. Among higher-order cognitive skills, evaluation is almost absent from the textbooks while attention to analysis and synthesis follows a random pattern so that while analysis increases from first year to second year textbooks, it again decreases from the second to the third year course books. Furthermore, while synthesis decreases from the first year to the second year course books, it increases from the second to the third year English course books. Therefore, a logical order cannot be assigned for these levels in the textbooks. As a whole though, application had the highest frequency while comprehension, analysis, knowledge, and synthesis followed it in turn. Evaluation had the lowest frequency on this continuum.

In the pre-university textbook, comprehension was also observed to be the most frequent objective while knowledge was the second. Attention to lower-order cognitive skills is given more than to higher-order cognitive skills in this textbook as well. It is important to note the frequency of occurrence of evaluation in pre-university English textbook. While it was absent from high school senior English textbooks, it occurred even more than analysis in the pre-university textbook (See Table 4). This shows that attention to higher-order cognitive skills is considerable in the pre-university textbook. Although lower-order cognitive skills are still more frequent in this textbook, the frequency of occurrence of the skill at the highest level of the taxonomy has increased considerably. An interesting point is that while one may expect the frequency of knowledge to decrease in pre-university textbook in comparison to the high school textbooks, in actuality, the reverse occurs, as the results of this study indicate. Furthermore, analysis decreases from the high school to the pre-university textbooks. This is what is to be expected; this higher-order cognitive skill appears more frequent in the highest grade than in the lower ones.

An overall conclusion is that lower-order cognitive skills were more frequent than higher-order cognitive skills. This could be a result of the fact that in the educational system of Iran, the major emphasis is on acquiring knowledge in the form of rote learning and memorization, rather than constructing it through higher-levels of cognitive skills such as analysis and synthesis. The university entrance examination exerts a real negative backwash effect on schools and students in that students are required to learn exactly what is included in the textbooks without any alterations on the part of the learners. That is, learners are to memorize the materials and reproduce them on the exam sheets. As such, students are not asked or given the opportunity to use the language. Sometimes, even the meaning of the texts is given to students in their native language.

Teachers try to help students attain the required skills to successfully perform on the university entrance exam (Konkoor) which is a high-stakes multiple choice test. The English section of the test is based on the high school and pre-university textbooks and can be answered just by memorization of the vocabulary and the structural points in the textbooks. There is no need to be able to use the language in a functional way (e.g., speaking or writing) for this very important exam. Students need to learn and practice how to manage their time, acquire test-taking skills, and perform well on the questions rather than to learn how to use the language.

Due to this fact students are not required to analyze, synthesize or evaluate the content of the textbooks. The teachers do not feel any need to go through these higher-order cognitive skills and they prefer to have time to teach their students the grammatical points they need for the entrance exam or to familiarize them with multiple choice questions. Textbook developers have also been affected by this phenomenon and have just worked in favor of this short term objective.

Another finding of interest was the significant difference between the frequency of occurrence of learning objectives in high school senior and pre-university textbooks. This could be related to the fact that the pre-university textbook has been revised recently and the writers have tried to use activities that use higher-order cognitive skills more. Although attempts to improve it have been made, and it is better than the senior high school textbooks, the pre-university textbook still includes a higher frequency of the lower-order cognitive skills than it does of the higher-order cognitive skills. This could be due to the lack of harmony between the high school and pre-university textbooks which can might motivate textbook developers to revise the high school textbooks to bring them into harmony and let the pre-university textbook use more higher than lower cognitive skills.

Conclusion

The overall findings of this study demonstrated that the most frequent learning objectives pursued in the high school and pre-university English textbooks in Iran were lower-order cognitive skills, that is, knowledge, comprehension, and application. There is even a lack of progression from the lowest (knowledge) to the highest (evaluation) cognitive levels as we move from Grade 1 to pre-university textbooks. Although the pre-university English textbook has been revised just recently and the authors have tried to include some higher-order learning objectives, it is far behind in the development of higher-order cognitive skills. Despite the fact that the writers occasionally have tried to change the exercises to address higher cognitive skills, the overt attention to the development of learners’ comprehension is vivid.

Among the six levels of Bloom’s Taxonomy of cognitive domain, application and comprehension were the most prevalent in the senior high school textbooks while in pre-university textbook, comprehension and knowledge were the most dominant. We can thus conclude that based on the results of this study, the main objectives of the textbooks were the development of lower-order cognitive skills.

In order to promote the content of the textbooks, some strategies can be proposed:

An appropriate plan should be organized in which the roles of textbook developers, teachers, students, and educational managers at a specific stage of learning and teaching are stated clearly.
In revising the textbooks, good qualities of the textbooks should be retained and the shortcomings should be eliminated or at least reduced.
The standards of teaching and testing should be revised according to what the research and different studies have suggested. In particular, the negative backwash effect of the university entrance exam should be reduced to the extent possible.
Textbook developers should try to devise exercises and activities that go beyond lower-order cognitive skills and to include higher-order ones.
Finally, materials development is by now an area of specialization with abundant literature; therefore, textbook writers are required to appreciate and use principles of materials development in the process of writing and revising books.

About the Authors

Abdol Mehdi Riazi began his career as an English language instructor at Shiraz University. He obtained his Ph.D. from the University of Toronto, Canada. In July 2009 he joined the Department of Linguistics at Macquarie University, Australia. He has authored and co-authored books on research, writing, and ESP courses; has papers in refereed journals; and has presented extensively at conferences. His research interests include academic writing, disciplinary literacy, language testing and assessment, materials development and evaluation and language learning strategies.

Narjess Mosallanejad got her BA in TEFL from Shahid Rajaee University, Tehran and her MA degree in TEFL from Shiraz University, Iran. She currently teaches English in high schools and English language institutes. She is interested in materials development, methodology, concepts, and linguistics.

References

Aviles, C. B. (2000). Teaching and testing for critical thinking with Bloom’s taxonomy of educational objectives. ERIC Document ED446023. Available: http://www.eric.ed.gov/ERICWebPortal/contentdelivery/servlet/ERICServlet?accno=ED446023.

Ansary, H., & Babaii, E. (2002). Universal characteristics of EFL/ESL textbook: A step towards systematic textbook evaluation. The Internet TESL Journal, 2, 1-8. Available: http://iteslj.org/Articles/Ansary-Textbooks/.

Bloom, B. S. (Ed.). (1956). Taxonomy of educational objectives handbook: Cognitive domain. New York: McKay.

Carter, R., & Nunan, D. (Eds.). (2001). Teaching English to speakers of other languages. Cambridge, UK: Cambridge University Press.

Cunningsworth, A. (1995). Choosing your coursebook. Oxford, UK: Heinemann.

Darali, G. (2007). Pragmatics dimension in Spectrum textbooks. Unpublished master’s thesis, Shiraz University, Iran.

Ellis, R. (1997). The empirical evaluation of language teaching materials. ELT Journal, 51(1), 36-42.

Genesee, F. (2001). Evaluation. In R. Carter, & D. Nunan (Eds.), The Cambridge guide to teaching English to speakers of other languages (pp. 144-150). Cambridge: Cambridge University Press.

Harmer, J. (1996). The practice of English language teaching. London: Longman.

Haycroft, J. (1998). An introduction To English language teaching. London: Longman.

Jahangard, A. (2007). The evaluation of the EFL materials taught at Iranian public high schools. Karen’s Linguistics Issues. Retrieved October 2008 from: http://www3.telus.net/linguisticsissues/bymonth.html.

Kartner, P. (2003). Humanizing your coursebook. ELT Journal, 57(4), 405-407.

Litz, D. R. A. (2005). Textbook evaluation and ELT management: a South Korean case study. Asian EFL Journal. Retrieved from: http://www.asian-efl-journal.com/Litz_thesis.pdf.

Manafi, A. (2005). Evaluation of the place of peace and war in Iranian guidance school textbooks content in 2005-2006. Unpublished master’s thesis. Shahid Beheshti University, Tehran, Iran.

Marzban, P. (2005). Evaluation of primary school textbooks content with regard to the views and perspectives in Islamic Republic of Iran cultural developmental programs. Unpublished master’s thesis. Alzahra University, Tehran, Iran.

Morgan, T. (2003). IELTS preparation materials. ELT Journal, 57(1), 66-76.

O’Neill, R. (1982). Why use textbooks? ELT Journal, 36(2), 104-111.

Riazi, A.M., & Aryashokouh, A. (2007). Lexis in English textbooks in Iran: Analysis of exercises and proposals for consciousness-raising activities. Pacific Association of Applied Linguists, 11, 17-34.

Searle, J.R. (1976). Speech acts: An essay in the philosophy of language. London: Cambridge University Press.

Sheldon, L. (1988). Evaluating ELT textbooks and materials. ELT Journal, 42(2), 237-246.

Sultana, Q. (2001). Scholarly teaching – Application of Bloom’s taxonomy in Kentucky’s Classrooms. Paper presented at the Third Annual Conference on Scholarship and Teaching, Bowling Green, KY. (ERIC Document Reproduction Service No. ED 471982.)

Tavakoli, F. (1995). Functional analysis of the dialogues in the Iranian senior high school English textbooks. Unpublished master’s thesis. Allameh University, Tehran.

Tomlinson, B. (2001). Materials development. In R. Carter, & D. Nunan (Eds.), The Cambridge guide to teaching English to speakers of other languages (pp. 66-71). Cambridge, UK: Cambridge University Press.

Vellenga, H. (2004). Learning pragmatics from ESL and EFL textbooks: How likely? TESL-EJ, 8(2). Retrieved September 2007 from: http://www.tesl-ej.org/wordpress/past-issues/volume8/ej30/ej30a3/

Weiten, W., Deguara, D., Rehmke, E., & Sewell, L. (1999). University, community college, and high school students’ evaluations of textbook pedagogical aids. Teaching of psychology, 26(1), 19-21.

Williams, D. (1983). Developing criteria for textbook evaluation. ELT Journal, 37(3), 251-261.

Editor’s Note: The HTML version contains no page numbers. Please use the PDF version of this article for citations.

Appendix 1: Coding scheme based on Bloom’s Taxonomy of cognitive domain

Level	Definition	Key Words and Examples	Sample Task Rubrics
Knowledge	Recalling data or information	Key Words: define, describe, identify, label, list, match, name, outline, reproduce, select, state. Examples: The student recalls and/or quotes information from memory to the teacher.	Fill in the blanks with appropriate words
Comprehension	Understanding the meaning, translation, and interpretation of instructions and problems; Stating a problem in one’s own words	Key Words: describe, estimate, explain, extend, generalize, infer, interpret, paraphrase, predict, rewrite, summarize, translate. Examples: The student translates, comprehends, or interprets information s/he has received.	Answer the questions according to the reading
Application	Using a concept in a new situation or unprompted use of an abstraction; Applying what was learned in the classroom into novel situations.	Key Words: apply, change, compute, demonstrate, discover, manipulate, modify, predict, prepare, produce, relate, show, solve, use. Examples: The student applies the new information in his/her future assignments or classroom activities.	Make sentences using the given pattern and words.
Analysis	Separating material or concepts into component parts so that its organizational structure may be understood	Key Words: analyze, breaks down, compare, contrast, discriminate, distinguish, identify, illustrate, infer, outline, relate, select, separate. Examples: The student compares and contrasts a new structure to the ones previously learned.	Compare the following words to see how they sound differently.
Synthesis	Putting parts together to form a whole, with emphasis on creating a new meaning or structure	Key Words: categorize, create, devise, design, explain, organize, plan, arrange, reconstruct, relate, revise, rewrite, summarize, tell, write. Examples: The student integrates information from several sources to solve a specific problem or to answer a question.	Make sentences using the scrambled words.
Evaluation	Making judgments about the value of ideas or materials	Key Words: appraise, conclude, critiques, evaluate, judge, justifies, relate, support. Examples: The student selects the most effective solution to a problem and is able to justify it.	On the scale, show how you evaluate the words. Which of the followings is the best answer to the question? Why?

March 2010 – Volume 13, Number 4