 |
|
||||||||||
|
|
|
||||||||||
|
Usefulness and Enjoyableness of Teaching Materials as Predictors of On-task Behavior
Matthew Peacock
Department of English
City University of Hong Kong
<enmatt@cityu.edu.hk>
This article reports on a small-scale study that investigated the relationship between on-task behavior, usefulness of materials as perceived by learners, and enjoyableness of materials. It was hypothesized before the study that learners spend a greater proportion of the lesson on task (that is, actively engaged in the task that they were set) when they see the lesson as useful and/or enjoyable.
Data were collected daily over one term in two South Korean university EFL classrooms. The correlation between on-task behavior and usefulness of materials was very low at r = .1608; with enjoyableness, it was also low at r = .2463. It was concluded that the learners did not spend more time on task because they perceived the materials to be more useful, or more enjoyable, but for some other reason.
The project also replicated part of a previous study. Green (1993) found a correlation averaging r = .6480 between the EFL learner-rated usefulness and enjoyableness of the activities in Puerto Rico. The present study also found the correlation high, at r = .7040.
(1) Correlating on-task behavior and usefulness; and (2) Correlating on-task behavior and enjoyableness.
Most (or all) EFL teachers would agree that maintaining a high level of on-task behavior (a combination of learner activity and persistence; see below) is both necessary and desirable within the classroom. There is some support for this contention: Crookes and Schmidt (1991, pp. 498-502) maintain that teachers are "convinced" that motivation as displayed in learner persistence with the task, activity, and concentration is crucial to second language success; and also that teachers are concerned not with the reasons for study, but with ensuring that learners are engaged in sustained study without continual encouragement (p. 480). Ushioda (1993, pp. 1-3) calls this a "practitioner-validated" view of motivation, and adds that learner participation, attentiveness, and enthusiasm can be regarded as significant outcomes in themselves, apart from the fact that they are "crucial" to L2 development. Schmidt (1990, p. 129) concludes that "paying attention is probably facilitative" to linguistic progress. Graves asserts (1993, p. 4) that most teachers regard learners' being "active, eager and involved" as a measure of motivation, and defines motivation as a "continued desire to engage in learning."
The following three assertions were made by Crookes and Schmidt in their influential 1991 article (pp. 486 and 492):
1. "Individuals will have a difficult time forcing themselves to attend for long to tasks that they perceive as irrelevant."
2. "It seems likely that personal relevance can be [a] good determinant of selective attention."
3. "A program which appears to meet the students' own expressed needs ... will be more motivating, more efficient, and thus more successful."
Thus, the question investigated in the present pilot study is important to EFL teachers. Maintaining high levels of on-task behavior is both necessary and desirable; the existence of a consistent high correlation between on-task behavior and learner-perceived usefulness and enjoyableness has implications for the selection of materials for the EFL classroom. The premise that higher levels of learner on-task behavior positively affect learning outcomes can be accepted, in the absence of evidence to the contrary, as the default assumption. [-2-]
(3) Correlating usefulness and enjoyableness
The association between these two factors has received little attention, as Green (1993, p. 2) notes: "No previous studies known to me have investigated possible relationships between the reported enjoyableness and perceived effectiveness of different kinds of activities." Green examined this relationship in his university-level EFL classroom in Puerto Rico. He reports (p. 8) that effectiveness and enjoyment ratings were highly correlated; the average correlation was r = .6480. In his conclusions (p. 8) he comments: "If the relationship between enjoyableness and effectiveness is ... circular and mutually reinforcing, then teachers would be well advised to cultivate an enjoyable classroom environment." He also calls (p. 8) for the study to be replicated with other populations in order to "identify potentially troublesome differences between teacher styles and student expectations." As Green states, it is important that learner preferences for and opinions on EFL classroom materials are investigated, and that teachers do not rely solely on intuition and experience when selecting materials. My aim was to replicate this part of Green's study in a South Korean setting.
Learners speaking to others in their group or to the teacher, or actively working within their assigned group, were regarded for the purposes of this study as being on task. The referent for on-task behavior was a score on an observation sheet designed to quantify on-task behavior. An observer recorded levels of on-task behavior.
It could be argued that it is not possible to observe on-task behavior because it takes place within the learner's mind, and that a non-participant observer can hardly know whether a learner is on task or not. However, the inter-rater reliability of the on-task behavior observation sheet was computed at the high level of r = .9745 (p = .031) when two observers independently rated the on-task behavior of learners during a previous study in the same school (this will be further discussed below under "Collecting the Data.") Both observers commented that they did not find it difficult--not as difficult as might be assumed--to assess and record levels of on-task behavior, and these comments are supported by the high [-3-] reliability computed for the instrument. This reliability is some indication (though not, of course, proof) that the behavior is observable. The inter-rater reliability of on-task behavior measures needs further empirical and theoretical checking in different settings and contexts.
"Usefulness" was defined as relevance to the goals of the learners in the target classes, effectiveness for language learning, and appropriacy. The referent was the combined score for four out of the ten items on a post-class questionnaire filled out daily by all learners in both classes. The questionnaire is found in the Appendix .
"Enjoyableness" was defined as that learner interest, enjoyment and enthusiasm that is generated by the materials in use. The referent was the combined score for six out of the ten items on the learner daily post-class questionnaire. One point to be considered regarding the concept of enjoyableness of materials is whether it has the same meaning in South Korea as in the West. Might it be possible that Korean learners do not regard enjoyableness of materials as desirable, or helpful? (The same question can, of course, be asked of all EFL learners around the world.) It could be argued that South Korean learners might not want to be "interested" in materials--that they might regard "interest" as an irrelevant or even undesirable feeling in EFL class; and also that they might accede to the teacher's opinion on the materials. However, a certain degree of familiarity with Korean learners and culture (after eight years TEFL in South Korea) allows me to propose that South Korean learners want (as least as much as, for example, university-level learners of Spanish as a second language in the USA) to be given interesting and enjoyable materials, and also that they regard interesting and enjoyable materials as a desirable and necessary part of learning EFL. This is particularly true of the young adult learners in the present study. Some evidence that learners do not defer to the teacher was obtained in a previous study (Peacock, 1997, pp. 151-152) where learners in the same school frequently expressed negative as well as positive opinions about materials during post-class interviews. Deference to the teacher's opinion was also reduced, of course, by having learners complete questionnaires anonymously in the present study. I have now come to regard the concept of enjoyableness of materials as applying almost equally in the West and in South Korea. The factor of potentially different responses to materials in differing cultures remains, however, an important one to consider before applying any data-collection instrument outside the culture in which it was first prepared and used.
(1) A tally sheet for quantifying learner on-task behavior
(2) A self-report questionnaire for learners
As no suitable instruments could be found in the literature, two existing instruments were extensively adapted:
(1) A tally sheet from Hopkins (1985, p. 95). The sheet used in the study consisted of 12 columns for scans and 18 lines for learners, and was designed to be filled out by a non-participant observer (or the class teacher) while learners were working in groups. The observer had to enter "1" if learners were on task and "2" if they were not on task for each scan, until all the learners had been observed 12 times.
(2) A questionnaire (from Gliksman, Gardner, & Smythe, 1982, p. 646) was very extensively revised and adapted for the study. It was filled out anonymously by individual learners. The aim of this written questionnaire was to let learners express their opinion on the usefulness and enjoyableness of the supplementary materials they had worked with on the day in question. The questionnaire was translated into Korean to avoid learner confusion; the English and Korean translation were given together. The questionnaire consisted of a list of ten items in opposition. Multiple items, chosen for their lexical similarity, were chosen for each variable in order to increase the reliability of the instrument. The items were: interesting/boring, relevant to my goals/not relevant to my goals, unenjoyable/enjoyable, meaningless/meaningful, appropriate for me/not appropriate for me, dull/exciting, satisfying/unsatisfying, useful for me/useless for me, unappealing/appealing, and absorbing/monotonous.
Items two, four, five, and eight (relevant, meaningful, appropriate, and useful), taken together, constituted the usefulness factor; and items one, three, six, seven, nine, and ten (interesting, enjoyable, exciting, satisfying, appealing, and absorbing) together made up the enjoyableness component. The appropriacy of dividing the ten items into these two distinct components was confirmed by factor analysis after the study.
Items were graded on a seven-point semantic differential scale. Learners put an "X" against the place on the scale that best described their opinion of the materials. For example, an "X" next to "boring" scored one on that item; an "X" next to "interesting" scored seven; and an "X" halfway between the two words scored four.
With the questionnaire, learners anonymously reported how useful and enjoyable they found the materials to be. The results from this instrument were perhaps less controversial than those from the on-task observation sheet, because learners were able to report their feelings directly. [-5-]
On-task tally sheets and questionnaires were collected every day for seven weeks (over 20 separate days, as class was held four days a week, interrupted by holidays.) The stages of all activities were the same--the teacher introduced the activity, handed out the materials, went over the task, divided the class into groups of three, and started the activity. When the class had finished working and feedback had been conducted, learners filled out the post-activity questionnaires. A questionnaire was completed by each learner daily and then handed back to the teacher. A total of 40 tally sheets (20 from each class) and 391 questionnaires were collected over the term.
The inter-rater and intra-rater reliability of the on-task observation sheet had been computed in a previous study at r = .9124 (p = .031, N = 60) and r = .9745 (p < .001, N = 34) respectively (for details, see Peacock, 1997).
The internal consistency of the questionnaire was assessed after the present study via an alpha reliability check. The two components of the questionnaire, "usefulness" and "enjoyableness", were checked separately. The result for "usefulness" was r = .8038 (p = .0161, N = 391), and for "enjoyableness" r = .9295 (p < .001, N = 391). I suggest that these results are an indication of a high level of internal consistency. [-6-]
The three-way correlations between on-task behavior and learner self-report of the factors "usefulness" and "enjoyableness" were checked via Pearson r. Conditions for the use of Pearson r were checked and were met.
(2) Results indicate a low positive correlation between on-task behavior and learner report of enjoyableness; r = .2463 (p < .001, N = 391).
(3) Results indicate a high positive correlation between learner report of usefulness and enjoyableness; r = .7040 (p < .001, N = 391).
(A) Results (1) and (2): little correlation between on-task behavior and the learner-rated usefulness and enjoyableness, of materials.
These results indicate that the learners in this study did not spend a higher proportion of the time on task because they perceived the materials to be more useful, or more enjoyable, but for some other reason. It should be noted that these findings run counter to the assertions made by Crookes and Schmidt (1991), reviewed above, that personal relevance will be a good determinant of selective learner attention.
The broad question raised in this study, of what factors affect levels of EFL learner activity in class, has not been answered even for the subject classes, except in a very limited way. However, I propose that the indications that (1) usefulness and (2) enjoyableness were not associated with increased levels of on-task behavior in the present study are of considerable value and interest to EFL teachers, as well as researchers, because they raise the question of what other factor or factors affect levels of on-task behavior in any EFL classroom.
It is noteworthy that class levels of on-task behavior varied considerably from day to day, and not just the levels of individual learners; that is, the classes tended to vary as a whole. I propose that it is possible to speculate that some or all the following other factors may be associated with increased or decreased levels of on-task behavior. The list of factors will be followed by separate subsections covering suggestions for EFL teachers, and for researchers: [-7-]
(2) The type of activity undertaken by learners
(3) The teaching approach adopted by the teacher
(4) The motivation that brings learners to class--characterized by Gardner and Lambert (1972) as either "instrumental" or "integrative"
(5) Cultural factors
There are many factors that may affect levels of on-task behavior, either singly or in combination with each other; the possible existence of these unknown combinations may frustrate the search for the identification of individual factors. However, I suggest that further exploratory research such as the present study is needed in order to separate and identify these factors. If it is accepted that maintaining high levels of on-task behavior is both necessary and desirable, I propose that the factor or factors with which maintaining those levels might be associated deserves much further investigation within the EFL classroom.
There may also be a link between extrinsic motivation, the use of particular materials, and on-task behavior. For example, with a class that was primarily integratively motivated, materials referring to the target language culture may increase on-task behavior. This also has implications for lesson planning.
Cultural factors may also play a role. Green's study was in Puerto Rico and the present study in South Korea. I propose, though, that Korean learners want (as least as much as learners in the USA or Europe) to be given interesting and enjoyable materials, and regard them as desirable and as an important and necessary part of learning EFL.
I do not intend to generalize my findings outside the context of this study. The question raised, however--of what factors affect on-task behavior--should be examined in the local setting of any EFL teacher. The question of how far levels of on-task behavior vary from country to country was not, of course, investigated in the present study. The question deserves investigation, via cross-cultural comparison. [-8-]
(B) Result (3): high correlation between the learner-rated usefulness and enjoyableness of materials.
This result is very similar to Green's (1993, p. 6), who reports an average correlation of r = .6480 between the student-rated usefulness and enjoyableness of the activities in his EFL classroom; the correlation in the present study was r = .7040. I agree with Green's proposal (1993, p. 8; reviewed above) that usefulness and enjoyableness are "mutually reinforcing," and that therefore teachers would be well advised to cultivate an enjoyable classroom environment.
As a final conclusion, I propose that it is important to try to identify the factor or factors with which increased levels of on-task behavior are associated; the question of what these factors are has not yet been answered.
Crookes, G., & Schmidt, R. W. (1991). Motivation: reopening the research agenda. Language Learning, 41/4, 469-512.
Gardner, R. C. & Lambert, W. E. (1972). Attitudes and motivation in second-language learning. Rowley, MA: Newbury House. [-9-]
Gliksman, L., Gardner, R. C., & Smythe, P. C. (1982). The role of the integrative motive on students' participation in the French classroom. Canadian Modern Language Review, 38, 625-647.
Graves, K. (1993). Japanese method. EFL Gazette, 165, 4.
Green, J. M. (1993). Student attitudes toward communicative and non-communicative activities: do enjoyment and effectiveness go together? Modern Language Journal, 77/1, 1-10.
Hopkins, D. (1985). A teacher's guide to classroom research. Milton Keynes: Open University Press.
Nunan, D. (1992). Research methods in language learning. Cambridge: Cambridge University Press.
Peacock, M. (1997). The effect of authentic materials on the motivation of EFL learners. English Language Teaching Journal, 51/2, 144-156.
Schmidt, R. W. (1990). The role of consciousness in second language learning. Applied Linguistics, 11/2, 129-158.
Ushioda, E. (1993). Redefining motivation from the L2 learner's point of view. Teanga, 13, 1-12.
Matthew Peacock teaches in the Department of English at the City University of Hong Kong. He taught EFL at university level in South Korea for eight years, before completing doctoral studies in the University of Essex. His current research interests include programme evaluation, materials evaluation, authentic materials, motivation, and learner attitudes and behavior.
|
||||||||||
|
© Copyright rests with authors. Please cite TESL-EJ appropriately. Editor's Note: Dashed numbers in square brackets indicate the end of each page in the paginated ASCII version of this article, which is the definitive edition. Please use these page numbers when citing this work. |
APPENDIX
STUDENT QUESTIONNAIRE
DATE: DD MM YY
*Teaching Materials:
*Do not write your name on this sheet - fill it out and
give it back to your teacher.
*The purpose of this questionnaire is to assess the value
of the above teaching materials which were used in class
today - NOT to assess the performance of you or you
teacher.
*This is not a test. There are no right or wrong answers;
we want your own ideas and impressions.
*Please mark ONE "X" on each scale to show how you rate the
following concepts. Use the scales as follows:-
*If the word at either end of the scale very strongly
describes your ideas and impressions about the concept, you
would place your check-mark as shown below:
boring__X__:_____:_____:_____:_____:_____:_____interesting
OR
boring_____:_____:_____:_____:_____:_____:__X__interesting
*If the word at either end of the scale describes somewhat
your ideas and impressions about the concept (but not
strongly so), you would place your check-mark as follows:
boring_____:__X__:_____:_____:_____:_____:_____interesting
OR
boring_____:_____:_____:_____:_____:__X__:_____interesting
*If the word at the end of the scale only slightly
describes your ideas and impressions about the concept, you
would place your check-mark as follows:
boring_____:_____:__X__:_____:_____:_____:_____interesting
OR
[-11-]
boring_____:_____:_____:_____:__X__:_____:_____interesting
*Mark ONE "X" on each line:
interesting_____:_____:_____:_____:_____:_____:_____boring
relevant to my not relevant to
goals_____:_____:_____:_____:_____:_____:_____my goals
unenjoyable_____:_____:_____:____:_____:_____:____enjoyable
meaningless_____:_____:____:____:_____:_____:____meaningful
appropriate not appropriate
for me_____:_____:_____:_____:_____:_____:_____for me
dull_____:_____:_____:_____:_____:_____:_____exciting
satisfying_____:____:_____:_____:____:____:____unsatisfying
useful for me____:___:___:___:____:____:____useless for me
unappealing_____:____:_____:_____:_____:____:_____appealing
absorbing_____:____:_____:_____:_____:_____:_____monotonous
[-12-]