Vol. 6. No. 3 — December 2002
Questionnaires in Second Language Research: Construction, Administration, and Processing
Zoltan Dörnyei (2003)
Mahwah, New Jersey: Lawrence Erlbaum Associates, Publishers.
Pp. viii + 156
ISBN 0-8058-3909-7 (paper)
US $19.95
The central methodological problem in any type of research concerns how the relevant data is to be collected. In the field of second language research, a very popular data collection technique has been the use of various types of questionnaires. But as Zoltan Dornyei of the University of Nottingham points out in his brand new book, Questionnaires in Second Language Research, “there does not seem to be sufficient awareness in the profession about the theory of questionnaire design and processing. The usual–and in most cases false–perception is that anybody with a bit of common sense can construct a good questionnaire.” (p. 1)
Dornyei believes that this erroneous perception is based on a lack of recognition “that there is considerable relevant knowledge and experience accumulated in various branches of the social sciences (e.g. psychometrics, social psychology, sociology).”(p. 1) The purpose of his book is to present practical information on how to make and use quality questionnaires and thereby “save ourselves a lot of trouble.” (p. 2)
The book is short, but very much to the point, and consists of four chapters. The first ‘Questionnaires in Second Language Research,’ is a general introduction to questionnaires. Dornyei (quoting Brown, 2001) defines a questionnaire as “any written instruments that present respondents with a series of questions or statements to which they react either by writing out their answers or selecting from among existing answers.” (p. 6) Dornyei also notes that questionnaires should not be confused with tests or any type of discourse completion. He next shows that questionnaires yield three types of data about the respondent: attitudinal, behavioral, and factual.
Dornyei believes that questionnaires are especially valuable because they are efficient “in terms of (a) researcher time, (b), researcher effort, and (c), financial resources.” (p. 9) He also examines the major drawbacks of questionnaires: the simplicity of answers yielded, the problem of respondents who are unmotivated or unreliable, the famous halo effect, the acquiescence and prestige biases, issues concerning self-deception and respondent literacy, and the effect of fatigue in cases where the questionnaire is long.
Chapter two, ‘Constructing the Questionnaire,’ discusses the main parts and general features of a questionnaire. Dornyei also explains rating scales, both multi-item and closed-ended, as well as open-ended questions. He then draws up a set of simple rules, which describe how to write good items, how to group and order the items, and how to pilot a questionnaire and perform item analysis. [-1-]
The next chapter, ‘Administrating the Questionnaire,’ concerns the topics of sample size, questionnaire confidentiality, and administration. Dornyei also gives tips on how to maximize both the quantity and quality of participant responses.
In chapter four, ‘Processing Questionnaire Data,’ Dornyei tells how to code questionnaire data, and put data into a computer file, as well as ways to process closed questions and do content analysis of questions that are open-ended. He also lists several computer programs that are good for processing data. He closes the chapter by writing about how to report questionnaire data, covering the areas of “general guidelines about what to report and how. The technical information about the survey that needs to be included in the professional report to accompany the actual findings. (And) Presentation methods that can make the data more reader-friendly and digestible.” (p. 119)
The final section of the book is a 39-part checklist of what Dornyei thinks are the most important elements of questionnaire design, implementation, and analysis.
The book also contains an appendix which lists six pages of published L2 questionnaires, on diverse topics ranging from attitudes and language anxiety, to language learning styles and teacher self-evaluation.
All in all, Questionnaires in Second Language Research is a brisk, no nonsense introduction to a complex subject. Dornyei writes clearly and concisely and can explain complicated technical matters in a simple manner. But I have two problems with the book’s content. First, it would have been extremely helpful if Dornyei had written a chapter that had taken the reader through the entire process of making, administrating, and processing a questionnaire. Dornyei’s numerous pointers and checklists are very important and informative, but they are ultimately not enough, he should have also supplemented his presentation with more concrete and detailed examples. The book jacket states that Dornyei has been using questionnaires in his research on language learning motivation for over 15 years, that he has surveyed almost 10,000 respondents, and that he has published over 30 articles based upon this work. It would have been very informative if Dornyei would have described in detail how he went about designing and using one of his research questionnaires.
In addition, Dornyei fails to mention anywhere in this work a key problem which plagues any second language researchers who use questionnaires and conducts their research overseas or writes the questionnaires in the native language of the participants: namely the question of how does one go about making sure that the items used are correctly translated into the native language? Since cross-cultural questionnaires and questionnaires done in the first language of the respondents have recently become quite common in research journals, the topic is of general concern. To use a personal example, for the past two and a half years, I have been involved with a colleague in Japan in an extended research project investigating Chinese and Japanese attitudes towards English in relation to issues concerning language purism and identity. One major and continual problem we have had with the construction of our questionnaires involves ensuring that each of our questionnaires precisely match the other in terms of meaning for all of the items we use. Thus, it is problematic that Dornyei has neglected to discuss this important aspect of questionnaire design.
References
Brown, J.D. (2001). Using Surveys in Language Programs. Cambridge, UK: Cambridge University Press.
Ronald Gray
Beijing Language and Culture University, Beijing, China
<mnenomic_2000@yahoo.com>
|
© Copyright rests with authors. Please cite TESL-EJ appropriately. Editor’s Note: Dashed numbers in square brackets indicate the end of each page for purposes of citation.. |
[-2-]