Vol. 6. No. 1 R-1 June 2002

Return to Table of Contents Return to Main Page

In The Art of Nonconversation, Marysia Johnson analyzes the validity a common type of spoken language assessment, the Oral Proficiency Interview (OPI) by using discourse analysis and conversational analysis insights. Finding it's validity wanting, she offers an alternative theoretical model of spoken language ability, the Practical Oral Language Agility (POLA) based on the sociocultural and institutional contexts in which the language has been acquired. This is a most welcome book because it not only brings to question the validity of OPIs but offers test developers a solid theoretical and researched model for developing, assessing and interpreting locally-constructed tests of spoken ability.

The book has a clear Table of Contents, acknowledgements, and an extremely useful List of Abbreviations. Its two Appendices (one for OPI coding sheet, the other a coding document listing the various codes used in analyzing the OPI), an excellent Bibliography, and a very useful Subject Index.

The book has nine chapters, the first of which gives an overview of the text. Chapters 2 through 7 give a description, history, and theoretical and empirical analysis of the Oral Proficiency Interview. The OPI's validity is investigated within Messick's framework of validity, given in this chapter. Chapters 8 and 9 investigate what speaking ability outside of tests is and proposes a new model of speaking ability.

In Chapter 1, Johnson briefly introduces the OPI in its many forms and poses the question of whether the OPI's use of an interview format is the most appropriate and desirable way to assess second language speaking ability. She then introduces her discourse analysis methodology and then introduces the question of what speaking ability is by offering a skeleton description of Vygotsky's sociocultural theory, the basis for Johnson's POLA, being the direct challenge to the current testing view of interaction as cognitively and psycholinguistically based.

In Chapter 2, a history of the OPI system, from its beginnings in the 1950's to the early 1990's improving and refining the earlier OPI, is outlined with many examples of the different types of the OPI. The structure of OPI, its elicitation techniques and rating procedures are then clearly described. The OPI's six general test characteristics are listed and a discussion of OPI's reliability and validity from the supportive viewpoint of academia's proficiency movement (late 70's to late 80's) is done. This movement tried to promote the OPI as the main assessment instrument for second and foreign language proficiency.

Chapter 3 begins the critical analysis of the OPI with a short history of its popularity and its becoming more institutionalized without having had a complete empirical investigation of it. Johnson then gives a historical outline of validity in general and cites Messick's momentous article on validity as the point to discuss two periods of lines of thought about validity, pre- and post-Messick. Johnson then discusses OPI's opponents' viewpoints, referring to the serious flaws in the OPI's claim that it measures conversational ability. To empirically test the OPI, she proposes to analyze the OPI using discourse and conversational analysis methodology and see if the OPI has its own speech event and rules outside those of conversation. [-1-]

The theoretical bases for exploring whether the OPI is a speech event in its own right are laid down in Chapter 4. Johnson does this by carefully grounding her methodology on recent discourse and conversational analysis findings beginning with defining a speech event. She then discusses conversation as a speech event, and its prototypical features of turn-taking, repair, adjacency pair systems, and topic that create order in conversation. She then explains both an interview and classroom interaction as speech events. Finally, she summarizes these prototypical features in all three speech events comparing and contrasting their similarities and differences.

Chapter 5 begins the qualitative and quantitative empirical analysis of the OPI in order to see what the prototypical features of the OPI are and to compare these with the features in interviews, conversations and classroom interactions. In this chapter she describes the quantitative study's data, coding system, coding process and a summary of findings. From the well-crafted summary of the discourse analysis findings, Johnson discovers that the OPI is a combination of a survey-research interview and a sociolinguistic interview, both not representative of real life conversations, the opposite of what Educational Testing Services says the OPI tests reflect.

The qualitative study of testers' and non-testers' perceptions of the OPI speech event is described in Chapter 6. This study was done to see if the findings of the earlier discourse analysis study are consistent with the findings of this perception study. This study used a semantic differential instrument, a well-established and validated instrument to compare individual native-speaker perceptions of the OPI. The data was analyzed in three parts and found to support the discourse analysis findings in Chapter 5.

Chapter 7 fleshes out the prototypical features of the OPI communicative event into a model for it. Within this model, Johnson describes four major sections, the warm-up phase, explanation of testing procedures, level check and probes, and the wind-down phase. None of these reflect conversation in real-life, but support the findings in chapters 5 and 6. Johnson then offers suggestions to those who sell OPIs regarding the advertising of what OPIs really test, changes in how OPI testers are trained, and implementing an official policy for procedures of testing for OPIs.

In chapter 8, Johnson gives a short history of communicative competence, the most widespread theoretical basis of second language teaching and testing. She then describes the two most widely accepted models of communicative competence in second language theory and testing. A discussion of communicative competence compared to proficiency follows, along with the problem with their being used interchangeably. To exemplify this problem, the Test of Spoken English (TSE) and Speaking Proficiency English Assessment Kit (SPEAK), both sponsored by ETS, are examined against their claims to measure communicative language ability. Johnson argues that the tests are indirect OPI tests and their validity is questionable because it is unclear what they actually measure--communicative competence or proficiency. In the last section, Johnson proposes an alternative framework to the communicative competence models, the interactional competence theory.This theory differs from the others in that co-construction of conversation and localized, specific context competencies, not general language competence, are recognized as central to participants' oral interactive practices. It is a theory of knowledge, specifying what participants has to have in order to participate in interactive contexts. This interactive competence is acquired in three processes ". . . observation, reflection and creation (p. 177)." This theory, Johnson says, is actually rooted in Vygotsky's sociocultural theory and the connections are then discussed. It is this sociocultural theory that Johnson states is the best theory at had to answer theoretical and practical issues related to speaking ability. [-2-]

In the first part of Chapter 9, the main beliefs of Vygotsky's sociocultural theory are described along with their implications for Second Language Acquisition (SLA) instruction and research. Johnson ends this chapter and the book with her Practical Oral Language Agility (POLA), which applies Vygotsky's theory to spoken language testing, specifically International Teaching Assistant's speaking tests given to determine spoken ability in a variety of academic situations. Some features of this test follow. This testing procedure should be locally developed and interpreted. This variety and number of contexts would be clearly defined, along with the intended audience and the test's purpose. Each context should be carefully analyzed in terms of functions, tasks, skills, and abilities. The oral events tested should be score separately by a group of evaluators. Testers' interactive abilities and their exposure to many training instances of the target, spoken language would be required. Feedback would be practical and informative, not just one score purporting to represent a speaking ability but including suggestions for improvement of weaker tested situations and descriptions of what the test taker can actually do.

This book is long overdue because it not only empirically measures what commonly used (and overused) OPIs actually do and are, but also offers a viable replacement for many other spoken language assessment contexts. Other applications of the POLA could be in such disparate spoken language contexts that now use OPI tests to evaluate language proficiency and gain in proficiency as Adult Pre-Employment English, Workplace English, Pre-Academic Speaking (formal presentation and social), and any variety of English for Specific Situations. This book should be required reading for anyone involved in a spoken language context. Graduate students, teachers, test designers, course designers, admission officers, and program administrators would all benefit from its insights.

Jim Bame
Utah State University
<fabame@cc.usu.edu>