TOEFL Mentor

Record of progress

Is the current working level always reported?

There is a report of test results after each section of the tests is taken.

Can progress be checked at any time?

No. When taking the test there is no report on whether items have been answered correctly or not, nor is there a report of the current percent correct.

Is a progress report maintained from session to session?

The test report can be saved to disk as a text file but there is no way to read the data back into the program. If a student does only one or two of the test sections, quits, and then returns, a total TOEFL score cannot be calculated. All three parts must be done at one sitting.

Presentation

Navigation

Is the program designed for easy navigation and control?

The structure is straight-forward since the students can only progress in a linear fashion inside any one exercise set. Each page is clearly labeled in the following manner: “Page 5 of 15”. TOEFL Mentor, however, does not allow students to easily navigate to other parts of the current section being studied since the only options are “Back” “Next” and “Menu.”

For the listening items, the student must press ‘play’ in order for the item to load and then play. This is true even for the timed tests. When the allotted 12 seconds is up, the program automatically proceeds to the next item but then waits for the student to press “Play.” It would have been more realistic to have the item play automatically, and perhaps provide a “Pause” button for the student to use if needed.

Also, there is no way to go back if “next” is accidentally hit one time too many. This can be a problem since students CAN hit “next” if they want to proceed to the next question before the 12 second limit is up. If the key is hit just as the time limit arrives, the program will go ahead two items instead of one. A warning could be built in to ask if the user really intended to skip the question, or the next button could have been relabelled ‘skip question’ until the item had been answered.

The listening items are rather cumbersome due to excessive dependence on the mouse. In order to hear an item, the user first has to press the ‘next’ button in the lower right corner and then, when the next item appears, press the “Play” button in the right middle of the screen. Even though Hypercard normally allows the keyboard arrow keys to control card changes, this feature seems to have been disabled in TOEFL Mentor. (Command+2 still works for moving to the previous card, but this would not be obvious to the normal user.)

While the Hypercard menu bar has been hidden, some of the key combinations do appear to work after a fashion. Command + P will print the current card; however, there is no way to control the size since the Page Setup menu item is not available. A single card thus prints out wastefully on two sheets of paper.

For the structure and reading tests, the students must proceed linearly through the items even though it is important when taking timed tests to learn to skip difficult items and to come back to them later.

In the Reading Comprehension section, even though the questions for each passage generally proceed in order from beginning to end, each new item resets the reading passage to the top, forcing the test-taker to scroll down again to the relevant section. It would have been much better if the passage was scrolled to the line being questioned each time.

When taking the tests, the user is allowed to return to previous items by laboriously clicking “back” but once the last item has been answered and “next” clicked, the score is reported and further revision is prohibited. It would have been better to have students click a “Done” button to confirm their intentions.

Is the interface intuitive?

Yes, to the extent that functions exist, their operation is obvious without explicit explanation.

Can the learner go back to earlier steps easily?

No, only the practice sections allow the student to go back, and even then through an arduous page by page manner.

Can the learner control the order of presentation?

Yes. TM suggests that the students take one of the two complete tests, then do the practice items, and finally take the second complete test to note their improvement. Inside the practice items the student is free to select any section to practice.

Can the learner control how long to spend at each step?

Yes, except for the tests themselves, which are timed, as they should be.

Pacing

The items are self-paced except for the tests, as mentioned above.

Graphical Presentation

Is the material easy to read?

Many people would prefer a printed booklet to the screen since reading items are presented in a scrolling window which utilizes only the upper portion of the screen — even when the lower portion is not being used for presenting an item.

Are the graphics useful?

Graphics are used lightly to ‘garnish’ the pages in a professional manner.

Is the layout well-designed?

It is a shame that the full area of the computer screen isn’t used. Roughly, the upper third of the TM window shows 7 lines of the target text, while the lower part contains the question and the multiple choice answers. Here, clearly, a low-tech textbook outshines the computer since the whole text can be seen at a glance. Students complained about having to scroll through the text. Further, even though the students were using 17″ monitors, the presentation stays at the “least common denominator” size of 13 inches.

Audio

Is the sound clear?

Yes, the sounds themselves have been recorded clearly but the beginning of each sound-bite often came out clipped when played. I tried the disk on three different models of Power Macintoshes, all with internal CD-ROM drives, with no difference in the result. Once the sound bite had been read into memory, repetitions were clear. Unfortunately, for the two listening tests, repetitions are not allowed. The students found this problem frustrating.

Is the rate of speaking appropriate?

Yes.

Are sound effects use appropriately?

Yes. There is no annoying sound when an item is wrong and only a light ‘ting’ when a correct item is selected.

Technical considerations

Is the program easy to install?

The program can be run directly from the CD-ROM. Placing the sound bites on the machines’ hard disk makes the sounds load more quickly, but this step requires 96MB of space.

Does the program load quickly?

The program does seem slightly sluggish. Part of this can be attributed to the CD-ROM and part to the general sluggishness of hypercard stacks.

Does the program work as expected, without bugs?

Unfortunately the answer to this is NO! The greatest annoyance was clicking on the “Sub Menu” button and then being greeted with a blank page! This seems to happen anytime one is a few pages deep into a section. The only cure seems to be to quit and restart TM.

Does the program take up excessive disk space?

It probably isn’t practical to transfer the contents to a hard disk, unless you have 110MB of disk space to spare!

Value

Does the program offer enough value for money?

The material provides yet one more way to study for the TOEFL. Individual students might find the product too expensive for private use. For institutions, however, TOEFL Mentor could provide numerous hours of useful study material despite the flaws in the current version.

Could the goals be achieved more easily without a computer?

With the exception of the ability to listen to items repeatedly at the push of the mouse button, I don’t think there is anything in the current version of the program that could not be done as well with more traditional materials. The listening feature, however, is superior to the non-tech alternative of using a text with a cassette tape since it is much easier to find and repeat specific passages. With the addition of some other features suggested in the ‘wish list’ below and elsewhere in this article, a product could be made that is superior to traditional texts, even for the Structure and Reading sections.

Wish List

It would be nice if…

• clicking on an unknown vocabulary item produced a definition.
• clicking a button would highlight the location of the correct answer within long reading exercises, rather than locations having to be specified by line number.
• a record of errors were kept so that students could return to items that they had got wrong the first time through.
• the material was available in the major languages of ESL/EFL student populations: Spanish, Japanese, Arabic, etc. Study materials like this often require the student to understand explanations which are syntactically more complex than the concept being explained. Use of the native language would allow them to study the material more effectively, particularly since they probably are not familiar with English grammar terms. Lexical glosses in the native language would also be welcomed by most students.

Summary

While the program is still somewhat “buggy” and lacks many of the programming niceties that could make this a top notch program, even in its present form it can provide students with considerable practice at test-taking techniques while providing a thorough review of the standard item types and grammatical patterns that appear on TOEFL exams. I would recommend, however, that students do not spend time puzzling out the feedback on their answers since the explanations are often unsatisfying.

I would hope that this review and other user reports will stimulate Encomium to develop a revised version of TOEFL Mentor with enhanced functions and more accurate items and explanations.

Please see Appendix 2 for the publisher’s comments to this review.

Reviewed by

Thomas N. Robb
Kyoto Sangyo University
Kyoto, Japan
<trobb@cc.kyoto-su.ac.jp>

© Copyright rests with authors. Please cite TESL-EJ appropriately. Editor’s Note: Dashed numbers in square brackets indicate the end of each page in the paginated ASCII version of this article, which is the definitive edition. Please use these page numbers when citing this work.