{"id":3007,"date":"2009-08-20T11:34:11","date_gmt":"2009-08-20T18:34:11","guid":{"rendered":"http:\/\/www.tesl-ej.org\/wordpress\/?page_id=3007"},"modified":"2009-08-20T11:47:52","modified_gmt":"2009-08-20T18:47:52","slug":"ej26int","status":"publish","type":"page","link":"https:\/\/tesl-ej.org\/wordpress\/issues\/volume7\/ej26\/ej26int\/","title":{"rendered":"Google as a Quick ‘n Dirty Corpus Tool"},"content":{"rendered":"

September 2003 — Volume 7, Number 2<\/h3>\n

From the Editor<\/h3>\n

Concordancing is a venerable old idea whose potential has never been reached, according to its most assiduous proponents. Until recently it was assumed that specialized software was required to do concordancing, but it turns out that a search engine such as Google can generate queries into almost limitless corpora (using the Advanced Search feature from the main portal page, for example). This paper by Tom Robb addresses more refined issues regarding the integrity of the data thus derived, and how we might improve on the integrity of that data through more defined searches, as explained here. <\/p>\n

Vance Stevens, Editor
\nOn the Internet<\/P>


\n
<\/p>\n

Google as a Quick ‘n Dirty Corpus Tool<\/h2>\n

by Thomas Robb
\n<trobb@cc.kyoto-su.ac.jp<\/a>><\/p>\n

<\/center><\/p>\n

The Problem<\/h3>\n

This message appeared on August 6, 2003 on the JALTTALK mailing list, a list for language teachers in Japan. <\/p>\n\n\n
<\/p>\n

Date: Wed, 06 Aug 2003 10:28:22 +0900
\nFrom: Russell Willis <russe11@eigotown.com>
\nSubject: [jalttalk 25027] Eiken Question
\nThis from the Eiken test:<\/p>\n

Her wedding dress was very a)beautiful b)gorgeous c)wonderful <\/p>\n

The correct answer is beautiful — but can anyone explain exactly why?<\/p>\n

<\/font><\/p>\n<\/td>\n<\/tr>\n<\/table>\n

The first respondent supplied the standard explanation that “Very” cannot be used with non-gradable adjectives. <\/p>\n\n\n
<\/p>\n

Date: Wed, 06 Aug 2003 10:40:07 +0900
\nFrom: <app1epie@minos.ocn.ne.jp>
\nSubject: [jalttalk 25028] Re: Eiken Question<\/p>\n

The adverb “Very” rarely co-occurs with strong meaning adjectives, such as wonderful, delicious, fantastic, excellent, magnificent etc, unless you want to mean something sarcastic.<\/p>\n

<\/font>\n<\/td>\n<\/tr>\n<\/table>\n

Here is the explanation given on one web site concerning non-gradable adjectives. [-1-]<\/p>\n\n\n
<\/p>\n

Unique <\/b>means “one of a kind.” Therefore, comparatives, superlatives, and words like very, so, <\/b>or extremely <\/b>should not be used to modify it. If it is one of a kind, it cannot be compared! <\/p>\n

Incorrect:<\/b> He is a very unique personality. <\/p>\n

\nCorrect:<\/b> He is a unique personality. <\/p>\n

This same logic applies to other words which reflect some kind of absolute: absolute, overwhelmed, straight, opposite, right, dead, entirely, eternal, fatal, final, identical, infinite, mortal, opposite, perfect, immortal, finite, or irrevocable. <\/p>\n

<\/b><\/p>\n

http:\/\/englishplus.com\/grammar\/00000269.htm<\/a><\/p>\n

<\/font>\n<\/td>\n<\/tr>\n<\/table>\n

While “wonderful” might be considered “non-gradable” since it means “full of wonder” and, if something is “full” it is therefore non-gradable. But then, “beautiful” should mean “full of beauty” and by the same argument should also be ungrammatical. So much for logic! What we need is real data. <\/p>\n

In fact, the next poster, then claimed that both “very gorgeous” and “very wonderful” could be found in a Google search. (Petersen:[jalttalk 25029]) While this may be true, anyone can make a web page these days including non-native speakers and first graders. This would seem to make Google useless as a source for such answers—or is it? <\/p>\n

Possible Approaches<\/h3>\n

Google can be used to derive creditable information in a number of ways: <\/p>\n

    \n
  1. Instances might be found from “impeccable sources” which would authenticate its usage. <\/li>\n
  2. We can show, by a careful selection of domains, that the frequency of occurrence of the word or phrase under scrutiny shows up with a similar relative frequency even in domains where one would expect educated usage. <\/li>\n
  3. Strength of collocation – We can demonstrate that “very” collocates with the word in question, even in domains indicating educated usage. <\/li>\n<\/ol>\n

    Procedure<\/h3>\n
  4. Impeccable Sources\n

    Project Gutenberg is a huge, volunteer-based effort to create e-texts of material that is out of copyright, particularly works of literature and texts of historical value. See http:\/\/www.gutenberg.org<\/a> for further information. <\/p>\n

    To search within a specific domain, two separate techniques were used, neither of which was perfect. [-2-]<\/p>\n

    Method 1: In addition to the target phrase in quotation marks, “Project Gutenberg” and “Michael S. Hart” were specified in order to restrict the search to texts, which were part of the project. All such texts contain header material with this information. The largest problem with this technique is that there are multiple sites containing the text, some authorized by the Gutenberg Project, and others not, so the “hit count” is inflated. One could assume, however that all hit counts will be inflated by approximately the same amount and thus will not affect the frequency ratios to any great extent. (= Guten-1 <\/b>)<\/p>\n

    The search string “very beautiful” “Project Gutenberg” “Michael S. Hart” <\/b>produces something similar to the results shown here: <\/p>\n


    \n
    \n<\/center><\/p>\n

    Method 2: In addition to the target phrase in quotation marks, site:gutenberg.org <\/b>was specified. This is a Google-specific search parameter to restrict the search to a specific domain. While this would seem to be ideal, there were two problems, a) the site has sub-sites which contain some, but not all of the same texts perhaps in differing formats, and b) some legitimate texts discovered by method #1, “Main Street” by Sinclair Lewis, for one, failed to be returned. (= Guten-2 <\/b>) [-3-]<\/p>\n

    The search string “very beautiful” site:Gutenberg.org <\/b>produces the results are partially shown below: <\/p>\n


    \n
    \n<\/center><\/p>\n

    I thus decided to search using both techniques and compare the figures derived by each method. <\/p>\n

  5. Relative frequency of occurrence\n

    Using a similar technique to that described above, sites ending in ” edu <\/b>” mainly U.S.-based educational entities, as well as their British ( ac.uk <\/b>) and Australian ( edu.au <\/b>) counterparts, and ” .jp <\/b>“, Japan-based pages in English were searched, as well as pages with no additional criteria in order to search the entire Google database. The total number of hits was recorded. In instances where a number of less than 100 was reported, Google often mentioned that there were more pages with substanitially the same content that were not displayed. In such instances, the lower number was used. <\/p>\n

    From the data thus derived, the ratio of hits of each phrase to “very beautiful” was calculated, since “very beautiful” is undeniably an acceptable phrase (and the correct answer in the original examination question). Any variance in the ratio can be explained as a greater or lesser tendency to use the phrase in the domains under investigation. “Window pane,” included in the data for comparison purposes, shows an interesting pattern, with three of the academic domains showing a higher ratio of usage than “All Google” or the other columns. This cannot be interpreted to mean that “window pane” is more acceptable, only that for some odd reaason the academic domain has a greater need to refer to “window panes.” The ratios presented merely mean that there is a greater or lesser likelihood of this term being needed for the genre distribution of that particular domain. [-4-]<\/p>\n

    Very unique <\/b>was included in the searches since it is a standard example of a usage looked upon with approbation, and one that educated speakers are often aware of and will try to avoid. <\/p>\n

    Results <\/h3>\n

    There were approximately 90 instances of “very wonderful” by authors including one by Mark Twain. For “very gorgeous,” I found 11 instances including one by A.A. Milne and another by Herman Melville. See the examples at the end of the paper. <\/p>\n

    Frequency data:<\/b><\/p>\n\n\n\n\n\n\n\n\n
     <\/th>\nAll Google<\/th>\n.edu<\/th>\nac.uk<\/th>\nedu.au<\/th>\n.jp<\/th>\nGuten-1<\/th>\nGuten-2<\/th>\n<\/tr>\n
    very beautiful<\/b><\/td>\n430,000 <\/td>\n13,800 <\/td>\n1,490 <\/td>\n1,250 <\/td>\n5,670 <\/td>\n2,750 <\/td>\n357 <\/td>\n<\/tr>\n
    very gorgeous<\/b><\/td>\n10,100 <\/td>\n78 <\/td>\n5 <\/td>\n10 <\/td>\n106 <\/td>\n11 <\/td>\n0 <\/td>\n<\/tr>\n
    very wonderful<\/b><\/td>\n56,000 <\/td>\n2,210 <\/td>\n89 <\/td>\n55 <\/td>\n961 <\/td>\n90 <\/td>\n71 <\/td>\n<\/tr>\n
    very unique<\/b><\/td>\n316,000 <\/td>\n11,700 <\/td>\n323 <\/td>\n303 <\/td>\n2,130 <\/td>\n11 <\/td>\n1 <\/td>\n<\/tr>\n
    window pane<\/b><\/td>\n53,700<\/td>\n3,150<\/td>\n295<\/td>\n289<\/td>\n144<\/td>\n84<\/td>\n54<\/td>\n<\/tr>\n
    Table 1 – Frequency data<\/th>\n<\/tr>\n<\/table>\n

    Ratio of other phrases to “very beautiful”: <\/b><\/p>\n\n\n\n\n\n\n\n
     <\/th>\nAll Google<\/th>\n.edu<\/th>\nac.uk<\/th>\nedu.au<\/th>\n.jp<\/th>\nGuten-1<\/th>\nGuten-2<\/th>\n<\/tr>\n
    very gorgeous<\/b><\/td>\n0.023<\/td>\n0.006<\/td>\n0.003<\/td>\n0.008<\/td>\n0.019<\/td>\n0.004<\/td>\n0.000<\/td>\n<\/tr>\n
    very wonderful<\/b><\/td>\n0.130<\/td>\n0.160<\/td>\n0.060<\/td>\n0.044<\/td>\n0.169<\/td>\n0.033<\/td>\n0.199<\/td>\n<\/tr>\n
    very unique<\/b><\/td>\n0.735<\/td>\n0.848<\/td>\n0.217<\/td>\n0.242<\/td>\n0.376<\/td>\n0.004<\/td>\n0.003<\/td>\n<\/tr>\n
    window pane<\/b><\/td>\n0.125<\/td>\n0.228<\/td>\n0.198<\/td>\n0.231<\/td>\n0.025<\/td>\n0.031<\/td>\n0.151<\/td>\n<\/tr>\n
    Table 2 – Ratio of other phrases to “very beautiful”<\/th>\n<\/tr>\n<\/table>\n

    [-5-]<\/p>\n

    Comparing the acceptable form “very beautiful” with both the raw number of hits and the frequency ratios, we can see that very unique <\/b>is almost as frequent as very beautiful <\/b>in the overall count, and not far behind in the . edu <\/b>and . jp <\/b>domains. The ac.uk <\/b>and edu.au <\/b>domains are strikingly similar. Surprisingly, the . jp <\/b>domain tends to conform closely to the . edu <\/b>domain, but the cause will have to wait for another paper. Gutenberg <\/b>authors apparently disdain very unique <\/b>although examples such as the Sherlock Holmes instance, (” . . . some very unique features”) indicates that when suitably modified, the phrases appears to be more acceptable. A true corpus-based study might be able to reveal the exact conditions, which allow it to be construed as “grammatical”. <\/p>\n

    In the case of very wonderful <\/b>, the frequencies are similar across all domains although ac.uk <\/b>and edu.au <\/b>show approximately one-third the hits of the other domains. Very gorgeous <\/b>reveals few instances overall. Many of the tokens in the “general Google” search are from porno sites advertising their wares. (non-wears? :-). A second search with the “safe search” filter on yielded 5220 hits, approximately half the original number. <\/p>\n

    Strength of Collocation<\/h3>\n

    The following two charts show the total number of occurrences of the word in question in Google, and the proportion of those occurrences that collocate with “very.” <\/p>\n\n\n\n\n\n\n\n
    Totals<\/th>\nAll Google<\/th>\n.edu<\/th>\nac.uk<\/th>\nedu.au<\/th>\n.jp<\/th>\nGuten-1<\/th>\nGuten-2<\/th>\n<\/tr>\n
    beautiful<\/b><\/td>\n5,920,000<\/td>\n1,150,000<\/td>\n88,500<\/td>\n44,300<\/td>\n243,000<\/td>\n39,700<\/td>\n2,080<\/td>\n<\/tr>\n
    gorgeous<\/b><\/td>\n1,820,000<\/td>\n101,000<\/td>\n3,270<\/td>\n2,500<\/td>\n40,900<\/td>\n3,750<\/td>\n417<\/td>\n<\/tr>\n
    wonderful<\/b><\/td>\n8,830,000<\/td>\n1,030,000<\/td>\n44,800<\/td>\n42,000<\/td>\n95,800<\/td>\n14,600<\/td>\n1,600<\/td>\n<\/tr>\n
    unique<\/b><\/td>\n6,740,000<\/td>\n2,640,000<\/td>\n439,000<\/td>\n145,000<\/td>\n298,000<\/td>\n3,600<\/td>\n482<\/td>\n<\/tr>\n
    Table 3 – Total hits for content words<\/th>\n<\/tr>\n<\/table>\n

    \n\n\n\n\n\n\n\n
    Strength of collocation<\/th>\nAll Google<\/th>\n.edu<\/th>\nac.uk<\/th>\nedu.au<\/th>\n.jp<\/th>\nGuten-1<\/th>\nGuten-2<\/th>\n<\/tr>\n
    beautiful<\/b><\/td>\n0.073<\/td>\n0.012<\/td>\n0.017<\/td>\n0.028<\/td>\n0.023<\/td>\n0.069<\/td>\n0.172<\/td>\n<\/tr>\n
    gorgeous<\/b><\/td>\n0.006<\/td>\n0.001<\/td>\n0.002<\/td>\n0.004<\/td>\n0.003<\/td>\n0.003<\/td>\n0.000<\/td>\n<\/tr>\n
    wonderful<\/b><\/td>\n0.006<\/td>\n0.002<\/td>\n0.002<\/td>\n0.001<\/td>\n0.010<\/td>\n0.006<\/td>\n0.044<\/td>\n<\/tr>\n
    unique<\/b><\/td>\n0.047<\/td>\n0.004<\/td>\n0.001<\/td>\n0.002<\/td>\n0.007<\/td>\n0.003<\/td>\n0.002<\/td>\n<\/tr>\n
    Table 4 – Strength of collocation<\/th>\n<\/tr>\n<\/table>\n

    [-6-]<\/p>\n

    From these data we can see that “wonderful” collocates as highly with “very” in the Gutenberg texts as it does in the overall Google data, which might indicate that educated speakers are being more careful with this phrase than is warranted by past usage as revealed in the historical literature. On the other hand the number of collocates of “very unique” is one magnitude larger in the total Google corpus than in any of the domains we have investigated, indicating that this usage is indeed being spurned by current educated users. “Very gorgeous” shows the same strength of collocation across all domains studied. <\/p>\n

    Discussion <\/h3>\n

    As a corpus tool, using Google in this way has these drawbacks: <\/p>\n