Using the Web to Obtain Frequencies for Unseen Bigrams-Reference-Cited by-同舟云学术

Using the Web to Obtain Frequencies for Unseen Bigrams

Published:2003-09 Issue:3 Volume:29 Page:459-484
ISSN:0891-2017
Container-title:Computational Linguistics
language:en
Short-container-title:Computational Linguistics

Author:

Keller Frank¹,Lapata Mirella²

Affiliation:

1. University of Edinburgh, School of Informatics, 2 Buccleuch Place, Edinburgh EH8 9LW, UK.

2. University of Sheffield, Department of Computer Science, 211 Portobello Street, Sheffield S1 4DP, UK.

Abstract

This article shows that the Web can be employed to obtain frequencies for bigrams that are unseen in a given corpus. We describe a method for retrieving counts for adjective-noun, noun-noun, and verb-object bigrams from the Web by querying a search engine. We evaluate this method by demonstrating: (a) a high correlation between Web frequencies and corpus frequencies; (b) a reliable correlation between Web frequencies and plausibility judgments; (c) a reliable correlation between Web frequencies and frequencies recreated using class-based smoothing; (d) a good performance of Web frequencies in a pseudo disambiguation task.

Publisher

MIT Press - Journals

Subject

Artificial Intelligence,Computer Science Applications,Linguistics and Language,Language and Linguistics

Link

https://www.mitpressjournals.org/doi/pdf/10.1162/089120103322711604

Reference11 articles.

1. Magnitude Estimation of Linguistic Acceptability

2. Class-Based Probability Estimation Using a Semantic Hierarchy

3. Syntactic priming in English sentence production: Categorical and latency evidence from an Internet-based study

Cited by 129 articles. 订阅此论文施引文献订阅此论文施引文献，注册后可以免费订阅5篇论文的施引文献，订阅后可以查看论文全部施引文献

1. A Study on the Semantic Interpretation of Chinese Noun Compounds;Chinese Language Resources;2023

2. 9. Predicting Parameters of Variation in the Use of Academic Multiword Expressions in University Student Writing;Vocabulary Theory, Patterning and Teaching;2022-12-31

3. Chapter 3. Medical topics and style from 1500 to 2018;Pragmatics & Beyond New Series;2022-06-15

4. Quantifying Context With and Without Statistical Language Models;Handbook of Cognitive Mathematics;2022

5. An analysis of the writing of ‘suicide cult’ members;Digital Scholarship in the Humanities;2021-06-03