Language models align with human judgments on key grammatical constructions-Reference-Cited by-同舟云学术

Language models align with human judgments on key grammatical constructions

Published:2024-08-26 Issue:36 Volume:121 Page:
ISSN:0027-8424
Container-title:Proceedings of the National Academy of Sciences
language:en
Short-container-title:Proc. Natl. Acad. Sci. U.S.A.

Author:

Hu Jennifer¹,Mahowald Kyle²,Lupyan Gary³^ORCID,Ivanova Anna⁴^ORCID,Levy Roger⁵^ORCID

Affiliation:

1. Kempner Institute for the Study of Natural and Artificial Intelligence, Harvard University, Cambridge, MA 02138

2. Department of Linguistics, The University of Texas at Austin, Austin, TX 78712

3. Department of Psychology, University of Wisconsin-Madison, Madison, WI 53706

4. School of Psychology, Georgia Tech, Atlanta, GA 30332

5. Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA 02139

Publisher

Proceedings of the National Academy of Sciences

Link

https://pnas.org/doi/pdf/10.1073/pnas.2400917121

Reference10 articles.

1. Systematic testing of three Language Models reveals low language accuracy, absence of response stability, and a yes-response bias

2. Metalinguistic Performance and Interlinguistic Competence

3. Y. Han “Grammaticality judgment tests: How reliable and valid are they?” in Applied Language Learning L. Woytak Ed. (Defense Language Institute Foreign Language Center 2000) vol. 11 pp. 177–204.

4. N. Chomsky, Knowledge of Language: Its Nature, Origin, and Use (Praeger Scientific, 1986).

5. R. Marvin T. Linzen “Targeted syntactic evaluation of language models” in Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing E. Riloff D. Chiang J. Hockenmaier J. Tsujii Eds. (Association for Computational Linguistics Brussels Belgium 2018) pp. 1192–1202.

Cited by 1 articles. 订阅此论文施引文献订阅此论文施引文献，注册后可以免费订阅5篇论文的施引文献，订阅后可以查看论文全部施引文献

1. Reply to Hu et al.: Applying different evaluation standards to humans vs. Large Language Models overestimates AI performance;Proceedings of the National Academy of Sciences;2024-08-26