Building a Large Corpus and Pre-trained Language Models from National and Local Assembly Minutes-Reference-Cited by-同舟云学术

Building a Large Corpus and Pre-trained Language Models from National and Local Assembly Minutes

Published:2024 Issue:2 Volume:31 Page:707-732
ISSN:1340-7619
Container-title:Journal of Natural Language Processing
language:en
Short-container-title:Journal of Natural Language Processing

Author:

Nagafuchi Keiyu¹,Kimura Yasutomo²³,Kadowaki Kazuma⁴,Araki Kenji⁵

Affiliation:

1. Graduate School of Information Science and Technology, Hokkaido University

2. Otaru University of Commerce

3. Center for Advanced Intelligence Project, RIKEN

4. The Japan Research Institute, Limited

5. Faculty of Information Science and Technology, Hokkaido University

Publisher

Association for Natural Language Processing

Link

https://www.jstage.jst.go.jp/article/jnlp/31/2/31_707/_pdf

Reference32 articles.

1. Araci, D. (2019). “FinBERT: Financial Sentiment Analysis with Pre-trained Language Models.”.

2. Baevski, A., Edunov, S., Liu, Y., Zettlemoyer, L., and Auli, M. (2019). “Cloze-driven Pretraining of Self-attention Networks.” In Inui, K., Jiang, J., Ng, V., and Wan, X. (Eds.), Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 5360–5369, Hong Kong, China. Association for Computational Linguistics.

3. Beltagy, I., Lo, K., and Cohan, A. (2019). “SciBERT: A Pretrained Language Model for Scientific Text.” In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 3615–3620.

4. Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., and Amodei, D. (2020). “Language Models are Few-Shot Learners.” In Advances in Neural Information Processing Systems, Vol. 33, pp. 1877–1901. Curran Associates, Inc.

5. Chalkidis, I., Fergadiotis, M., Malakasiotis, P., Aletras, N., and Androutsopoulos, I. (2020). “LEGAL-BERT: The Muppets Straight Out of Law School.” In Findings of the Association for Computational Linguistics: EMNLP 2020, pp. 2898–2904.