A Combined Term Extraction Method for the Problem of Monitoring Thematic Discussions in Social Media

Abstract

Term extraction is an important stage in the automated construction of knowledge systems based on natural language texts, since it provides the formation of a basic concept system, which is then used in applied problems of intellectual information processing. The article discusses the problem of automated extraction of terms from natural language texts for their further use in the construction of formalized knowledge systems (ontologies, thesauruses, knowledge graphs) within the problem of monitoring thematic discussions in social media. This problem is characterized by the need to include in the formed knowledge system both concepts from several different domains, and some general concepts used by the audience of social media within thematic discussions. In addition, the generated knowledge system is dynamic both in terms of the composition of the domains it covers and the composition of relevant concepts to be included in the system. The use of existing classical methods for term extraction in this case is difficult, since they are focused on extracting terms within one domain. Based on this, to solve the problem under consideration, a combined method is proposed, combining approaches based on dictionaries, NER tools and rules. The results of the experiments demonstrate the effectiveness of the proposed combination of approaches to term extraction, which makes it possible to extract terms for the problem of monitoring and analyzing thematic discussions in social media. The developed method significantly exceeds the precision of the considered term extraction tools. As a further direction of research, the possibility of developing a method for solving the problem of identifying nested terms or entities is considered.