On Detecting and Removing Superficial Redundancy in Vector Databases-Reference-Cited by-同舟云学术

On Detecting and Removing Superficial Redundancy in Vector Databases

Published:2018 Issue: Volume:2018 Page:1-14
ISSN:1024-123X
Container-title:Mathematical Problems in Engineering
language:en
Short-container-title:Mathematical Problems in Engineering

Author:

DeCastro-García Noemí¹^ORCID,Muñoz Castañeda Ángel Luis²,Fernández Rodríguez Mario²,Carriegos Miguel V.¹

Affiliation:

1. Departamento de Matemáticas, Universidad de León, Campus de Vegazana, s/n, ES-24071 León, Spain

2. Research Institute on Applied Sciences in Cybersecurity, Universidad de León, Campus de Vegazana, s/n, ES-24071 León, Spain

Abstract

A mathematical model is proposed in order to obtain an automatized tool to remove any unnecessary data, to compute the level of the redundancy, and to recover the original and filtered database, at any time of the process, in a vector database. This type of database can be modeled as an oriented directed graph. Thus, the database is characterized by an adjacency matrix. Therefore, a record is no longer a row but a matrix. Then, the problem of cleaning redundancies is addressed from a theoretical point of view. Superficial redundancy is measured and filtered by using the 1-norm of a matrix. Algorithms are presented by Python and MapReduce, and a case study of a real cybersecurity database is performed.

Funder

Spanish National Cybersecurity Institute (INCIBE)

Publisher

Hindawi Limited

Subject

General Engineering,General Mathematics

Link

http://downloads.hindawi.com/journals/mpe/2018/3702808.pdf

Reference12 articles.

1. The Changing Paradigm of Data-Intensive Computing

2. Data-intensive applications, challenges, techniques and technologies: A survey on Big Data

3. Big Data: Promises and Problems

Cited by 8 articles. 订阅此论文施引文献订阅此论文施引文献，注册后可以免费订阅5篇论文的施引文献，订阅后可以查看论文全部施引文献

1. A Brief Survey of Vector Databases;2023 9th International Conference on Big Data and Information Analytics (BigDIA);2023-12-15

2. On directional accuracy of some methods to forecast time series of cybersecurity aggregates;Logic Journal of the IGPL;2022-02-28

3. Measuring the Quality Information of Sources of Cybersecurity by Multi-Criteria Decision Making Techniques;Lecture Notes in Computer Science;2022

4. GDC-a-CGI: efficient algorithms for dynamic graph data cleaning and indexing;International Journal of Computational Science and Engineering;2021

5. On Aggregation and Prediction of Cybersecurity Incident Reports;IEEE Access;2021