Affiliation:
1. Qatar Computing Research Institute (QCRI), Doha, Qatar
Abstract
Data profiling comprises a broad range of methods to efficiently analyze a given data set. In a typical scenario, which mirrors the capabilities of commercial data profiling tools, tables of a relational database are scanned to derive metadata, such as data types and value patterns, completeness and uniqueness of columns, keys and foreign keys, and occasionally functional dependencies and association rules. Individual research projects have proposed several additional profiling tasks, such as the discovery of inclusion dependencies or conditional functional dependencies.
Data profiling deserves a fresh look for two reasons: First, the area itself is neither established nor defined in any principled way, despite significant research activity on individual parts in the past. Second, more and more data beyond the traditional relational databases are being created and beg to be profiled. The article proposes new research directions and challenges, including interactive and incremental profiling and profiling heterogeneous and non-relational data.
Publisher
Association for Computing Machinery (ACM)
Subject
Information Systems,Software
Cited by
84 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献
1. Model of a system for forecasting the production of textile fibers;PROCEEDINGS OF THE INTERNATIONAL CONFERENCE ON MATHEMATICAL SCIENCES AND TECHNOLOGY 2022 (MATHTECH 2022): Navigating the Everchanging Norm with Mathematics and Technology;2024
2. A Metadata Reconstruction Algorithm Based on Heterogeneous Sensor Data for Marine Observations;Journal of Ocean University of China;2023-11-28
3. Can Large Language Models Predict Data Correlations from Column Names?;Proceedings of the VLDB Endowment;2023-09
4. Context-aware Big Data Quality Assessment: A Scoping Review;Journal of Data and Information Quality;2023-08-22
5. Fast Algorithm for Embedded Order Dependency Validation;35th International Conference on Scientific and Statistical Database Management;2023-07-10