Affiliation:
1. University of Washington
2. INRIA Roquencourt
Abstract
Query processing in data integration occurs over network-bound, autonomous data sources. This requires extensions to traditional optimization and execution techniques for three reasons: there is an absence of quality statistics about the data, data transfer rates are unpredictable and bursty, and slow or unavailable data sources can often be replaced by overlapping or mirrored sources. This paper presents the Tukwila data integration system, designed to support adaptivity at its core using a two-pronged approach. Interleaved planning and execution with partial optimization allows Tukwila to quickly recover from decisions based on inaccurate estimates. During execution, Tukwila uses adaptive query operators such as the double pipelined hash join, which produces answers quickly, and the dynamic collector, which robustly and efficiently computes unions across overlapping data sources. We demonstrate that the Tukwila architecture extends previous innovations in adaptive execution (such as query scrambling, mid-execution re-optimization, and choose nodes), and we present experimental evidence that our techniques result in behavior desirable for a data integration system.
Publisher
Association for Computing Machinery (ACM)
Subject
Information Systems,Software
Reference27 articles.
1. Query caching and optimization in distributed mediator systems
2. Query processing and optimization in Oracle Rdb
3. Y. Arens C. A. Knoblock and W.-M. Shen. Query reformulation for dynamic information integration. /nternationat Journal on Intelligent and Cooperative Information System ~ (6) 2/3:99-130 June 1996. 10.1007/BF00122124 Y. Arens C. A. Knoblock and W.-M. Shen. Query reformulation for dynamic information integration. /nternationat Journal on Intelligent and Cooperative Information System ~ (6) 2/3:99-130 June 1996. 10.1007/BF00122124
4. Data access for the masses through OLE DB
5. Memory-adaptive scheduling for large query execution
Cited by
94 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献