Affiliation:
1. IBM Almaden Research Center, San Jose, CA
2. Inria Saclay and Université Paris-Sud Orsay, France
3. Inria Saclay and Université, Paris-Sud Orsay, France
Abstract
In content-based publish-subscribe (pub/sub) systems, users express their interests as queries over a stream of publications. Scaling up content-based pub/sub to very large numbers of subscriptions is challenging: users are interested in low
latency
, that is, getting subscription results fast, while the pub/sub system provider is mostly interested in
scaling
, i.e., being able to serve large numbers of subscribers, with low
computational resources utilization.
We present a novel approach for scalable content-based pub/sub in the presence of constraints on the available CPU and network resources, implemented within our pub/sub system Delta. We achieve scalability by off-loading some subscriptions from the pub/sub server, and leveraging view-based query rewriting to feed these subscriptions from the data accumulated in others. Our main contribution is a novel algorithm for organizing views in a multi-level dissemination network, exploiting view-based rewriting and powerful linear programming capabilities to scale to many views, respect capacity constraints, and minimize latency. The efficiency and effectiveness of our algorithm are confirmed through extensive experiments and a large deployment in a WAN.
Subject
General Earth and Planetary Sciences,Water Science and Technology,Geography, Planning and Development
Cited by
8 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献
1. Optimizing Data Pipelines for Machine Learning in Feature Stores;Proceedings of the VLDB Endowment;2023-09
2. Graphsurge;Proceedings of the 2021 International Conference on Management of Data;2021-06-09
3. Materialized View Maintenance: Issues, Classification, and Open Challenges;International Journal of Cooperative Information Systems;2019-03
4. Data Stream Management;Real-Time & Stream Data Management;2019
5. Selecting subexpressions to materialize at datacenter scale;Proceedings of the VLDB Endowment;2018-03