Automatic Evaluation and Comparison of Pub/Sub Systems Performance Improvements

Abstract

Event-driven architectures are becoming more prevalent recently in multiple technological paradigms, especially in web applications, with message brokers being the cornerstone of these architectures. One of the most relevant implementations of these message brokers are content-based publish/subscribe systems. The performance of these systems is a critical factor for web engineering, since the web applications they support need to be reactive despite increases and fluctuations in workloads. However, an obstacle to the research of these systems is the lack of real and publicly available workloads, due to the privacy issue involved in disclosing the interests (subscriptions) of users and other commercial interests of the companies. In this paper we present a parameterizable automated system designed to syntactically translate workloads from different content-based publish/subscribe systems as a means to increase the availability of public workloads to solve the aforementioned problem. As a case study, we describe the evolution of a context-aware content-based publish/subscribe system (i.e. E-SilboPS) designed by the authors, which improves up to 5 times the performance of its previous version by reaching the maximum throughput limited by the physical resources of the hardware where it is deployed, as demonstrated by the conducted quantitative evaluation. Then, we validate the utility of the proposed automated workload generation system by using it to make the performance comparison between this new version E-SilboPS and one of the most cited publish/subscribe systems called PADRES, through a real trace of a massively multiplayer online game (MMOG) generated by the latter.