Cpartition: A correlation-based space partitioning for content-based publish/subscribe systems with skewed workload

Abstract

It is difficult to develop a messaging system that processes live content efficiently. Real-world workload is skewed because users of a publish/subscribe (pub/sub) system use only a small portion of the entire contents. For this reason, the distribution of subscriptions within the content space is seriously imbalanced, and events are not processed efficiently. In this study, we propose CPartition, a correlation-based content space partitioning technique for alleviating load imbalance caused by skewed subscription workload in a content-based pub/sub system. This work aims to assign attributes that have low correlation to the same dimension group. By doing so, the balance between the number of subscriptions among brokers can be improved by scattering the linearly distributed subscriptions to many more subspaces. We have implemented the content-based pub/sub system for evaluation. The evaluation demonstrates a load balance comparison between CPartition and existing methods. Under various configurations of partitioning granularity, the experimental results reveal that CPartition outperforms the existing methods on both skewed subscription workload and balanced subscription workload.