Scalability with HA cluster for one single queue

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

Scalability with HA cluster for one single queue

This post has NOT been accepted by the mailing list yet.

searching for RabbitMQ scalability that functions like hard disk arrays scalability I search for a simple way to scale one single RabbitMQ queue up in terms of memory allocation (space) in a HA Cluster. The scalability should work the following:

I add another server to the cluster. The queue should replicate partly to this new server and scale up in terms of memory size at the same time. Therefore with e.g. 4 servers in the cluster each server should keep e.g. 50% of the total amount of queue data and not the 100% replicated usually in the ha mode of RabbitMQ.

Is there a way to achieve this kind of scalability similar to disk arrays where you increase storage size and have failover replication at the same time with new disks added to the array? I search for a method where I don't have to spread lots of queues to the cluster nodes replicating on different servers. It should work with one single queue.

The couchbase database server for example provides such scalability too.

Kind regards,