Max messages allowed in a queue in RabbitMQ?

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Max messages allowed in a queue in RabbitMQ?

mag8
This post was updated on .
Can anyone provide me with answers for my below questions:

1) Max queues allowed in RabbitMQ?
2) Max messages allowed in a queue in RabbitMQ?
3) Max size of a message in a queue?
4) Does broker creates separate queue for each client? Or just 1 queue until the queue becomes full?
5) How the new queue being created? is it auto created?
6) Does the queue implementing any kind of buffer such as circular buffer?

Basically i am trying to find answers for any opensource MQs. Not only RabbitMQ.
Please explain.
Reply | Threaded
Open this post in threaded view
|

Re: Max messages allowed in a queue in RabbitMQ?

Tim Watson-6
Hi,

On 18 Apr 2013, at 04:30, mag8 wrote:

> Can anyone provide me with answers for my below questions:
>

Each of these will be limited by some combination of the available memory, disk space and the level of producer/consumer throughput you're able to achieve as they're combined in various ways. There are no system enforced limits. You might like to look at http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/2009-February/003042.html and

> 1) Max queues allowed in RabbitMQ?

Thousands (or even tens of thousands) of queues should be no problem at all, though each object (e.g., queues, exchanges, bindings, etc) will take up some memory and/or disk space. By default, Erlang will enforce a maximum number of concurrent processes (i.e., lightweight threads) at around 32768 IIRC. Each queue is managed by its own process and each connection can result in several more, so if you're planning on having a very large number of active queues in a single node (?) and using them all at the same time, then you may need to tweak the emulator arguments rabbit passes the VM by setting +P <new limit> to a higher limit.

You're also likely to use up many Gb just with the overhead for each queue / connection pretty fast, so you're going to need a pretty meaty server to handle millions of both. Tens of thousands should be no problem at all, providing they fit into RAM.

> 2) Max messages allowed in a queue in RabbitMQ?

There is no limit to the number of messages in a queue (and we frequently see hundreds of thousands and even millions of enqueued messages discussed on this mailing list), although each message will take up some amount of memory (which varies depending on whether the message is persistent or transient, whether or not it has been paged to disk anyway to prevent memory overload, etc) and queue response times may vary depending on size.

> 3) Max size of a message in a queue?
>


Although there is no max message size enforced by rabbit, messages *must* be able to fit into the available memory. Using 32-bit Erlang, 536870911 bytes is maximum binary size, which with 64-bit it's 2305843009213693951 bytes. This seems unlikely to bite you (no pun intended), and you're likely to bump into other system limits first I would've thought.

Cheers,
Tim
_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Max messages allowed in a queue in RabbitMQ?

Tim Watson-6
Oh and...

On 18 Apr 2013, at 09:40, Tim Watson wrote:
> you may need to tweak the emulator arguments rabbit passes the VM by setting +P <new limit> to a higher limit.
>

We set this to 1048576 already in our startup scripts, so you're unlikely to hit that. :)
_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Max messages allowed in a queue in RabbitMQ?

Jesper Louis Andersen
This has also been bumped in R16B as well since the allocation is more dynamic now than it were.


On Thu, Apr 18, 2013 at 10:51 AM, Tim Watson <[hidden email]> wrote:
Oh and...

On 18 Apr 2013, at 09:40, Tim Watson wrote:
> you may need to tweak the emulator arguments rabbit passes the VM by setting +P <new limit> to a higher limit.
>

We set this to 1048576 already in our startup scripts, so you're unlikely to hit that. :)
_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss



--
J.

_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Max messages allowed in a queue in RabbitMQ?

Tim Watson-6
On 18 Apr 2013, at 10:51, Jesper Louis Andersen wrote:
> This has also been bumped in R16B as well since the allocation is more dynamic now than it were.
>

That's good to know - thanks!
_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss