Alternative to "immediate" in RabbitMQ 3.x

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Alternative to "immediate" in RabbitMQ 3.x

Chris-3
Hi All,

We have a perfect use case for the immediate flag now, but just discovered it was deprecated in RabbitMQ 3.x.  Oops.

We considered the suggested alternative of using a message TTL of 0 and dead-letter exchanges, but ran into these two issues:
  • If x-message-ttl is set to 0 on the queue, then what should be a message-specific option now effects every message on the queue.  This isn't acceptable for us.
  • If expiration is set to 0 on a message, the message will not be dead-lettered until it gets to the head of the queue (which could be a long time).
We need to support scenarios where the publisher chooses on a message-by-message basis if the message should be "immediate".  

The best I can come up with is splitting every queue into two queues now-- one that has x-message-ttl set to 0 and one that doesn't, and each consumer would need to consume from both queues.  And publishers would need to specify different routing keys based on if they want "immediate" or not...  I don't really like this idea much.

Question #1: Is there some other way of doing it that I'm not thinking of?

Question #2: Has there been any thought to supporting "immediate" again?  Or is it definitely a thing of the past?

Thanks!
Chris

_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Alternative to "immediate" in RabbitMQ 3.x

Michael Klishin-2
On 21 Oct 2013, at 23:52, Chris <[hidden email]> wrote:

> Question #1: Is there some other way of doing it that I'm not thinking of?

This is basically a presence problem. I'd recommend using a coordination
service such as etcd or ZooKeeper, or at least a K/V store like Redis, when
you need presence or coordination between publishers and consumers.

MK



_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Alternative to "immediate" in RabbitMQ 3.x

Chris-3
Hi Michael,

It's a little more than presence.  It is not enough to know the service is running-- I need to know if it can handle a message right now.  So I guess I would need to use a solution to keep track not only of presence but of capacity-- which might be hard to do in a transactional manner.

Still-- your point is taken and may be the best solution given the requirements and capabilities.

Thanks,
Chris


On Mon, Oct 21, 2013 at 4:24 PM, Michael Klishin <[hidden email]> wrote:
On 21 Oct 2013, at 23:52, Chris <[hidden email]> wrote:

> Question #1: Is there some other way of doing it that I'm not thinking of?

This is basically a presence problem. I'd recommend using a coordination
service such as etcd or ZooKeeper, or at least a K/V store like Redis, when
you need presence or coordination between publishers and consumers.

MK



_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss


_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Alternative to "immediate" in RabbitMQ 3.x

Matthias Radestock-3
On 21/10/13 22:19, Chris wrote:
> It's a little more than presence.  It is not enough to know the service
> is running-- I need to know if it can handle a message right now.  So I
> guess I would need to use a solution to keep track not only of presence
> but of capacity-- which might be hard to do in a transactional manner.
>
> Still-- your point is taken and may be the best solution given the
> requirements and capabilities.

I actually think the two-queues solution you proposed would work well.

Matthias.
_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Alternative to "immediate" in RabbitMQ 3.x

Tim Watson-5
In reply to this post by Chris-3
On 21 Oct 2013, at 20:52, Chris <[hidden email]> wrote:
  • If x-message-ttl is set to 0 on the queue, then what should be a message-specific option now effects every message on the queue.  This isn't acceptable for us.
That's only true for per queue TTL - you can still set per message TTL instead.
  • If expiration is set to 0 on a message, the message will not be dead-lettered until it gets to the head of the queue (which could be a long time).
This however, is true. It's not clear that we could expire messages that aren't at the head of the queue whilst maintaining acceptable levels of per queue throughput, which is why that behaviour remains ATM.

Question #2: Has there been any thought to supporting "immediate" again?  Or is it definitely a thing of the past?

There aren't any plans to reintroduce it at the moment.

Cheers,
Tim

_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss