|
I've developed an RPC system using RabbitMQ (with different semantics than the built-in RPC). I'd like to get the server to send the client some kind of a notification if a message it has sent will never be delivered (like if the queue which it was routed to was deleted before the message was consumed from it). Is there any way to accomplish this? Client side timeouts are not good enough either because the request might get processed at some point then anyway.
_______________________________________________ rabbitmq-discuss mailing list [hidden email] https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss |
|
Hi,
If your clients publish messages with the mandatory flag set, they will get a Basic.Return from the server if the message can't be routed to a queue. Does that solve your problem? Java example: http://www.rabbitmq.com/api-guide.html#returning Cheers, Valentin. On 2 déc, 12:33, Alex Grönholm <[hidden email]> wrote: > I've developed an RPC system using RabbitMQ (with different semantics than > the built-in RPC). I'd like to get the server to send the client some kind > of a notification if a message it has sent will never be delivered (like if > the queue which it was routed to was deleted before the message was > consumed from it). Is there any way to accomplish this? Client side > timeouts are not good enough either because the request might get processed > at some point then anyway. > > _______________________________________________ > rabbitmq-discuss mailing list > [hidden email]://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss rabbitmq-discuss mailing list [hidden email] https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss |
|
If on the other hand Alex is interested in learning about messages that
have been deleted once they've made it to queues then there's currently no way to find out about that. We are currently looking at implementing dead lettering, which might help with this. Cheers, Simon On 02/12/11 12:12, Valentin BERNARD wrote: > Hi, > > If your clients publish messages with the mandatory flag set, they > will get a Basic.Return from the server if the message can't be routed > to a queue. Does that solve your problem? > > Java example: http://www.rabbitmq.com/api-guide.html#returning > > Cheers, > > Valentin. > > On 2 déc, 12:33, Alex Grönholm<[hidden email]> wrote: >> I've developed an RPC system using RabbitMQ (with different semantics than >> the built-in RPC). I'd like to get the server to send the client some kind >> of a notification if a message it has sent will never be delivered (like if >> the queue which it was routed to was deleted before the message was >> consumed from it). Is there any way to accomplish this? Client side >> timeouts are not good enough either because the request might get processed >> at some point then anyway. >> >> _______________________________________________ >> rabbitmq-discuss mailing list >> [hidden email]://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss > _______________________________________________ > rabbitmq-discuss mailing list > [hidden email] > https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss -- Simon MacMullen RabbitMQ, VMware _______________________________________________ rabbitmq-discuss mailing list [hidden email] https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss |
|
In reply to this post by Valentin Bernard
That only happens if the queue is not there to begin with. If the queue is there and it expires, no return message is ever generated.
_______________________________________________ rabbitmq-discuss mailing list [hidden email] https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss |
|
In reply to this post by Simon MacMullen-2
Yes, that is exactly my problem. Seems there is no solution then :(
I want to keep my "rpc" queue alive even when the RPC server software is being upgraded so that users will only see a delay in the responses (and no lost requests). _______________________________________________ rabbitmq-discuss mailing list [hidden email] https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss |
|
On 02/12/11 12:36, Alex Grönholm wrote:
> I want to keep my "rpc" queue alive even when the RPC server software is > being upgraded so that users will only see a delay in the responses (and > no lost requests). So just make sure the rpc queue stays around during the upgrade. Matthias. _______________________________________________ rabbitmq-discuss mailing list [hidden email] https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss |
|
That was the plan from the start, but I wanted to cover cases where something goes wrong (server starts with rabbitmq but not the rpc server) in which case the client would be stuck waiting forever.
But I guess that dead lettering feature will fix that. _______________________________________________ rabbitmq-discuss mailing list [hidden email] https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss |
|
In reply to this post by Matthias Radestock-3
Dear Alex,
Just to be precise about this: the 'standard' RpcServer uses a passed in queue or creates its queue as follows: queueDeclare() which is equivalent to: queueDeclare("", false, true, true, null) which creates a non-durable, exclusive, auto-delete queue, named by the broker. If you want your queue to survive the connection, it mustn't be exclusive to the connection that creates it. In order to survive a server failure you would make it durable. To re-use the same queue later (after upgrade?), you had better name it yourself (and re-use the name each time). If the queue should persist after the last consumer (RpcServer?) is cancelled from the queue, then it should not be auto-delete. Does this give you the flexibility you require? I do not know your situation exactly, but it looks as though you need a named non-exclusive queue, at the very least. If you expect each server instance to be solely responsible for the request queue it serves, then a simple naming scheme will suffice, but you still have to be careful to have at most one instance of that server running at any one time. Of course, having more than one server processing requests from a queue is alright, too; if you are happy with way requests are shared out. Steve Powell [hidden email] [wrk: +44-2380-111-528] [mob: +44-7815-838-558] On 2 Dec 2011, at 12:44, Matthias Radestock wrote: > On 02/12/11 12:36, Alex Grönholm wrote: >> I want to keep my "rpc" queue alive even when the RPC server software is >> being upgraded so that users will only see a delay in the responses (and >> no lost requests). > > So just make sure the rpc queue stays around during the upgrade. > > Matthias. > _______________________________________________ > rabbitmq-discuss mailing list > [hidden email] > https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss _______________________________________________ rabbitmq-discuss mailing list [hidden email] https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss |
|
In reply to this post by Alex Grönholm-3
This is where the "mandatory" feature comes in.
1. Declare your queue as durable. 2. Send your RPC requests with "mandatory" flag. Then:
Regards, On 2 December 2011 09:21, Alex Grönholm <[hidden email]> wrote: That was the plan from the start, but I wanted to cover cases where something goes wrong (server starts with rabbitmq but not the rpc server) in which case the client would be stuck waiting forever. _______________________________________________ rabbitmq-discuss mailing list [hidden email] https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss |
if rabbitmq is up, but the RPC server is down, and the queue was previously created by somebody, then the request will be enqueued. As long as the RPC server returns to service in a timely way, everyone's happy. And if it doesn't? How can I get the request cancelled then? I'm not sure I should rely on client generated timestamps. Their clocks could be wrong. _______________________________________________ rabbitmq-discuss mailing list [hidden email] https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss |
|
In reply to this post by Steve Powell-2
Dear Steve,
I know how to create queues properly. That was never the issue. The issue was that if I created a leased queue, take the consumer offline and leave it that way and the queue expires, the client is never notified that the message was dropped and it will continue to wait for a response forever. That is not how I want it to work. I expect the server to send some return message when the queue dies. The mandatory flag does not help since it only returns the message if it's not routed to a queue (which it is). But like I said before, the dead lettering feature should help with this. _______________________________________________ rabbitmq-discuss mailing list [hidden email] https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss |
|
In reply to this post by Alex Grönholm-3
On 4 December 2011 21:45, Alex Grönholm <[hidden email]> wrote:
_______________________________________________ rabbitmq-discuss mailing list [hidden email] https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss |
|
In reply to this post by Alex Grönholm-3
Sorry Alex, I'm sure you do. I'm learning and trying to understand your
situation. Although it is reasonable to create a "leased" queue (with an 'x-expires' argument) for replies to Rpc requests (on the grounds that the clients cannot be replied upon to drain them), I do not think it reasonable to do this for the server queues unless you can tolerate requests being (quietly) lost. If you don't want the server queue requests to be 'lost' then don't allow the queues to expire! On the other hand your clients apparently wait forever. Is there a reason I don't understand why the server queues expire but the clients don't have a timeout for the responses? Well, I think I can answer my own question -- please confirm if this is right: You want requests to either be processed or (reliably) not processed. You never want the clients to be in any doubt about whether the request was processed or not. So you wait forever in the client. However, requests that get discarded never return so the client doesn't know. So you want expiry to trigger replies to the clients. (Per-queue message TTLs also don't return.) If the server dies there is still the possibility that it did so after pulling the request from the queue and before posting a reply -- unless the requests are 'transactionally' processed. So it looks like you need the request processing to be a transaction so that all requests not in the queue (when it expires) have definitely been processed. Let's see -- how would we do this without a dead-letter mechanism? Well, apart from making the server transactional, it might be useful to have an 'offline' server that can take over any server's queue when it fails and either deal with the requests [by issuing an 'offline' response?] or arrange for the server to be restarted (and/or upgraded). In one of your original posts: > "I want to keep my "rpc" queue alive even when the RPC server software is > being upgraded so that users will only see a delay in the responses (and no > lost requests)" it looks like you could actually schedule a takeover of the 'offline' server from the RpcServer, and have complete control of the policy for handling requests in 'dead' server queues. A dead-letter mechanism would be convenient in this case, because you would not then have to 'take-over' any queues and the 'offline' server could be run permanently, without a need to monitor the servers for failure. Have I understood now? Steve Powell (a learning bunny) ----------some more definitions from the SPD---------- avoirdupois (phr.) 'Would you like peas with that?' distribute (v.) To denigrate an award ceremony. definite (phr.) 'It's hard of hearing, I think.' modest (n.) The most mod. On 5 Dec 2011, at 02:48, Alex Grönholm wrote: > Dear Steve, > > I know how to create queues properly. That was never the issue. The issue was that if I created a leased queue, take the consumer offline and leave it that way and the queue expires, the client is never notified that the message was dropped and it will continue to wait for a response forever. That is not how I want it to work. I expect the server to send some return message when the queue dies. The mandatory flag does not help since it only returns the message if it's not routed to a queue (which it is). But like I said before, the dead lettering feature should help with this. > _______________________________________________ > rabbitmq-discuss mailing list > [hidden email] > https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss _______________________________________________ rabbitmq-discuss mailing list [hidden email] https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss |
|
Sorry for taking this long to respond. I've been otherwise engaged and forgot about this conversation. Thank you for your replies. If the upgrade of the server software fails and the developers do nothing, the queue expires eventually so that stale requests are dropped and the clients will start getting proper error messages informing them of this condition. This is not the ideal solution of course (per-message TTL and return notification would be). You want requests to either be processed or (reliably) not processed. You Sounds about right. Let's see -- how would we do this without a dead-letter mechanism? Well, apart This seems fairly complicated. I'll have to look into transactions to see if they will help in my case. _______________________________________________ rabbitmq-discuss mailing list [hidden email] https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss |
|
11.12.2011 14:14, Alex Grönholm kirjoitti:
Now that the dead-letter mechanism is part of the server, I don't still don't see how this could be done. The dead lettered RPC commands would still have to be processed somewhere.
_______________________________________________ rabbitmq-discuss mailing list [hidden email] https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss |
|
Alex,
On 26/04/12 06:08, Alex Grönholm wrote: >> You want requests to either be processed or (reliably) not >> processed. You never want the clients to be in any doubt about >> whether the request was processed or not. So you wait forever in >> the client. However, requests that get discarded never return so >> the client doesn't know. So you want expiry to trigger replies to >> the clients. (Per-queue message TTLs also don't return.) >> >> >> Sounds about right. >> >> Let's see -- how would we do this without a dead-letter mechanism? >> > Now that the dead-letter mechanism is part of the server, I don't still > don't see how this could be done. The dead lettered RPC commands would > still have to be processed somewhere. Right. Ideally you'd need a way to specify a dead-letter destination *per message*. However, working with what is possible today, instead you could just have a single "dead request" queue to which all expired requests get routed. And then have a small app simply consuming messages from that queue and replying to them with some sort of error indication. Regards, Matthias. _______________________________________________ rabbitmq-discuss mailing list [hidden email] https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss |
|
30.04.2012 19:59, Matthias Radestock kirjoitti:
> Alex, > > On 26/04/12 06:08, Alex Grönholm wrote: >>> You want requests to either be processed or (reliably) not >>> processed. You never want the clients to be in any doubt about >>> whether the request was processed or not. So you wait forever in >>> the client. However, requests that get discarded never return so >>> the client doesn't know. So you want expiry to trigger replies to >>> the clients. (Per-queue message TTLs also don't return.) >> >>> >>> Sounds about right. >>> >>> Let's see -- how would we do this without a dead-letter mechanism? >> >> Now that the dead-letter mechanism is part of the server, I don't still >> don't see how this could be done. The dead lettered RPC commands would >> still have to be processed somewhere. > > Right. Ideally you'd need a way to specify a dead-letter destination > *per message*. However, working with what is possible today, instead > you could just have a single "dead request" queue to which all expired > requests get routed. And then have a small app simply consuming > messages from that queue and replying to them with some sort of error > indication. app, why not the real one too while I'm at it? Thanks for the answer, but is there any hope for a real solution? Could this be cleanly done with a plugin? > > Regards, > > Matthias. _______________________________________________ rabbitmq-discuss mailing list [hidden email] https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss |
|
Alex,
On 01/05/12 23:38, Alex Grönholm wrote: > 30.04.2012 19:59, Matthias Radestock kirjoitti: >> Right. Ideally you'd need a way to specify a dead-letter destination >> *per message*. However, working with what is possible today, instead >> you could just have a single "dead request" queue to which all expired >> requests get routed. And then have a small app simply consuming >> messages from that queue and replying to them with some sort of error >> indication. > That did occur to me, but if I was able to launch an "answering machine" > app, why not the real one too while I'm at it? > > Thanks for the answer, but is there any hope for a real solution? Could > this be cleanly done with a plugin? Yep. It could be done cleanly with a plugin that employs the Erlang client in 'direct' mode to implement said "answering machine" app. Of course if you take that approach to the extreme then you'd write your *entire* app as a plugin ;) Matthias. _______________________________________________ rabbitmq-discuss mailing list [hidden email] https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss |
|
02.05.2012 08:26, Matthias Radestock kirjoitti:
> Alex, > > On 01/05/12 23:38, Alex Grönholm wrote: >> 30.04.2012 19:59, Matthias Radestock kirjoitti: >>> Right. Ideally you'd need a way to specify a dead-letter destination >>> *per message*. However, working with what is possible today, instead >>> you could just have a single "dead request" queue to which all expired >>> requests get routed. And then have a small app simply consuming >>> messages from that queue and replying to them with some sort of error >>> indication. >> That did occur to me, but if I was able to launch an "answering machine" >> app, why not the real one too while I'm at it? >> >> Thanks for the answer, but is there any hope for a real solution? Could >> this be cleanly done with a plugin? > > Yep. It could be done cleanly with a plugin that employs the Erlang > client in 'direct' mode to implement said "answering machine" app. > > Of course if you take that approach to the extreme then you'd write > your *entire* app as a plugin ;) > > Matthias. the ideas and patterns it employs, Python's syntax still wins big time for its power and efficiency. _______________________________________________ rabbitmq-discuss mailing list [hidden email] https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss |
| Powered by Nabble | Edit this page |
