Using Shovel to Copy Messages for Testing

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Using Shovel to Copy Messages for Testing

David Legg
Hi All,

I'd like to get everyone's opinion on how best to do something.

At the moment I have a requirement to copy messages verbatim from a queue on a RabbitMQ server to another, primarily for testing and troubleshooting purposes. The messages on the original RabbitMQ server must remain intact and copied before any other consumer running on the queue deals with and acknowledges them. I'm looking at the Shovel plugin at the moment to do this, but can anyone see any pitfalls?

Thanks in advance,

David

_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Multiple Bindings to the Default Exchange

David Legg
OK, so no opinions there.

Things are somewhat complicated because there are external publishers which I cannot change that publish to the default queue with the queue name as the routing key. If they were publishing to the direct exchange I could bind another queue with the same routing key and route messages to two queues. Is there any possible way of doing this with the default exchange as I haven't found one so far?


From: David Legg <[hidden email]>
Date: Thursday, 20 June 2013 14:57
To: "[hidden email]" <[hidden email]>
Subject: Using Shovel to Copy Messages for Testing

Hi All,

I'd like to get everyone's opinion on how best to do something.

At the moment I have a requirement to copy messages verbatim from a queue on a RabbitMQ server to another, primarily for testing and troubleshooting purposes. The messages on the original RabbitMQ server must remain intact and copied before any other consumer running on the queue deals with and acknowledges them. I'm looking at the Shovel plugin at the moment to do this, but can anyone see any pitfalls?

Thanks in advance,

David

_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Multiple Bindings to the Default Exchange

michaelklishin

2013/6/24 David Legg <[hidden email]>
If they were publishing to the direct exchange I could bind another queue with the same routing key and route messages to two queues. Is there any possible way of doing this with the default exchange as I haven't found one so far?

The default exchange is a direct exchange.
--
MK

http://github.com/michaelklishin
http://twitter.com/michaelklishin

_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Multiple Bindings to the Default Exchange

David Legg
Yes, I know it's direct. What I want is to bind the same routing key to it in the same way as other exchanges and distribute to multiple queues.

_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Multiple Bindings to the Default Exchange

michaelklishin

2013/6/24 David Legg <[hidden email]>
What I want is to bind the same routing key to it in the same way as other exchanges and distribute to multiple queues

You should be able to do anything you can do with a direct exchange.
--
MK

http://github.com/michaelklishin
http://twitter.com/michaelklishin

_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Multiple Bindings to the Default Exchange

Simon MacMullen-2
On 24/06/13 11:04, Michael Klishin wrote:
> 2013/6/24 David Legg <[hidden email]
> <mailto:[hidden email]>>
>
>     What I want is to bind the same routing key to it in the same way as
>     other exchanges and distribute to multiple queues
>
>
> You should be able to do anything you can do with a direct exchange.

That's not correct if the direct exchange in question is the default
exchange - the default exchange is quite a lot more magical than other
direct exchanges. Really it can be thought of as a way to express
"publish to queue" in an API which assumes publishing to exchanges.

So to the OP, I'm afraid you can't do that.

The best you can do would be use the firehose
http://www.rabbitmq.com/firehose.html to grab the messages as they are
published. But that will rewrite message headers (not bodies). Other
than that you'd need to modify the publisher.

Cheers, Simon

--
Simon MacMullen
RabbitMQ, Pivotal
_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Multiple Bindings to the Default Exchange

David Legg
Yer, I thought so. Thanks Simon.


Yes, it is the default exchange I'm talking about. You can't just
arbitrarily bind a queue to the default exchange with a routing key like
you can with others. Alas, changing the publishers is not going to be
possible right now.

Firehose looks like the only way I'll be able to approach this.


On 24/06/2013 11:11, "Simon MacMullen" <[hidden email]> wrote:

>That's not correct if the direct exchange in question is the default
>exchange - the default exchange is quite a lot more magical than other
>direct exchanges. Really it can be thought of as a way to express
>"publish to queue" in an API which assumes publishing to exchanges.
>
>So to the OP, I'm afraid you can't do that.
>
>The best you can do would be use the firehose
>http://www.rabbitmq.com/firehose.html to grab the messages as they are
>published. But that will rewrite message headers (not bodies). Other
>than that you'd need to modify the publisher.
>
>Cheers, Simon
>
>--
>Simon MacMullen
>RabbitMQ, Pivotal

_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Multiple Bindings to the Default Exchange

Matthias Reik-2
Sticking out my head here, but ...

... if you can't change your publishers, can you change your consumers?
If so, can't you reroute your messages from the default exchange to
another exchange
and do the processing then against that other exchange?

Cheers
Maze


On 2013-06-24 15:35 , David Legg wrote:

> Yer, I thought so. Thanks Simon.
>
>
> Yes, it is the default exchange I'm talking about. You can't just
> arbitrarily bind a queue to the default exchange with a routing key like
> you can with others. Alas, changing the publishers is not going to be
> possible right now.
>
> Firehose looks like the only way I'll be able to approach this.
>
>
> On 24/06/2013 11:11, "Simon MacMullen" <[hidden email]> wrote:
>> That's not correct if the direct exchange in question is the default
>> exchange - the default exchange is quite a lot more magical than other
>> direct exchanges. Really it can be thought of as a way to express
>> "publish to queue" in an API which assumes publishing to exchanges.
>>
>> So to the OP, I'm afraid you can't do that.
>>
>> The best you can do would be use the firehose
>> http://www.rabbitmq.com/firehose.html to grab the messages as they are
>> published. But that will rewrite message headers (not bodies). Other
>> than that you'd need to modify the publisher.
>>
>> Cheers, Simon
>>
>> --
>> Simon MacMullen
>> RabbitMQ, Pivotal
> _______________________________________________
> rabbitmq-discuss mailing list
> [hidden email]
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss

_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss