Check Queues Exist on Publish (and Read?)

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Check Queues Exist on Publish (and Read?)

Patrick Long
RabbitMQ 3.2.4
ErLang R16B02

Using BasicPublish and BasicGet

I made some changes to how we use RabbitMQ recently that included only checking that queues and exchanges existed when we Publish. I did this for performance reasons.

On the whole the 5 queues we use will exist in production due to having had messages published.

However in test environments that is not always there. The Windows Service we have that reads messages (BasicGet) is configured to Read from all 5 queues.

If the queue is not there i get a soft error.
=ERROR REPORT==== 23-May-2014::16:21:32 ===
connection <0.12152.0>, channel 12 - soft error:
{amqp_error,not_found,
            "no queue 'Work_3.0.0185.01_Normal' in vhost 's2_stackatf'",
            'basic.get'}


1. Is there a way to tell RabbitMQ to chill out about queues not being found?

2. Is my performance tweak worth it? Should I just be checking and creating a queue before trying to read from it?

Thanks

Pat


--
Patrick Long - Munkiisoft Ltd

_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Check Queues Exist on Publish (and Read?)

Michael Klishin-2
 On 27 May 2014 at 17:18:51, Patrick Long ([hidden email]) wrote:
> > 1. Is there a way to tell RabbitMQ to chill out about queues not  
> being found?

You can do that queue.declare with passive = true on a separate channel or via HTTP API

> 2. Is my performance tweak worth it? Should I just be checking  
> and creating a queue before trying to read from it?

It depends on how you read. If you use basic.consume, which is often used
just once in the entire consumer lifecycle, then simply declare the queue you need,
bind it, etc.

If you use basic.get, you need to perform declaration and so on once. Every time you
declare a queue, it's a network roundtrip, so it will affect throughput.
--  
MK  

Software Engineer, Pivotal/RabbitMQ
_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Check Queues Exist on Publish (and Read?)

Simon MacMullen-2
On 27/05/14 14:21, Michael Klishin wrote:
>   On 27 May 2014 at 17:18:51, Patrick Long ([hidden email]) wrote:
>>> 1. Is there a way to tell RabbitMQ to chill out about queues not
>> being found?
>
> You can do that queue.declare with passive = true on a separate channel or via HTTP API

Well, both passive=true and passive=false will assert that the queue
exists, but passive=true will fail rather than create the queue if it is
missing. Since the OP is mostly concerned with making sure the basic.get
works, then it makes more sense to declare the queue passive=false.

Ultimately the AMQPish way to do things is declare what you need

> If you use basic.get, you need to perform declaration and so on once. Every time you
> declare a queue, it's a network roundtrip, so it will affect throughput.

Unless you issue the declare with nowait=true, in which case the client
will just send the declare request and not wait for a response. If the
queue already exists, this is almost free. Obviously if it doesn't you
incur the cost of creating it, but you probably wanted that anyway :-)

Cheers, Simon


--
Simon MacMullen
RabbitMQ, Pivotal
_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Check Queues Exist on Publish (and Read?)

Patrick Long
>>Since the OP is mostly concerned with making sure the basic.get works, then it makes more sense to declare the queue passive=false.<<
 I am mostly concerned with BasicGet not causing a log entry.If the queue is not there I don't care just don't error.

Are you guys saying I should always check the queue exist, Read and Write and that it is how I check that is worth changing. The change should be passing true for NoWait?

It does not look like the RabbitMQ .NET Client Library supports passing nowait to QueueDeclare.

Pat


On 27 May 2014 14:28, Simon MacMullen <[hidden email]> wrote:
On 27/05/14 14:21, Michael Klishin wrote:
  On 27 May 2014 at 17:18:51, Patrick Long ([hidden email]) wrote:
1. Is there a way to tell RabbitMQ to chill out about queues not
being found?

You can do that queue.declare with passive = true on a separate channel or via HTTP API

Well, both passive=true and passive=false will assert that the queue exists, but passive=true will fail rather than create the queue if it is missing. Since the OP is mostly concerned with making sure the basic.get works, then it makes more sense to declare the queue passive=false.

Ultimately the AMQPish way to do things is declare what you need

If you use basic.get, you need to perform declaration and so on once. Every time you
declare a queue, it's a network roundtrip, so it will affect throughput.

Unless you issue the declare with nowait=true, in which case the client will just send the declare request and not wait for a response. If the queue already exists, this is almost free. Obviously if it doesn't you incur the cost of creating it, but you probably wanted that anyway :-)

Cheers, Simon


--
Simon MacMullen
RabbitMQ, Pivotal



--
Patrick Long - Munkiisoft Ltd

_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Check Queues Exist on Publish (and Read?)

Matthias Radestock-3
Patrick,

On 27/05/14 15:09, Patrick Long wrote:
>   I am mostly concerned with BasicGet not causing a log entry.If the
> queue is not there I don't care just don't error.

There is no way to do that in AMQP.

> Are you guys saying I should always check the queue exist

That won't help since a passive queue.declare will *still* result in an
error and log entry when the queue doesn't exist.

> The change should be passing true for NoWait?

What Simon suggested is that you make sure that the queue *does* exist
prior to the basic.get by (actively) declaring it with queue.declare.
The nowait flag is only there to make that more efficient, by turning
the command into an asynchronous request.

> It does not look like the RabbitMQ .NET Client Library supports passing
> nowait to QueueDeclare.

Correct. You'd have to drop to the internal API by casting to
RabbitMQ.Client.Impl.IFullModel and invoking _Private_QueueDeclare(...).

Matthias.
_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Check Queues Exist on Publish (and Read?)

Patrick Long
Thanks Matthias


On 28 May 2014 10:10, Matthias Radestock <[hidden email]> wrote:
Patrick,

On 27/05/14 15:09, Patrick Long wrote:
  I am mostly concerned with BasicGet not causing a log entry.If the
queue is not there I don't care just don't error.

There is no way to do that in AMQP.

Are you guys saying I should always check the queue exist

That won't help since a passive queue.declare will *still* result in an error and log entry when the queue doesn't exist.

The change should be passing true for NoWait?

What Simon suggested is that you make sure that the queue *does* exist prior to the basic.get by (actively) declaring it with queue.declare. The nowait flag is only there to make that more efficient, by turning the command into an asynchronous request.

It does not look like the RabbitMQ .NET Client Library supports passing
nowait to QueueDeclare.

Correct. You'd have to drop to the internal API by casting to RabbitMQ.Client.Impl.IFullModel and invoking _Private_QueueDeclare(...).

Matthias.



--
Patrick Long - Munkiisoft Ltd

_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss