[rabbitmq-c] amqp_queue_unbind eat a lot of memory in some usecases

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

[rabbitmq-c] amqp_queue_unbind eat a lot of memory in some usecases

Haster
Hi there!

I have a following problem usecase:
1) Send a lot of message to some queue (I do it using java client)
2) Make connection to RabbitMQ using rabbitmq-c library and bind some exchange to that queue
3) Call amqp_basic_consume method and start loop for reading messages from the queue
4) After some time I try to make unbind... and this function eat a lot of memory.
I think this function recive a lot of messages and save it to buffer until it get unbind-ok.

I decided to solve this problem by calling basic.cancel befor making unbind, but the behaviour was the same.

How can I solve this problem?
Is it possible to call cancel or unbind on another channel or it also will accumulate all messages?
Or shoud I use another connection for it and will it work?

Best regards,
Yury
Reply | Threaded
Open this post in threaded view
|

Re: [rabbitmq-c] amqp_queue_unbind eat a lot of memory in some usecases

alan.antonuk
On Thu, Sep 5, 2013 at 11:32 AM, Haster <[hidden email]> wrote:

I think this function recive a lot of messages and save it to buffer until
it get unbind-ok.
You are correct, this is likely what is going on inside the library.


How can I solve this problem?
Assuming you're not using the no-ack option of basic.consume, you could do a basic.qos (amqp_basic_qos()) on the channel with the consumer, which will limit the number of un-acked messages the broker sends to the client.
 
Is it possible to call cancel or unbind on another channel or it also will
accumulate all messages?
Calling it on a different channel won't help, the frame buffer is a per-connection thing.
 
Or shoud I use another connection for it and will it work?
This is also an option. 

-Alan

_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: [rabbitmq-c] amqp_queue_unbind eat a lot of memory in some usecases

Haster
This post was updated on .
Alan, thanks for your reply.

I set no-ack to 0 in basic.consume, so I suggest that server will wait for basic.ack from client on each message. Or I have to set no-ack to 1 for this behaviour?

Also I use prefetch-count set to 0, and I suggest that server will not push addition messages to me.
But it do it
Reply | Threaded
Open this post in threaded view
|

Re: [rabbitmq-c] amqp_queue_unbind eat a lot of memory in some usecases

alan.antonuk

On Thu, Sep 5, 2013 at 11:06 PM, Haster <[hidden email]> wrote:

I set no-ack to 0 in basic.consume, so I suggest that server will wait for
basic.ack from client on each message.

Yes, if no-ack is set to 0, the broker won't consider the message delivered until you acknowledge it. By default the broker does not pause sending messages if there are outstanding unacknowledged messages. You can tune this by using basic.qos to set a message prefetch on the channel, this will limit the number of unacknowledged messages on a channel before the broker will pause and wait for the client to acknowledge a message.

If no-ack is set to 1, the broker considers the message acknowledged as soon as it leaves the broker, and basic.qos will have no effect consumers declared with this set.

You can acknowledge a single message with basic.ack (amqp_basic_ack()), or all messages up to a message by setting the multiple flag when doing a basic.ack.


-Alan

_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: [rabbitmq-c] amqp_queue_unbind eat a lot of memory in some usecases

Matthias Radestock-3
On 06/09/13 08:03, Alan Antonuk wrote:
> Yes, if no-ack is set to 0, the broker won't consider the message
> delivered until you acknowledge it. [...]
> If no-ack is set to 1, the broker considers the message acknowledged as
> soon as it leaves the broker, and basic.qos will have no effect
> consumers declared with this set.

OP: you may want to (re)read tutorial #2,
https://www.rabbitmq.com/tutorials/tutorial-two-python.html, which
covers acks and qos in some detail.

Regards,

Matthias.
_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: [rabbitmq-c] amqp_queue_unbind eat a lot of memory in some usecases

Haster
In reply to this post by alan.antonuk
Alan,
and if no-ack is false and prefetch_count set to 0 (as in my case)... What does it mean?
Does it mean that server use default setting for prefetch and send additional messages to client?

I think that if prefetch_count = 0 server must not push messages to me? Is it correct?
Reply | Threaded
Open this post in threaded view
|

Re: [rabbitmq-c] amqp_queue_unbind eat a lot of memory in some usecases

Haster
In reply to this post by Matthias Radestock-3
Matthias,
thanks for link, it is time to refresh knowladges
Reply | Threaded
Open this post in threaded view
|

Re: [rabbitmq-c] amqp_queue_unbind eat a lot of memory in some usecases

Haster
In reply to this post by Haster
OK? I found in AMQP 0-9-1 Complete Reference Guide that setting prefetch-count  to zero meaning "no specific limit".
So I think that problem was in this parameter
p.s. Yep I set it to 1 and amqp_basic_unbind doesn't eat memory