"reconnect-delay" seems to be ignored for Federation Upstreams

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

"reconnect-delay" seems to be ignored for Federation Upstreams

Roman Gaufman
Hi there,

I have my reconnect-delay set to 300s (see attached). However when I do:

tail -f /var/log/rabbitmq/* | grep -i vpn.plus/247cctv

I see these types of messages appearing in the logs every 2-3 seconds:

Federation exchange 'xanview' in vhost '247cctv' did not connect to exchange 'xanview' in vhost '247cctv' on amqp://vpn.plus/247cctv
Federation queue 'xanview.247cctv.portal' in vhost '247cctv' did not connect to queue 'xanview.247cctv.portal' in vhost '247cctv' on amqp://vpn.plus/247cctv
Federation exchange 'xanview' in vhost '247cctv' did not connect to exchange 'xanview' in vhost '247cctv' on amqp://vpn.plus/247cctv
Federation queue 'xanview.247cctv.portal' in vhost '247cctv' did not connect to queue 'xanview.247cctv.portal' in vhost '247cctv' on amqp://vpn.plus/247cctv

That server is presently offline and I would like the Federation plugin to retry every 5 minutes as requested, how would I do this?

Thank you for your time,

Roman

_______________________________________________
rabbitmq-discuss mailing list has moved to https://groups.google.com/forum/#!forum/rabbitmq-users,
please subscribe to the new list!

[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss

Screen Shot 2014-07-14 at 00.07.12.png (49K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: "reconnect-delay" seems to be ignored for Federation Upstreams

Simon MacMullen-2
On 14/07/2014 12:10AM, Roman Gaufman wrote:
> Hi there,
>
> I have my reconnect-delay set to 300s (see attached). However when I do:
>
> tail -f /var/log/rabbitmq/* | grep -i vpn.plus/247cctv
>
> I see these types of messages appearing in the logs every 2-3 seconds:

So I've been looking at this and I think you may have found a bug; it
looks like the reconnect_delay is ignored if it takes more than one
second for the link to fail to start.

Could you try replacing your federation plugin with:

http://www.rabbitmq.com/releases/plugins/v3.3.x/rabbitmq_federation-3.3.x.reconnect-fix.ez

(assuming you're running 3.3.x)

and see if that fixes it?

Cheers, Simon

--
Simon MacMullen
RabbitMQ, Pivotal
_______________________________________________
rabbitmq-discuss mailing list has moved to https://groups.google.com/forum/#!forum/rabbitmq-users,
please subscribe to the new list!

[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: "reconnect-delay" seems to be ignored for Federation Upstreams

Simon MacMullen-2
On 14/07/2014 1:44PM, Simon MacMullen wrote:
> Could you try replacing your federation plugin with:
>
> http://www.rabbitmq.com/releases/plugins/v3.3.x/rabbitmq_federation-3.3.x.reconnect-fix.ez
>
>
> (assuming you're running 3.3.x)
>
> and see if that fixes it?

No, that's too simplistic a fix. Actually this will take some effort to
fix properly. It will get fixed though.

In the mean time, you can at least prevent noise in the logs by
modifying the log_level config item for 'federation' (see
http://www.rabbitmq.com/configure.html#configuration-file).

Cheers, Simon

--
Simon MacMullen
RabbitMQ, Pivotal
_______________________________________________
rabbitmq-discuss mailing list has moved to https://groups.google.com/forum/#!forum/rabbitmq-users,
please subscribe to the new list!

[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: "reconnect-delay" seems to be ignored for Federation Upstreams

Roman Gaufman
Thank you for the log suggestion :) - Any rough estimate for when this can be fixed permanently?

I have many servers that are offline a lot (mobile clients) and I suspect that all the reconnection attempts are causing my RabbitMQ load to be 30% CPU despite only having around 2-3 messages/sec.


On 14 July 2014 14:51, Simon MacMullen <[hidden email]> wrote:
On 14/07/2014 1:44PM, Simon MacMullen wrote:
Could you try replacing your federation plugin with:

http://www.rabbitmq.com/releases/plugins/v3.3.x/rabbitmq_federation-3.3.x.reconnect-fix.ez


(assuming you're running 3.3.x)

and see if that fixes it?

No, that's too simplistic a fix. Actually this will take some effort to fix properly. It will get fixed though.

In the mean time, you can at least prevent noise in the logs by modifying the log_level config item for 'federation' (see http://www.rabbitmq.com/configure.html#configuration-file).


Cheers, Simon

--
Simon MacMullen
RabbitMQ, Pivotal


_______________________________________________
rabbitmq-discuss mailing list has moved to https://groups.google.com/forum/#!forum/rabbitmq-users,
please subscribe to the new list!

[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: "reconnect-delay" seems to be ignored for Federation Upstreams

Simon MacMullen-2
On 16/07/2014 1:51PM, Roman Gaufman wrote:
> Any rough estimate for when this can be fixed permanently?

It's in the nightlies and will be in 3.3.5. It requires a small tweak to
the broker though, so I couldn't just send an updated plugin.

Cheer, Simon

--
Simon MacMullen
RabbitMQ, Pivotal
_______________________________________________
rabbitmq-discuss mailing list has moved to https://groups.google.com/forum/#!forum/rabbitmq-users,
please subscribe to the new list!

[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: "reconnect-delay" seems to be ignored for Federation Upstreams

Roman Gaufman
Ah, thank you, I will wait for 3.3.5 patiently :) - looking forward to it, it seems every reconnect attempt causes a bit of CPU spike too which will be nice to remove.


On 17 July 2014 09:38, Simon MacMullen <[hidden email]> wrote:
On 16/07/2014 1:51PM, Roman Gaufman wrote:
Any rough estimate for when this can be fixed permanently?

It's in the nightlies and will be in 3.3.5. It requires a small tweak to the broker though, so I couldn't just send an updated plugin.

Cheer, Simon


--
Simon MacMullen
RabbitMQ, Pivotal


_______________________________________________
rabbitmq-discuss mailing list has moved to https://groups.google.com/forum/#!forum/rabbitmq-users,
please subscribe to the new list!

[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss