Broker Unreachable Exception after Establishing certain number of connection

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

Broker Unreachable Exception after Establishing certain number of connection

chetan dev
Hi,


Why RabbitMq Client rejects further connections and gives brokerunreachable exception after establishing certain number of connections(480 approx).
There is no connection blockage (enough ram memory and socket descriptors are available)

Thanks


--
Cheten  Dev





_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Broker Unreachable Exception after Establishing certain number of connection

Alvaro Videla-2
Hi,

What client are you using?

Have you checked the RabbitMQ logs?

Are you sure RabbitMQ is picking the right amount of socket descriptors?

Regards,

Alvaro

On Tue, Mar 11, 2014 at 12:04 PM, chetan dev <[hidden email]> wrote:

> Hi,
>
>
> Why RabbitMq Client rejects further connections and gives brokerunreachable
> exception after establishing certain number of connections(480 approx).
> There is no connection blockage (enough ram memory and socket descriptors
> are available)
>
> Thanks
>
>
> --
> Cheten  Dev
>
>
>
>
>
> _______________________________________________
> rabbitmq-discuss mailing list
> [hidden email]
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Broker Unreachable Exception after Establishing certain number of connection

chetan dev

Hi,
I am using .net client .
I have attached the logs and the screenshot of management ui

Thanks




Inline image 1


On Tue, Mar 11, 2014 at 4:38 PM, Alvaro Videla <[hidden email]> wrote:
Hi,

What client are you using?

Have you checked the RabbitMQ logs?

Are you sure RabbitMQ is picking the right amount of socket descriptors?

Regards,

Alvaro

On Tue, Mar 11, 2014 at 12:04 PM, chetan dev <[hidden email]> wrote:
> Hi,
>
>
> Why RabbitMq Client rejects further connections and gives brokerunreachable
> exception after establishing certain number of connections(480 approx).
> There is no connection blockage (enough ram memory and socket descriptors
> are available)
>
> Thanks
>
>
> --
> Cheten  Dev
>
>
>
>
>
> _______________________________________________
> rabbitmq-discuss mailing list
> [hidden email]
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss



--
Cheten  Dev

B.Tech Final Year
Dept. of Electrical Engg.
IIT Delhi, New Delhi
ph 8527333215

_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss

rabbit@USER145.log (230K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Broker Unreachable Exception after Establishing certain number of connection

Alvaro Videla-2
The "handle.exe missing" message that you see under File descriptors seems to be related to this problem: http://rabbitmq.1065348.n5.nabble.com/nightly-2-8-7-31107-Admin-GUI-install-handle-from-sysinternals-td23216.html#a23218


On Tue, Mar 11, 2014 at 12:30 PM, chetan dev <[hidden email]> wrote:

Hi,
I am using .net client .
I have attached the logs and the screenshot of management ui

Thanks




Inline image 1


On Tue, Mar 11, 2014 at 4:38 PM, Alvaro Videla <[hidden email]> wrote:
Hi,

What client are you using?

Have you checked the RabbitMQ logs?

Are you sure RabbitMQ is picking the right amount of socket descriptors?

Regards,

Alvaro

On Tue, Mar 11, 2014 at 12:04 PM, chetan dev <[hidden email]> wrote:
> Hi,
>
>
> Why RabbitMq Client rejects further connections and gives brokerunreachable
> exception after establishing certain number of connections(480 approx).
> There is no connection blockage (enough ram memory and socket descriptors
> are available)
>
> Thanks
>
>
> --
> Cheten  Dev
>
>
>
>
>
> _______________________________________________
> rabbitmq-discuss mailing list
> [hidden email]
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss



--
Cheten  Dev

B.Tech Final Year
Dept. of Electrical Engg.
IIT Delhi, New Delhi
ph 8527333215

_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss



_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Broker Unreachable Exception after Establishing certain number of connection

chetan dev
Hi,

There are enough file descriptors available.
i have run the test again and here is the screenshot of management ui.

Inline image 1

 Thanks


On Tue, Mar 11, 2014 at 5:15 PM, Alvaro Videla <[hidden email]> wrote:
The "handle.exe missing" message that you see under File descriptors seems to be related to this problem: http://rabbitmq.1065348.n5.nabble.com/nightly-2-8-7-31107-Admin-GUI-install-handle-from-sysinternals-td23216.html#a23218


On Tue, Mar 11, 2014 at 12:30 PM, chetan dev <[hidden email]> wrote:

Hi,
I am using .net client .
I have attached the logs and the screenshot of management ui

Thanks




Inline image 1


On Tue, Mar 11, 2014 at 4:38 PM, Alvaro Videla <[hidden email]> wrote:
Hi,

What client are you using?

Have you checked the RabbitMQ logs?

Are you sure RabbitMQ is picking the right amount of socket descriptors?

Regards,

Alvaro

On Tue, Mar 11, 2014 at 12:04 PM, chetan dev <[hidden email]> wrote:
> Hi,
>
>
> Why RabbitMq Client rejects further connections and gives brokerunreachable
> exception after establishing certain number of connections(480 approx).
> There is no connection blockage (enough ram memory and socket descriptors
> are available)
>
> Thanks
>
>
> --
> Cheten  Dev
>
>
>
>
>
> _______________________________________________
> rabbitmq-discuss mailing list
> [hidden email]
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss



--
Cheten  Dev

B.Tech Final Year
Dept. of Electrical Engg.
IIT Delhi, New Delhi
ph 8527333215

_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss



_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss




--
Cheten  Dev

B.Tech Final Year
Dept. of Electrical Engg.
IIT Delhi, New Delhi
ph 8527333215

_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Broker Unreachable Exception after Establishing certain number of connection

Michael Klishin-2

On 11 Mar 2014, at 16:44, chetan dev <[hidden email]> wrote:

> There are enough file descriptors available.
> i have run the test again and here is the screenshot of management ui.

I’ve recently investigated a similar issue which boiled down to concurrency
hazards in the program. Can you be sharing connections between threads,
creating them lazily or something of that nature?

Also, are you sure you need over 480 connections from a program? Perhaps 1 connection
and 480 channels will do?

MK

Software Engineer, Pivotal/RabbitMQ


_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Broker Unreachable Exception after Establishing certain number of connection

chetan dev
Hi,

No, I am not sharing any connection between threads
I usually  open a connection when activity is done i close it along with the channel.
I was wondering what would happen if i don't close the connection , so I ran the test .


Thanks





On Tue, Mar 11, 2014 at 6:51 PM, Michael Klishin <[hidden email]> wrote:

On 11 Mar 2014, at 16:44, chetan dev <[hidden email]> wrote:

> There are enough file descriptors available.
> i have run the test again and here is the screenshot of management ui.

I’ve recently investigated a similar issue which boiled down to concurrency
hazards in the program. Can you be sharing connections between threads,
creating them lazily or something of that nature?

Also, are you sure you need over 480 connections from a program? Perhaps 1 connection
and 480 channels will do?

MK

Software Engineer, Pivotal/RabbitMQ


_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss



--
Cheten  Dev

B.Tech Final Year
Dept. of Electrical Engg.
IIT Delhi, New Delhi
ph 8527333215

_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Broker Unreachable Exception after Establishing certain number of connection

michaelklishin

2014-03-11 17:54 GMT+04:00 chetan dev <[hidden email]>:
No, I am not sharing any connection between threads
I usually  open a connection when activity is done i close it along with the channel.
I was wondering what would happen if i don't close the connection , so I ran the test .

Can you post an example program that reproduces the issue?


--
MK

http://github.com/michaelklishin
http://twitter.com/michaelklishin

_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Broker Unreachable Exception after Establishing certain number of connection

Matthias Radestock-3
On 11/03/14 18:39, Michael Klishin wrote:

> 2014-03-11 17:54 GMT+04:00 chetan dev <[hidden email]
> <mailto:[hidden email]>>:
>
>     No, I am not sharing any connection between threads
>     I usually  open a connection when activity is done i close it along
>     with the channel.
>     I was wondering what would happen if i don't close the connection ,
>     so I ran the test .
>
>
> Can you post an example program that reproduces the issue?

Also, as Alvaro requested previously, please check the broker logs for
any errors.

Also, I seem to recall that Windows can impose some restrictions on the
number of *outbound* connections that can be opened, so it is worth
checking that when you see the problem whether it is possible to connect
to the broker from a different machine.

Matthias.

_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Broker Unreachable Exception after Establishing certain number of connection

chetan dev
Hi,

I have already attached the logs.
I also think its windows created restriction . When i run another thread of similar program i am able to connect to broker .

Thanks


On Wed, Mar 12, 2014 at 12:37 AM, Matthias Radestock <[hidden email]> wrote:
On 11/03/14 18:39, Michael Klishin wrote:
2014-03-11 17:54 GMT+04:00 chetan dev <[hidden email]
<mailto:[hidden email]>>:


    No, I am not sharing any connection between threads
    I usually  open a connection when activity is done i close it along
    with the channel.
    I was wondering what would happen if i don't close the connection ,
    so I ran the test .


Can you post an example program that reproduces the issue?

Also, as Alvaro requested previously, please check the broker logs for any errors.

Also, I seem to recall that Windows can impose some restrictions on the number of *outbound* connections that can be opened, so it is worth checking that when you see the problem whether it is possible to connect to the broker from a different machine.

Matthias.


_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss



--
Cheten  Dev

B.Tech Final Year
Dept. of Electrical Engg.
IIT Delhi, New Delhi
ph 8527333215

_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Broker Unreachable Exception after Establishing certain number of connection

Michael Klishin-2
On 12 Mar 2014, at 10:16, chetan dev <[hidden email]> wrote:

> I have already attached the logs.

The log is full of

=INFO REPORT==== 11-Mar-2014::16:56:58 ===
accepting AMQP connection <0.7054.0> (192.168.1.173:54705 -> 192.168.1.173:5672)

=ERROR REPORT==== 11-Mar-2014::16:56:58 ===
closing AMQP connection <0.6951.0> (192.168.1.173:54670 -> 192.168.1.173:5672):
{handshake_timeout,handshake

which means a TCP connection is established but then clients send no data.

MK

Software Engineer, Pivotal/RabbitMQ


_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Broker Unreachable Exception after Establishing certain number of connection

Alvaro Videla-2
In reply to this post by chetan dev
Hi,

On Wed, Mar 12, 2014 at 7:16 AM, chetan dev <[hidden email]> wrote:

> I also think its windows created restriction . When i run another thread of
> similar program i am able to connect to broker .

This probably indicates that you should raise the socket descriptor
limit for your program as well.

-Alvaro
_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss