Connection is always in "flow" State.

classic Classic list List threaded Threaded
20 messages Options
Reply | Threaded
Open this post in threaded view
|

Connection is always in "flow" State.

Lost
I am run Rabbitmq in my machine(Win7 64bit, 4G ram, 2.9GHz cpu, Erlang OTP R16B03 (x64)).
I have a client to publish message to rabbitmq(1 connection and 1 channel, in same machine with rabbitmq), no consumer.
Message rates is 2000/s ~ 3000/s
Traffic is 300Kb/s ~ 750kB/s
The channel state is "running", but the connection state is always "flow".
I found erl.exe cpu is only 5% ~ 10%, but why connection is in "flow" state?

_______________________________________________
rabbitmq-discuss mailing list has moved to https://groups.google.com/forum/#!forum/rabbitmq-users,
please subscribe to the new list!

[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Connection is always in "flow" State.

Michael Klishin-2


On 8 July 2014 at 07:49:26, Lost ([hidden email]) wrote:
> > I have a client to publish message to rabbitmq(1 connection  
> and 1 channel, in same machine with rabbitmq), no consumer.
> Message rates is 2000/s ~ 3000/s
> Traffic is 300Kb/s ~ 750kB/s
> The channel state is "running", but the connection state is always  
> "flow".
> I found erl.exe cpu is only 5% ~ 10%, but why connection is in "flow"  
> state?

Flow control does not depend on CPU usage but rather whether consumers
can keep up with producers. Try adding a consumer. 
--  
MK  

Staff Software Engineer, Pivotal/RabbitMQ
_______________________________________________
rabbitmq-discuss mailing list has moved to https://groups.google.com/forum/#!forum/rabbitmq-users,
please subscribe to the new list!

[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Connection is always in "flow" State.

Michael Klishin-2
In reply to this post by Lost
 On 8 July 2014 at 07:49:26, Lost ([hidden email]) wrote:
> > I found erl.exe cpu is only 5% ~ 10%, but why connection is in "flow"  
> state?

Check how loaded the disk is. If you publish messages as persistent,
disk I/O is likely to be the bottleneck.
--  
MK  

Staff Software Engineer, Pivotal/RabbitMQ
_______________________________________________
rabbitmq-discuss mailing list has moved to https://groups.google.com/forum/#!forum/rabbitmq-users,
please subscribe to the new list!

[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Connection is always in "flow" State.

Michael Klishin-2
On 8 July 2014 at 12:47:11, Michael Klishin ([hidden email]) wrote:
> > Check how loaded the disk is. If you publish messages as persistent,
> disk I/O is likely to be the bottleneck.

or, if you run, 3.3, management UI can help you find this out:
http://www.rabbitmq.com/blog/2014/04/14/finding-bottlenecks-with-rabbitmq-3-3/
--
MK

Staff Software Engineer, Pivotal/RabbitMQ
_______________________________________________
rabbitmq-discuss mailing list has moved to https://groups.google.com/forum/#!forum/rabbitmq-users,
please subscribe to the new list!

[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Connection is always in "flow" State.

Lost
In reply to this post by Michael Klishin-2
I have any queue to consumer message, just publish, connection is in "flow" state, channel is in "running" state.
If i add a queue to consumer message, the connection and the channel is all in "flow" state.
 


------------------ Original message ------------------
From: "Michael Klishin";
Sendtime: Tuesday, Jul 8, 2014 4:33 PM
To: "Lost"; "rabbitmq-discuss";
Subject: Re: [rabbitmq-discuss] Connection is always in "flow" State.



On 8 July 2014 at 07:49:26, Lost ([hidden email]) wrote:
> > I have a client to publish message to rabbitmq(1 connection 
> and 1 channel, in same machine with rabbitmq), no consumer.
> Message rates is 2000/s ~ 3000/s
> Traffic is 300Kb/s ~ 750kB/s
> The channel state is "running", but the connection state is always 
> "flow".
> I found erl.exe cpu is only 5% ~ 10%, but why connection is in "flow" 
> state?

Flow control does not depend on CPU usage but rather whether consumers
can keep up with producers. Try adding a consumer. 
-- 
MK 

Staff Software Engineer, Pivotal/RabbitMQ
_______________________________________________
rabbitmq-discuss mailing list has moved to https://groups.google.com/forum/#!forum/rabbitmq-users,
please subscribe to the new list!

[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Connection is always in "flow" State.

Lost
In reply to this post by Michael Klishin-2
I publish message as non-persistent, so no disk I/O.


------------------ Original message ------------------
From: "Michael Klishin";
Sendtime: Tuesday, Jul 8, 2014 4:45 PM
To: "Lost"; "rabbitmq-discuss";
Subject: Re: [rabbitmq-discuss] Connection is always in "flow" State.

 On 8 July 2014 at 07:49:26, Lost ([hidden email]) wrote:
> > I found erl.exe cpu is only 5% ~ 10%, but why connection is in "flow" 
> state?

Check how loaded the disk is. If you publish messages as persistent,
disk I/O is likely to be the bottleneck.
-- 
MK 

Staff Software Engineer, Pivotal/RabbitMQ
_______________________________________________
rabbitmq-discuss mailing list has moved to https://groups.google.com/forum/#!forum/rabbitmq-users,
please subscribe to the new list!

[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Connection is always in "flow" State.

Lost
In reply to this post by Michael Klishin-2
Yes, i am run RabbitMQ 3.3.4, i have check this blog.
 
"A connection is in flow control, but none of its channels are - This means that one or more of the channels is the bottleneck; the server is CPU-bound on something the channel does, probably routing logic. This is most likely to be seen when publishing small transient messages."
 
I found this is just like my problem, it seems channels is the bottleneck, but i still have no idea to solve it.

 

------------------ Original ------------------
From:  "Michael Klishin";<[hidden email]>;
Send time: Tuesday, Jul 8, 2014 4:49 PM
To: "Lost"<[hidden email]>; "Legacy list about RabbitMQ"<[hidden email]>;
Subject:  Re: [rabbitmq-discuss] Connection is always in "flow" State.

On 8 July 2014 at 12:47:11, Michael Klishin ([hidden email]) wrote:
> > Check how loaded the disk is. If you publish messages as persistent,
> disk I/O is likely to be the bottleneck.

or, if you run, 3.3, management UI can help you find this out:
http://www.rabbitmq.com/blog/2014/04/14/finding-bottlenecks-with-rabbitmq-3-3/
--
MK

Staff Software Engineer, Pivotal/RabbitMQ

_______________________________________________
rabbitmq-discuss mailing list has moved to https://groups.google.com/forum/#!forum/rabbitmq-users,
please subscribe to the new list!

[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Connection is always in "flow" State.

Michael Klishin-2
On 8 July 2014 at 13:30:37, Lost ([hidden email]) wrote:
> > I found this is just like my problem, it seems channels is the  
> bottleneck, but i still have no idea to solve it.

What exchange types(s)  do you use? Can it be that you use a single queue that has grown very large?
A single queue can only use 1 CPU core. 
--  
MK  

Staff Software Engineer, Pivotal/RabbitMQ
_______________________________________________
rabbitmq-discuss mailing list has moved to https://groups.google.com/forum/#!forum/rabbitmq-users,
please subscribe to the new list!

[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Connection is always in "flow" State.

Lost
a fanout exchange, i have any queues, just publish message to a fanout exchange.


------------------ Original message ------------------
From: "Michael Klishin";
Sendtime: Tuesday, Jul 8, 2014 5:32 PM
To: "rabbitmq-discuss"; "Lost";
Subject: Re: [rabbitmq-discuss] Connection is always in "flow" State.

On 8 July 2014 at 13:30:37, Lost ([hidden email]) wrote:
> > I found this is just like my problem, it seems channels is the 
> bottleneck, but i still have no idea to solve it.

What exchange types(s)  do you use? Can it be that you use a single queue that has grown very large?
A single queue can only use 1 CPU core. 
-- 
MK 

Staff Software Engineer, Pivotal/RabbitMQ

_______________________________________________
rabbitmq-discuss mailing list has moved to https://groups.google.com/forum/#!forum/rabbitmq-users,
please subscribe to the new list!

[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Connection is always in "flow" State.

Lost
In reply to this post by Michael Klishin-2
a fanout exchange, i have NOT any queues, just publish message to a fanout exchange.


------------------ Original message ------------------
From: "Michael Klishin";
Sendtime: Tuesday, Jul 8, 2014 5:32 PM
To: "rabbitmq-discuss"; "Lost";
Subject: Re: [rabbitmq-discuss] Connection is always in "flow" State.

On 8 July 2014 at 13:30:37, Lost ([hidden email]) wrote:
> > I found this is just like my problem, it seems channels is the 
> bottleneck, but i still have no idea to solve it.

What exchange types(s)  do you use? Can it be that you use a single queue that has grown very large?
A single queue can only use 1 CPU core. 
-- 
MK 

Staff Software Engineer, Pivotal/RabbitMQ

_______________________________________________
rabbitmq-discuss mailing list has moved to https://groups.google.com/forum/#!forum/rabbitmq-users,
please subscribe to the new list!

[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Connection is always in "flow" State.

Michael Klishin-2
In reply to this post by Lost
On 8 July 2014 at 13:35:28, Lost ([hidden email]) wrote:
> > a fanout exchange, i have any queues, just publish message to  
> a fanout exchange.

Do you mean you *do not* have any queues bound to the exchange? 
--  
MK  

Staff Software Engineer, Pivotal/RabbitMQ
_______________________________________________
rabbitmq-discuss mailing list has moved to https://groups.google.com/forum/#!forum/rabbitmq-users,
please subscribe to the new list!

[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Connection is always in "flow" State.

Lost
Yes, just one connection, one channel and publish to a exchange(no any queues to bind it.)


------------------ Original message ------------------
From: "Michael Klishin";
Sendtime: Tuesday, Jul 8, 2014 5:36 PM
To: "rabbitmq-discuss"; "Lost";
Subject: Re: [rabbitmq-discuss] Connection is always in "flow" State.

On 8 July 2014 at 13:35:28, Lost ([hidden email]) wrote:
> > a fanout exchange, i have any queues, just publish message to 
> a fanout exchange.

Do you mean you *do not* have any queues bound to the exchange? 
-- 
MK 

Staff Software Engineer, Pivotal/RabbitMQ

_______________________________________________
rabbitmq-discuss mailing list has moved to https://groups.google.com/forum/#!forum/rabbitmq-users,
please subscribe to the new list!

[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Connection is always in "flow" State.

Michael Klishin-2
On 8 July 2014 at 13:39:59, Lost ([hidden email]) wrote:
> > Yes, just one connection, one channel and publish to a exchange(no
> any queues to bind it.)

You can simulate this setup with PerfTest [1]:

./runjava.sh com.rabbitmq.examples.PerfTest -y 0 -t "fanout"

and tweak various options such as message size.

For example, running PerfTest with the options above stabilizes
at about 65-70 K messages per second on my (1 gen old, not maxed-out)
MacBook Pro.

1. http://www.rabbitmq.com/java-tools.html
--
MK

Staff Software Engineer, Pivotal/RabbitMQ
_______________________________________________
rabbitmq-discuss mailing list has moved to https://groups.google.com/forum/#!forum/rabbitmq-users,
please subscribe to the new list!

[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Connection is always in "flow" State.

Lost
runjava.bat com.rabbitmq.examples.PerfTest -y 0 -t "fanout"
starting producer #0
time: 1.000s, sent: 31063 msg/s
time: 2.001s, sent: 59184 msg/s
time: 3.001s, sent: 61560 msg/s
time: 4.001s, sent: 62564 msg/s
time: 5.001s, sent: 59613 msg/s
time: 6.002s, sent: 38134 msg/s
time: 7.002s, sent: 53956 msg/s
connection: 5.7MB/s, flow state
channel: 62,706/s, running state

runjava.bat com.rabbitmq.examples.PerfTest -y 0 -t "fanout" -s 200 -r 2000
starting producer #0
time: 1.000s, sent: 1655 msg/s
time: 2.000s, sent: 2000 msg/s
time: 3.000s, sent: 2000 msg/s
time: 4.000s, sent: 2000 msg/s
time: 5.000s, sent: 2000 msg/s
time: 6.000s, sent: 2000 msg/s
time: 7.000s, sent: 2000 msg/s
connection: 563kB/s, running state(never see flow state)
channel: 2,000/s, running state
 
This is my test result, it really very faster than my applicaion.
I use rabbbitmq-c(a c language lib), i found it demo is also faster than my app.
I have add a rate limit to my app, my msg size is about 100~200 byte, i found if message rate > 2000, the connection is always going to flow state(<500KB/s).
It really strange, the different my app with this demo is my app is a tcp client, it receive msg from a tcp server, do some convert and publish to RabbitMQ.


------------------ Original message ------------------
From: "Michael Klishin";
Sendtime: Tuesday, Jul 8, 2014 6:07 PM
To: "Lost"; "rabbitmq-discuss";
Subject: Re: [rabbitmq-discuss] Connection is always in "flow" State.

On 8 July 2014 at 13:39:59, Lost ([hidden email]) wrote:
> > Yes, just one connection, one channel and publish to a exchange(no
> any queues to bind it.)

You can simulate this setup with PerfTest [1]:

./runjava.sh com.rabbitmq.examples.PerfTest -y 0 -t "fanout"

and tweak various options such as message size.

For example, running PerfTest with the options above stabilizes
at about 65-70 K messages per second on my (1 gen old, not maxed-out)
MacBook Pro.

1. http://www.rabbitmq.com/java-tools.html
--
MK

Staff Software Engineer, Pivotal/RabbitMQ
_______________________________________________
rabbitmq-discuss mailing list has moved to https://groups.google.com/forum/#!forum/rabbitmq-users,
please subscribe to the new list!

[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Connection is always in "flow" State.

mc717990
In reply to this post by Michael Klishin-2
Just a quick confirmation on this.  The documentation states that flow control is:
  • A per-connection mechanism that prevents messages being published faster than they can be routed to queues.
  • A global mechanism that prevents any messages from being published when the memory usage exceeds a configured threshold or free disk space drops below a configured threshold.
In my testing, I've actually seen this as well - where messages can backlog (so far seems to the point of available resources), while a consumer is chugging along at half the publish speed.  SO is flow control also triggered by consumer processing?  At what point does consumer handling trigger flow control if so?  Just looking for a little bit of clarification on this - I'm guessing there's some internal logic someplace I'm missing on this,
Thanks!
Jason


On Tue, Jul 8, 2014 at 3:33 AM, Michael Klishin <[hidden email]> wrote:


On 8 July 2014 at 07:49:26, Lost ([hidden email]) wrote:
> > I have a client to publish message to rabbitmq(1 connection
> and 1 channel, in same machine with rabbitmq), no consumer.
> Message rates is 2000/s ~ 3000/s
> Traffic is 300Kb/s ~ 750kB/s
> The channel state is "running", but the connection state is always
> "flow".
> I found erl.exe cpu is only 5% ~ 10%, but why connection is in "flow"
> state?

Flow control does not depend on CPU usage but rather whether consumers
can keep up with producers. Try adding a consumer. 
--
MK

Staff Software Engineer, Pivotal/RabbitMQ
_______________________________________________
rabbitmq-discuss mailing list has moved to https://groups.google.com/forum/#!forum/rabbitmq-users,
please subscribe to the new list!

[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss



--
Jason McIntosh
https://github.com/jasonmcintosh/
573-424-7612

_______________________________________________
rabbitmq-discuss mailing list has moved to https://groups.google.com/forum/#!forum/rabbitmq-users,
please subscribe to the new list!

[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Connection is always in "flow" State.

Simon MacMullen-2
On 10/07/2014 3:31PM, Jason McIntosh wrote:
> Just a quick confirmation on this.  The documentation states that flow
> control is:
>
>   * A per-connection mechanism that prevents messages being published
>     faster than they can be routed to queues.
>   * A global mechanism that prevents any messages from being published
>     when the memory usage
>     <http://www.rabbitmq.com/memory-use.html> exceeds a configured
>     threshold or free disk space drops below a configured threshold.

That documentation desperately needs a rewrite :-( It doesn't make very
clear at all that the "flow" state applies to the first of those two
cases only; the second will show as "blocking" or "blocked".

> In my testing, I've actually seen this as well - where messages can
> backlog (so far seems to the point of available resources), while a
> consumer is chugging along at half the publish speed.

Was this with 3.3.x?

3.3.x ensures that if consumers and producers are both going as fast as
they can for an individual queue, then the consumers will consume ~10%
faster than the producers produce.

For earlier releases there is no such guarantee; it's possible for the
queue to just naturally accept more messages than it delivers. See the
blog post on the subject:

http://www.rabbitmq.com/blog/2014/04/10/consumer-bias-in-rabbitmq-3-3/

> SO is flow
> control also triggered by consumer processing?

Not really; Michael was misinformed in the post you replied to. Flow
control is explained a bit more at
http://www.rabbitmq.com/blog/2014/04/14/finding-bottlenecks-with-rabbitmq-3-3/ 
but in short flow control just means that something in the chain
(connection -> channel -> queue -> message store) is working as hard as
it can, and that therefore a publisher is not publishing as fast as it
would otherwise like to.

> At what point does
> consumer handling trigger flow control if so?  Just looking for a little
> bit of clarification on this - I'm guessing there's some internal logic
> someplace I'm missing on this,

It doesn't, really. Well, it's possible that you could push the queue
over to being CPU-bound by adding another consumer and thus having it do
slightly more work, but you'd have to be teetering on the edge anyway.

Cheers, Simon

> Thanks!
> Jason
>
>
> On Tue, Jul 8, 2014 at 3:33 AM, Michael Klishin <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>
>
>     On 8 July 2014 at 07:49:26, Lost ([hidden email] <mailto:[hidden email]>)
>     wrote:
>      > > I have a client to publish message to rabbitmq(1 connection
>      > and 1 channel, in same machine with rabbitmq), no consumer.
>      > Message rates is 2000/s ~ 3000/s
>      > Traffic is 300Kb/s ~ 750kB/s
>      > The channel state is "running", but the connection state is always
>      > "flow".
>      > I found erl.exe cpu is only 5% ~ 10%, but why connection is in "flow"
>      > state?
>
>     Flow control does not depend on CPU usage but rather whether consumers
>     can keep up with producers. Try adding a consumer.
>     --
>     MK
>
>     Staff Software Engineer, Pivotal/RabbitMQ
>     _______________________________________________
>     rabbitmq-discuss mailing list has moved to
>     https://groups.google.com/forum/#!forum/rabbitmq-users,
>     please subscribe to the new list!
>
>     [hidden email]
>     <mailto:[hidden email]>
>     https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
>
>
>
> --
> Jason McIntosh
> https://github.com/jasonmcintosh/
> 573-424-7612
>
>
> _______________________________________________
> rabbitmq-discuss mailing list has moved to https://groups.google.com/forum/#!forum/rabbitmq-users,
> please subscribe to the new list!
>
> [hidden email]
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>

--
Simon MacMullen
RabbitMQ, Pivotal
_______________________________________________
rabbitmq-discuss mailing list has moved to https://groups.google.com/forum/#!forum/rabbitmq-users,
please subscribe to the new list!

[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Connection is always in "flow" State.

mc717990
Interesting on the 3.3 change.  SO I gotta ask about this situation on that (though it sounds off hand that'll fix the behavior I've seen on 3.2.4).  If a consumer is consumer at 80 msgs/second, but incoming rates are able to publish at 5k a second and the server CPU's/disks/etc. aren't being touched, will rabbit then put back pressure on the publishing rate forcing it down to 72 msgs/second (~10% less than consumes)?  I'd assume then that'd propagate up to the publisher if there's a chain of rabbit servers all running 3.3 - e.g. the code publish to rabbit which shovels to the remote rabbit where the code consumer runs.  In this case the code publisher would get back pressure that'd reduce it's publish rate to ~72msgs/second?

Jason


On Thu, Jul 10, 2014 at 10:01 AM, Simon MacMullen <[hidden email]> wrote:
On 10/07/2014 3:31PM, Jason McIntosh wrote:
Just a quick confirmation on this.  The documentation states that flow
control is:

  * A per-connection mechanism that prevents messages being published

    faster than they can be routed to queues.
  * A global mechanism that prevents any messages from being published
    when the memory usage
    <http://www.rabbitmq.com/memory-use.html> exceeds a configured

    threshold or free disk space drops below a configured threshold.

That documentation desperately needs a rewrite :-( It doesn't make very clear at all that the "flow" state applies to the first of those two cases only; the second will show as "blocking" or "blocked".


In my testing, I've actually seen this as well - where messages can
backlog (so far seems to the point of available resources), while a
consumer is chugging along at half the publish speed.

Was this with 3.3.x?

3.3.x ensures that if consumers and producers are both going as fast as they can for an individual queue, then the consumers will consume ~10% faster than the producers produce.

For earlier releases there is no such guarantee; it's possible for the queue to just naturally accept more messages than it delivers. See the blog post on the subject:

http://www.rabbitmq.com/blog/2014/04/10/consumer-bias-in-rabbitmq-3-3/


SO is flow
control also triggered by consumer processing?

Not really; Michael was misinformed in the post you replied to. Flow control is explained a bit more at http://www.rabbitmq.com/blog/2014/04/14/finding-bottlenecks-with-rabbitmq-3-3/ but in short flow control just means that something in the chain (connection -> channel -> queue -> message store) is working as hard as it can, and that therefore a publisher is not publishing as fast as it would otherwise like to.


At what point does
consumer handling trigger flow control if so?  Just looking for a little
bit of clarification on this - I'm guessing there's some internal logic
someplace I'm missing on this,

It doesn't, really. Well, it's possible that you could push the queue over to being CPU-bound by adding another consumer and thus having it do slightly more work, but you'd have to be teetering on the edge anyway.

Cheers, Simon

Thanks!
Jason


On Tue, Jul 8, 2014 at 3:33 AM, Michael Klishin <[hidden email]
<mailto:[hidden email]>> wrote:



    On 8 July 2014 at 07:49:26, Lost ([hidden email] <mailto:[hidden email]>)

    wrote:
     > > I have a client to publish message to rabbitmq(1 connection
     > and 1 channel, in same machine with rabbitmq), no consumer.
     > Message rates is 2000/s ~ 3000/s
     > Traffic is 300Kb/s ~ 750kB/s
     > The channel state is "running", but the connection state is always
     > "flow".
     > I found erl.exe cpu is only 5% ~ 10%, but why connection is in "flow"
     > state?

    Flow control does not depend on CPU usage but rather whether consumers
    can keep up with producers. Try adding a consumer.
    --
    MK

    Staff Software Engineer, Pivotal/RabbitMQ
    _______________________________________________
    rabbitmq-discuss mailing list has moved to
    https://groups.google.com/forum/#!forum/rabbitmq-users,
    please subscribe to the new list!

    [hidden email]
    <mailto:[hidden email]>

    https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss




--
Jason McIntosh
https://github.com/jasonmcintosh/
<a href="tel:573-424-7612" value="+15734247612" target="_blank">573-424-7612


_______________________________________________
rabbitmq-discuss mailing list has moved to https://groups.google.com/forum/#!forum/rabbitmq-users,
please subscribe to the new list!

[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss


--
Simon MacMullen
RabbitMQ, Pivotal



--
Jason McIntosh
https://github.com/jasonmcintosh/
573-424-7612

_______________________________________________
rabbitmq-discuss mailing list has moved to https://groups.google.com/forum/#!forum/rabbitmq-users,
please subscribe to the new list!

[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Connection is always in "flow" State.

Simon MacMullen-2
On 10/07/2014 4:23PM, Jason McIntosh wrote:
> Interesting on the 3.3 change.  SO I gotta ask about this situation on
> that (though it sounds off hand that'll fix the behavior I've seen on
> 3.2.4).  If a consumer is consumer at 80 msgs/second, but incoming rates
> are able to publish at 5k a second and the server CPU's/disks/etc.
> aren't being touched, will rabbit then put back pressure on the
> publishing rate forcing it down to 72 msgs/second (~10% less than
> consumes)?

No, no, no!

(And this is why writing documentation is hard! At least for me.)

I somewhat oversimplified. What really happens is that the queue will
choose to do work that relates to consumers rather than publishers as
long as:

1) The queue is running flat out; it never has any idle time.
2) The queue actually has work to do that pertains to consuming.
3) Messages are being consumed at less than 110% of the rate they're
being published.

In the case you describe above, 2) will be false 99% of the time, at
which point the queue will accept publishes as fast as it can.

In other words, when I said "if consumers and producers are both going
as fast as they can" I meant if the limiting factor is the queue itself,
not whatever work they're doing.

I hope that is clearer this time round.

> I'd assume then that'd propagate up to the publisher if
> there's a chain of rabbit servers all running 3.3 - e.g. the code
> publish to rabbit which shovels to the remote rabbit where the code
> consumer runs.  In this case the code publisher would get back pressure
> that'd reduce it's publish rate to ~72msgs/second?

Which would be a disaster. It's meant to be a queuing system, after all :-)

Cheers, Simon

--
Simon MacMullen
RabbitMQ, Pivotal
_______________________________________________
rabbitmq-discuss mailing list has moved to https://groups.google.com/forum/#!forum/rabbitmq-users,
please subscribe to the new list!

[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Connection is always in "flow" State.

mc717990
Yep glad for the clarification and that's the behavior I'd actually want thinking about this.  :)  It's like a file load - all IO, no CPU.  SO we use consumers to do the CPU processing - which is inherently slower in a ETL concept.  We're basically using RabbitMQ for distributed ETL work (go distributed computing).  Since file loads are intermittent, you wouldn't want to slow down file loads.

Thanks for the clarification and that makes it MUCH more clear!
Jason


On Thu, Jul 10, 2014 at 10:41 AM, Simon MacMullen <[hidden email]> wrote:
On 10/07/2014 4:23PM, Jason McIntosh wrote:
Interesting on the 3.3 change.  SO I gotta ask about this situation on
that (though it sounds off hand that'll fix the behavior I've seen on
3.2.4).  If a consumer is consumer at 80 msgs/second, but incoming rates
are able to publish at 5k a second and the server CPU's/disks/etc.
aren't being touched, will rabbit then put back pressure on the
publishing rate forcing it down to 72 msgs/second (~10% less than
consumes)?

No, no, no!

(And this is why writing documentation is hard! At least for me.)

I somewhat oversimplified. What really happens is that the queue will choose to do work that relates to consumers rather than publishers as long as:

1) The queue is running flat out; it never has any idle time.
2) The queue actually has work to do that pertains to consuming.
3) Messages are being consumed at less than 110% of the rate they're being published.

In the case you describe above, 2) will be false 99% of the time, at which point the queue will accept publishes as fast as it can.

In other words, when I said "if consumers and producers are both going as fast as they can" I meant if the limiting factor is the queue itself, not whatever work they're doing.

I hope that is clearer this time round.


I'd assume then that'd propagate up to the publisher if
there's a chain of rabbit servers all running 3.3 - e.g. the code
publish to rabbit which shovels to the remote rabbit where the code
consumer runs.  In this case the code publisher would get back pressure
that'd reduce it's publish rate to ~72msgs/second?

Which would be a disaster. It's meant to be a queuing system, after all :-)

Cheers, Simon


--
Simon MacMullen
RabbitMQ, Pivotal



--
Jason McIntosh
https://github.com/jasonmcintosh/
573-424-7612

_______________________________________________
rabbitmq-discuss mailing list has moved to https://groups.google.com/forum/#!forum/rabbitmq-users,
please subscribe to the new list!

[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Connection is always in "flow" State.

Lost
In reply to this post by Michael Klishin-2
continue previous email...
I have increase my app publish message rate(8 times),
Connection:
from client is 3.5M/s, state is flow
Channel:
publish rate is 20000/s, state is running
 
I am strange is why the before publish rate is only 2000/s(300K/s), the connection is still in flow state.
 


------------------ Original message ------------------
From: "Michael Klishin";
Sendtime: Tuesday, Jul 8, 2014 6:07 PM
To: "Lost"; "rabbitmq-discuss";
Subject: Re: [rabbitmq-discuss] Connection is always in "flow" State.

On 8 July 2014 at 13:39:59, Lost ([hidden email]) wrote:
> > Yes, just one connection, one channel and publish to a exchange(no
> any queues to bind it.)

You can simulate this setup with PerfTest [1]:

./runjava.sh com.rabbitmq.examples.PerfTest -y 0 -t "fanout"

and tweak various options such as message size.

For example, running PerfTest with the options above stabilizes
at about 65-70 K messages per second on my (1 gen old, not maxed-out)
MacBook Pro.

1. http://www.rabbitmq.com/java-tools.html
--
MK

Staff Software Engineer, Pivotal/RabbitMQ
_______________________________________________
rabbitmq-discuss mailing list has moved to https://groups.google.com/forum/#!forum/rabbitmq-users,
please subscribe to the new list!

[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss