Priority queue doesn't seem to work as expected

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Priority queue doesn't seem to work as expected

Nikita Zubrilov
Hello,

I'm using Rabbitmq 3.3.3 on Windows 7.
I installed rabbitmq-priority-queue plugin, enable it, and finally restart
Rabbitmq server. When I list rabbitmq plugins I see => [E] rabbitmq_priority_queue.

I tried the example from GitHub and it WORKS. Though, it is a very very simple example.

But,    when I tried some a little bit complex example, the queue seems to behave in a very strange way.

So, my setup is like this:

First, the common configuration of the queue, just like in the example:
private static final String QUEUE = "my-priority-queue";

    public static void main(String[] argv) throws Exception {
        ConnectionFactory factory = new ConnectionFactory();
        Connection conn = factory.newConnection();
        Channel ch = conn.createChannel();

        Map<String, Object> args = new HashMap<String, Object>();
        args.put("x-max-priority", 10);
        ch.queueDeclare(QUEUE, true, false, false, args);
...

I have a Producer thread that sends messages in intervals of couple of seconds. For each message I define priority from 0 to 10 just like in the example. For example sends 10 messages each 2 seconds:

the run() method looks like this:

while(true){
   for(Message message : MESSAGES){
BasicProperties props = 
MessageProperties.PERSISTENT_BASIC.builder().priority(message.priority)
.build();
        System.out.println("Sent " + message.body);
        ch.basicPublish("", QUEUE, props, message.body.getBytes());
}
Thread.sleep(2000);
}

Also, there is a Consumer thread which waits for incoming messages and prints them on the screen:

inside run() method:
QueueingConsumer consumer = new QueueingConsumer(channel);
channel.basicConsume("hello", true, consumer);
while (true) {
    QueueingConsumer.Delivery delivery = consumer.nextDelivery();
    String message = new String(delivery.getBody());
    System.out.println(" [x] Received '" + message + "'");       
}
nothing special...


And the behavior I get is the following: 
All the messages that was posted BEFORE the Consumer starts to run are received in prioritized order, BUT all the messages that sent AFTER the Consumer starts, are received in the same order they were sent without concerning a priority. Tried to put some Thread.sleep() in different places in code to give the queue some time to reorganize itself, but it didn't work.
So, it seems like priority works only on some static queues without constantly working Consumer and Producer.
Am I right?
Maybe I am doing something wrong?
Can you please give some complex example for using the priority-queue plugin?
In my project, we'll have a number of Producers and a number of Consumers asynchronously putting and taking items from the queue. Does the plugin support the following scenario?
 
Thank you,
Nik

_______________________________________________
rabbitmq-discuss mailing list has moved to https://groups.google.com/forum/#!forum/rabbitmq-users,
please subscribe to the new list!

[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Priority queue doesn't seem to work as expected

Michael Klishin-2
 On 14 July 2014 at 12:13:36, Nikita Zubrilov ([hidden email]) wrote:
> > All the messages that was posted BEFORE the Consumer starts  
> to run are received in prioritized order, BUT all the messages  
> that sent AFTER the Consumer starts, are received in the same  
> order they were sent without concerning a priority. Tried to  
> put some Thread.sleep() in different places in code to give the  
> queue some time to reorganize itself, but it didn't work.
> So, it seems like priority works only on some static queues without  
> constantly working Consumer and Producer.
> Am I right?

See http://markmail.org/thread/afp5d774qrnx6ukh#query:+page:1+mid:zklpftnv3pce5qzz+state:results.

Reordering is performed by the message store.
If a message goes straight to the consumer, no reordering will be done.
--  
MK  

Staff Software Engineer, Pivotal/RabbitMQ
_______________________________________________
rabbitmq-discuss mailing list has moved to https://groups.google.com/forum/#!forum/rabbitmq-users,
please subscribe to the new list!

[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Priority queue doesn't seem to work as expected

Simon MacMullen-2
In reply to this post by Nikita Zubrilov
Hopefully Michael has already answered your main question, but:

On 14/07/2014 9:11AM, Nikita Zubrilov wrote:
> In my project, we'll have a number of Producers and a number of
> Consumers asynchronously putting and taking items from the queue. Does
> the plugin support the following scenario?

Yes.

Cheers, Simon

--
Simon MacMullen
RabbitMQ, Pivotal
_______________________________________________
rabbitmq-discuss mailing list has moved to https://groups.google.com/forum/#!forum/rabbitmq-users,
please subscribe to the new list!

[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Priority queue doesn't seem to work as expected

Nikita Zubrilov
Thank you, it helped!

Regards, Nik


On Mon, Jul 14, 2014 at 12:10 PM, Simon MacMullen <[hidden email]> wrote:
Hopefully Michael has already answered your main question, but:


On 14/07/2014 9:11AM, Nikita Zubrilov wrote:
In my project, we'll have a number of Producers and a number of
Consumers asynchronously putting and taking items from the queue. Does
the plugin support the following scenario?

Yes.

Cheers, Simon

--
Simon MacMullen
RabbitMQ, Pivotal


_______________________________________________
rabbitmq-discuss mailing list has moved to https://groups.google.com/forum/#!forum/rabbitmq-users,
please subscribe to the new list!

[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss