Order in Queues (FIFO)

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Order in Queues (FIFO)

rabbit15
Hi, I have question about order of messages in queues of RabbitMQ.

Here are my messages

1 {"timestamp":"2013-08-20 11:00:27","type":"published","node":"rabbit@debian","exchange":"FUS","queue":"none","routing_keys":["ctrl"],"properties":{"reply_to":"AVS","correlation_id":"0","content_type":"fuosn"},"payload":"AA=="}
2 {"timestamp":"2013-08-20 11:00:27","type":"received","node":"rabbit@debian","exchange":"FUS","queue":"FUSControlQ","routing_keys":["ctrl"],"properties":{"reply_to":"AVS","correlation_id":"0","content_type":"fuosn"},"payload":"AA=="}
3 {"timestamp":"2013-08-20 11:00:27","type":"published","node":"rabbit@debian","exchange":"AVS","queue":"none","routing_keys":["fu_data"],"properties":{"content_type":"furi"},"payload":"DwABDAAAAAEIAAEAAAABCwACAAAACHRlbXBsYXRlCwADAAAANGh0dHA6Ly9sb2NhbGhvc3Q6ODA4MC9qYm9zc3dzL0ZVUy9mdXNQYWNrYWdlLnBhY2thZ2ULAAQAAAA4aHR0cDovL2xvY2FsaG9zdDo4MDgwL2pib3Nzd3MvRlVTL2Z1c1NpZ25hdHVyZS5zaWduYXR1cmUIAAYAAABACwAHAAAAQDAwMDEwMjAzMDQwNTA2MDcwODA5MEEwQjBDMEQwRTBGMTAxMTEyMTMxNDE1MTYxNzE4MTkxQTFCMUMxRDFFMUYIABQAAAAAAAA="}
4 {"timestamp":"2013-08-20 11:00:27","type":"published","node":"rabbit@debian","exchange":"AVS","queue":"none","routing_keys":["fu_data"],"properties":{"content_type":"furi"},"payload":"DwACDAAAAAEIAAEAAAABCwACAAAACHRlbXBsYXRlCwADAAAANGh0dHA6Ly9sb2NhbGhvc3Q6ODA4MC9qYm9zc3dzL0ZVUy9mdXNQYWNrYWdlLnBhY2thZ2ULAAQAAAA4aHR0cDovL2xvY2FsaG9zdDo4MDgwL2pib3Nzd3MvRlVTL2Z1c1NpZ25hdHVyZS5zaWduYXR1cmUIAAYAAABACwAHAAAAQDAwMDEwMjAzMDQwNTA2MDcwODA5MEEwQjBDMEQwRTBGMTAxMTEyMTMxNDE1MTYxNzE4MTkxQTFCMUMxRDFFMUYIAApSE0xFCAALUhNMRggADAAAAAAIABQAAAAAAAA="}
5 {"timestamp":"2013-08-20 11:00:27","type":"published","node":"rabbit@debian","exchange":"AVS","queue":"none","routing_keys":["fu_data"],"properties":{"content_type":"furi"},"payload":"DwADDAAAAAEIAAEAAAABCAACAAAAAQ8AAwoAAAABAADdzLuqAAEIAAcAAAAACgAIAAAAAAAAHLAMAAkIAAEAAAAAAAgAFAAAAAAADwAEDAAAAAEIAGUAAAABCgBmAADdzLuqAAEIAGcAAAABCABoAAAABgwACggAAQAAAAAACAAUAAAAAAAA"}
6 {"timestamp":"2013-08-20 11:00:27","type":"received","node":"rabbit@debian","exchange":"AVS","queue":"TAVSDataQ","routing_keys":["fu_data"],"properties":{"content_type":"furi"},"payload":"DwACDAAAAAEIAAEAAAABCwACAAAACHRlbXBsYXRlCwADAAAANGh0dHA6Ly9sb2NhbGhvc3Q6ODA4MC9qYm9zc3dzL0ZVUy9mdXNQYWNrYWdlLnBhY2thZ2ULAAQAAAA4aHR0cDovL2xvY2FsaG9zdDo4MDgwL2pib3Nzd3MvRlVTL2Z1c1NpZ25hdHVyZS5zaWduYXR1cmUIAAYAAABACwAHAAAAQDAwMDEwMjAzMDQwNTA2MDcwODA5MEEwQjBDMEQwRTBGMTAxMTEyMTMxNDE1MTYxNzE4MTkxQTFCMUMxRDFFMUYIAApSE0xFCAALUhNMRggADAAAAAAIABQAAAAAAAA="}
7 {"timestamp":"2013-08-20 11:00:27","type":"received","node":"rabbit@debian","exchange":"AVS","queue":"TAVSDataQ","routing_keys":["fu_data"],"properties":{"content_type":"furi"},"payload":"DwADDAAAAAEIAAEAAAABCAACAAAAAQ8AAwoAAAABAADdzLuqAAEIAAcAAAAACgAIAAAAAAAAHLAMAAkIAAEAAAAAAAgAFAAAAAAADwAEDAAAAAEIAGUAAAABCgBmAADdzLuqAAEIAGcAAAABCABoAAAABgwACggAAQAAAAAACAAUAAAAAAAA"}
8 {"timestamp":"2013-08-20 11:00:27","type":"received","node":"rabbit@debian","exchange":"AVS","queue":"TAVSDataQ","routing_keys":["fu_data"],"properties":{"content_type":"furi"},"payload":"DwABDAAAAAEIAAEAAAABCwACAAAACHRlbXBsYXRlCwADAAAANGh0dHA6Ly9sb2NhbGhvc3Q6ODA4MC9qYm9zc3dzL0ZVUy9mdXNQYWNrYWdlLnBhY2thZ2ULAAQAAAA4aHR0cDovL2xvY2FsaG9zdDo4MDgwL2pib3Nzd3MvRlVTL2Z1c1NpZ25hdHVyZS5zaWduYXR1cmUIAAYAAABACwAHAAAAQDAwMDEwMjAzMDQwNTA2MDcwODA5MEEwQjBDMEQwRTBGMTAxMTEyMTMxNDE1MTYxNzE4MTkxQTFCMUMxRDFFMUYIABQAAAAAAAA="}

problem is that messages order is different while publishing and receiving, not only in rabbit log but as well in applications using it (one connection, one channel per both application)
eq. payload sent to rabbit in in order - line: 3, 4, 5
but receiving is in order line: 6(same msg as 4), 7(same message as 5), 8 (same message as 3) - which gives payloads order 4,5,3

Cheers,
Artur
Reply | Threaded
Open this post in threaded view
|

Re: Order in Queues (FIFO)

Michael Klishin

On oct 4, 2013, at 3:54 p.m., rabbit15 <[hidden email]> wrote:

> eq. payload sent to rabbit in in order - line: 3, 4, 5
> but receiving is in order line: 6(same msg as 4), 7(same message as 5), 8
> (same message as 3) - which gives payloads order 4,5,3

RabbitMQ queues are FIFO but there are several factors that
you need to take into account:

 * Messages can be requeued
 * There can be multiple consumers per queue
 * Multiple messages can be pushed to a consumer in rapid succession.
   Deliveries can be dispatched in a thread pool and processed concurrently (this is indeed what several popular clients do)

What code produced the log above? Printing to stdout is not a very reliable way
when it comes to determining the order. Updating a synchronized collection
of some kind and inspecting it is a more reliable way.

MK




_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss

signature.asc (506 bytes) Download Attachment