MQTT plugin, QoS 1, Android, not consuming on reconnect

[brydev]

Hi,
I've implemented a Paho MQTT service in Android which is currently unable to consume unacked messages upon reconnect to the RabbitMQ broker with MQTT plugin. If I force the callback when the connectivity changes, it will consume the missed messages.

Sometimes forcing the callback will not work and the messages remain unacked on the broker. Due to the unusual behaviour of when the message consumes from the queue makes the service unreliable. The messages are QoS 1.

Also upon reconnect the messages will arrive as normal to the device but the missed ones never get re-queued.

The application is currently behaving fine when receiving messages from a mosquitto broker so am I missing any implementation?

Is this a bug or is there a setting I have possible missed for consuming messages from my topic.

 

[Michael Klishin-2]

On 25 July 2014 at 23:09:44, brydev ([hidden email]) wrote:

> > I've implemented a Paho MQTT service in Android which is currently
> unable to
> consume unacked messages upon reconnect to the RabbitMQ broker
> with MQTT
> plugin. If I force the callback when the connectivity changes,
> it will
> consume the missed messages.
>
> Sometimes forcing the callback will not work and the messages
> remain unacked
> on the broker. Due to the unusual behaviour of when the message
> consumes
> from the queue makes the service unreliable. The messages are
> QoS 1.

What do you mean by "forcing the callback"?

> Also upon reconnect the messages will arrive as normal to the
> device but the
> missed ones never get re-queued.
>
> The application is currently behaving fine when receiving messages
> from a
> mosquitto broker so am I missing any implementation?
>
> Is this a bug or is there a setting I have possible missed for consuming
> messages from my topic.

Can you post some example code that demonstrates the issue? What's in RabbitMQ log
files?
--
MK

Staff Software Engineer, Pivotal/RabbitMQ
_______________________________________________
rabbitmq-discuss mailing list has moved to https://groups.google.com/forum/#!forum/rabbitmq-users,
please subscribe to the new list!

[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss

[brydev]

So the log just has this output from the time of testing,
=INFO REPORT==== 25-Jul-2014::17:08:37 ===
closing AMQP connection <0.447.0> (127.0.0.1:49407 -> 127.0.0.1:5672)

=INFO REPORT==== 25-Jul-2014::17:08:50 ===
accepting MQTT connection (192.168.1.50:50777 -> 192.168.1.239:1883)

=WARNING REPORT==== 25-Jul-2014::17:08:50 ===
MQTT disconnecting duplicate client id "sub1_19c6ee855806c472" ("
192.168.100.123:40335 -> 192.168.1.239:1883")

This gets called after a disconnect, if I call
mClient.setCallback(MqttService.this); it forces the message queue onto the
device but it only works sometimes, usually only on a fresh start.

try {
mClient = new MqttClient(url,mDeviceId,mMemStore);
} catch (MqttException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
mConnHandler.post(new Runnable() {
@Override
public void run() {
try {
mClient.connect(mOpts);

mClient.subscribe("hello.alert", 1);


mClient.setCallback(MqttService.this);


mStarted = true;

startKeepAlives();
} catch(MqttException e) {
e.printStackTrace();
}
}
});
}

Thanks


On Fri, Jul 25, 2014 at 8:43 PM, Michael Klishin-2 [via RabbitMQ] <
[e-mail]> wrote:

> On 25 July 2014 at 23:09:44, brydev ([hidden email]
> <http://user/SendEmail.jtp?type=node&node=36890&i=0>) wrote:
>
> > > I've implemented a Paho MQTT service in Android which is currently
> > unable to
> > consume unacked messages upon reconnect to the RabbitMQ broker
> > with MQTT
> > plugin. If I force the callback when the connectivity changes,
> > it will
> > consume the missed messages.
> >
> > Sometimes forcing the callback will not work and the messages
> > remain unacked
> > on the broker. Due to the unusual behaviour of when the message
> > consumes
> > from the queue makes the service unreliable. The messages are
> > QoS 1.
>
> What do you mean by "forcing the callback"?
>
> > Also upon reconnect the messages will arrive as normal to the
> > device but the
> > missed ones never get re-queued.
> >
> > The application is currently behaving fine when receiving messages
> > from a
> > mosquitto broker so am I missing any implementation?
> >
> > Is this a bug or is there a setting I have possible missed for consuming
> > messages from my topic.
>
> Can you post some example code that demonstrates the issue? What's in
> RabbitMQ log
> files?
> --
> MK
>
> Staff Software Engineer, Pivotal/RabbitMQ
> _______________________________________________
> rabbitmq-discuss mailing list has moved to
> https://groups.google.com/forum/#!forum/rabbitmq-users,
> please subscribe to the new list!
>
> [hidden email] <http://user/SendEmail.jtp?type=node&node=36890&i=1>
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
>
> ------------------------------
>  If you reply to this email, your message will be added to the discussion
> below:
>
> http://rabbitmq.1065348.n5.nabble.com/MQTT-plugin-QoS-1-Android-not-consuming-on-reconnect-tp36888p36890.html
>  To unsubscribe from MQTT plugin, QoS 1, Android, not consuming on
> reconnect, click here
> <http://rabbitmq.1065348.n5.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=36888&code=YmNsaWZmZUBnbWFpbC5jb218MzY4ODh8MTQ1NzU4Njg1Mg==>
> .
> NAML
> <http://rabbitmq.1065348.n5.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>

[Michael Klishin-2]

 On 25 July 2014 at 23:54:35, brydev ([hidden email]) wrote:

> > =WARNING REPORT==== 25-Jul-2014::17:08:50  
> ===
> MQTT disconnecting duplicate client id "sub1_19c6ee855806c472"  
> ("10.144.103.154:40335(http://10.144.103.154:40335)  
> -> 143.239.72.239:1883(http://143.239.72.239:1883)")  
>  
>  
> This gets called after a disconnect, if I call mClient.setCallback(MqttService.this);  
> it forces the message queue onto the device but it only works sometimes,  
> usually only on a fresh start.

So if I understand it correctly, when a client (Paho Java) loses connection,
you get no deliveries for re-registered subscription with the same client id,
but creating a completely new connection with the same client id makes
things work some of the time?

MQTT spec requires brokers to disconnect existing clients with the same client ID
when handling CONNECT:

«If a client with the same Client ID is already connected to the server, the "older" client
must be disconnected by the server before completing the CONNECT flow of the new
client.»

So I'm curious if the issue may have something to do with lost connections not being
detected fast enough.

How do you simulate lost client connections? What [MQTT] heartbeat value do you use?
--  
MK  

Staff Software Engineer, Pivotal/RabbitMQ
_______________________________________________
rabbitmq-discuss mailing list has moved to https://groups.google.com/forum/#!forum/rabbitmq-users,
please subscribe to the new list!

[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss

[Michael Klishin-2]

+rabbitmq-discuss

On 26 July 2014 at 00:18:32, Bryan Cliffe ([hidden email]) wrote:
> > I'm disconnecting from wifi and reconnecting through cellular
> to test the robustness of the connection when the I.P changes.
> I wanted to see how the paho java client would handle message delivery
> on QoS 1 to Android. I'm then filling the queue during the handover
> phase and I can't consume on reconnect. I'm using two different
> values for the heartbeat which is 240000ms and 30000ms but I'm
> having the same problem with both. It would be interesting if
> it's not detecting the change over from wifi to cellular.

I'm not familiar with how exactly switching from WiFi to cellular data affects
TCP connections on Android but in general, this is exactly the kind of issue
heartbeats in MQTT are for.

Try using a value of 5-10 seconds. I'm all for improving RabbitMQ MQTT plugin
for this case but if Rabbit doesn't get a notification from the OS that the peer
disconnected, a low heartbeat interval is the only possible solution.
--
MK

Staff Software Engineer, Pivotal/RabbitMQ
_______________________________________________
rabbitmq-discuss mailing list has moved to https://groups.google.com/forum/#!forum/rabbitmq-users,
please subscribe to the new list!

[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss

[brydev]

Thanks for the help in understanding the issue better.

I won't be testing again for a few days so I will let you know how the 5 - 10 sec heartbeat works out. 

On Fri, Jul 25, 2014 at 9:24 PM, Michael Klishin-2 [via RabbitMQ] <[hidden email]> wrote:

+rabbitmq-discuss

On 26 July 2014 at 00:18:32, Bryan Cliffe ([hidden email]) wrote:
> > I'm disconnecting from wifi and reconnecting through cellular
> to test the robustness of the connection when the I.P changes.
> I wanted to see how the paho java client would handle message delivery
> on QoS 1 to Android. I'm then filling the queue during the handover
> phase and I can't consume on reconnect. I'm using two different
> values for the heartbeat which is 240000ms and 30000ms but I'm
> having the same problem with both. It would be interesting if
> it's not detecting the change over from wifi to cellular.

I'm not familiar with how exactly switching from WiFi to cellular data affects
TCP connections on Android but in general, this is exactly the kind of issue
heartbeats in MQTT are for.

Try using a value of 5-10 seconds. I'm all for improving RabbitMQ MQTT plugin
for this case but if Rabbit doesn't get a notification from the OS that the peer
disconnected, a low heartbeat interval is the only possible solution.

--
MK

Staff Software Engineer, Pivotal/RabbitMQ

_______________________________________________
rabbitmq-discuss mailing list has moved to https://groups.google.com/forum/#!forum/rabbitmq-users,
please subscribe to the new list!

[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss

---

If you reply to this email, your message will be added to the discussion below:

http://rabbitmq.1065348.n5.nabble.com/MQTT-plugin-QoS-1-Android-not-consuming-on-reconnect-tp36888p36895.html

To unsubscribe from MQTT plugin, QoS 1, Android, not consuming on reconnect, click here.
NAML

[brydev]

I have tested the lower heartbeat interval, having it as low as 1 second but the RabbitMQ broker is still disconnecting the new connection after an attempt in reconnect. Having a heartbeat interval this low also affects the power usage on the device. I was wondering maybe a possible solution could be a policy whereby if there is a duplicate connection attempt on the broker to then disconnect the oldest connection. Would this cause security issues? But in an environment where the nature of a mobile device is to change between I.P's regularly require this set up?

Anyway thanks again

[Michael Klishin-2]

On 27 July 2014 at 17:07:14, brydev ([hidden email]) wrote:

> > I have tested the lower heartbeat interval, having it as low  
> as 1 second but
> the RabbitMQ broker is still disconnecting the new connection  
> after an
> attempt in reconnect. Having a heartbeat interval this low also  
> affects the
> power usage on the device. I was wondering maybe a possible solution  
> could
> be a policy whereby if there is a duplicate connection attempt  
> on the broker
> to then disconnect the oldest connection.

MQTT 3.1 requires brokers to disconnect an old connection with the same
client id but (as I understand it) only after successful authentication.

Are you saying that it's not what you're observing? 
--  
MK  

Staff Software Engineer, Pivotal/RabbitMQ
_______________________________________________
rabbitmq-discuss mailing list has moved to https://groups.google.com/forum/#!forum/rabbitmq-users,
please subscribe to the new list!

[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss

[brydev]

The service on the application is unable to reconnect again after the duplicate id is detected. It reconnects sometimes but if it does it doesn't consume the missed messages on the queue. The log doesn't give me anything else besides duplicate client id disconnected. And my device is throwing the following error on loop and is unable to reconnect.

private boolean isConnected() {

if(mStarted && mClient != null && !mClient.isConnected()) {

Log.i(DEBUG_TAG,"Mismatch between what we think is connected and what is connected");

}

So I can't tell if the old, new or both connections are disconnected from the broker.  

On Sun, Jul 27, 2014 at 3:25 PM, Michael Klishin-2 [via RabbitMQ] <[hidden email]> wrote:

On 27 July 2014 at 17:07:14, brydev ([hidden email]) wrote:

> > I have tested the lower heartbeat interval, having it as low  
> as 1 second but
> the RabbitMQ broker is still disconnecting the new connection  
> after an
> attempt in reconnect. Having a heartbeat interval this low also  
> affects the
> power usage on the device. I was wondering maybe a possible solution  
> could
> be a policy whereby if there is a duplicate connection attempt  
> on the broker
> to then disconnect the oldest connection.

MQTT 3.1 requires brokers to disconnect an old connection with the same

client id but (as I understand it) only after successful authentication.

Are you saying that it's not what you're observing? 

--  
MK  

Staff Software Engineer, Pivotal/RabbitMQ

_______________________________________________
rabbitmq-discuss mailing list has moved to https://groups.google.com/forum/#!forum/rabbitmq-users,
please subscribe to the new list!

[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss

---

If you reply to this email, your message will be added to the discussion below:

http://rabbitmq.1065348.n5.nabble.com/MQTT-plugin-QoS-1-Android-not-consuming-on-reconnect-tp36888p36903.html

To unsubscribe from MQTT plugin, QoS 1, Android, not consuming on reconnect, click here.
NAML

[Michael Klishin-2]

 On 27 July 2014 at 18:36:16, brydev ([hidden email]) wrote:

> > The service on the application is unable to reconnect again  
> after the duplicate id is detected. It reconnects sometimes  
> but if it does it doesn't consume the missed messages on the queue.  
> The log doesn't give me anything else besides duplicate client  
> id disconnected. And my device is throwing the following error  
> on loop and is unable to reconnect.
>  
> private boolean isConnected() {
> if(mStarted && mClient != null && !mClient.isConnected())  
> {
> Log.i(DEBUG_TAG,"Mismatch between what we think is connected  
> and what is connected");
> }
>  
>  
> So I can't tell if the old, new or both connections are disconnected  
> from the broker.

We will investigate but the fundamental issue seems to be the same on both ends:
both peers do not notice connection loss.
--  
MK  

Staff Software Engineer, Pivotal/RabbitMQ
_______________________________________________
rabbitmq-discuss mailing list has moved to https://groups.google.com/forum/#!forum/rabbitmq-users,
please subscribe to the new list!

[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss

[kelvin]

hi, brydev.
i have met the same problem these days, i wonder how you fix it.
thx~