AMQP library APIs

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

AMQP library APIs

alan.antonuk
I've been doing a bit of thinking about building a more fully-featured native AMQP client library in C/C++, and have been pondering where to go with designing the public API for the library. 

A question for the authors of AMQP client libraries: if you could change the the public API to your library, what would you change? (and why?).

Thanks
-Alan

_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: AMQP library APIs

Michael Klishin-2
Alan Antonuk:

> A question for the authors of AMQP client libraries: if you could change the the public API to your library, what would you change? (and why?).

I wouldn't change much of the public API but have some recommendations about what clients
should focus on (according to me).

In most AMQP 0-9-1 clients at the moment, the single most painful thing is
network failure recovery. I believe this is not a trivial matter (protocol state does isn't helping, too), and will be harder to get right in
C and C++, but at the moment, most clients in garbage collected languages
don't even try.

amqp gem, Bunny serve as good examples of how it can be made much easier
for common cases. I hope to make HotBunnies and Langohr support the same
automatic recovery process fairly soon.

Hopefully some time after that there will be enough evidence to consider
adding the same set of features to Java and .NET clients, as well as Pika and amqp.node.

Some clients try to introduce their own abstractions to be more convenient. It's OK as long
as they are clearly documented and you still know how to access every protocol feature
there is.

So, expose the protocol as it is and make it as easy as possible to recover from network
failures. Then document it well.

HTH.
--
MK


_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss

signature.asc (506 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: AMQP library APIs

alan.antonuk

On Wed, Aug 7, 2013 at 1:11 PM, Michael Klishin <[hidden email]> wrote:

amqp gem, Bunny serve as good examples of how it can be made much easier
for common cases. I hope to make HotBunnies and Langohr support the same
automatic recovery process fairly soon.

Interesting, whole connection recovery is something I haven't really considered.  Any reason why individual channel recovery on error isn't implemented?

I guess one thought I have had is to abstract away AMQP channels from the public API.

_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: AMQP library APIs

Michael Klishin-2
Alan Antonuk:

> Interesting, whole connection recovery is something I haven't really considered.  Any reason why individual channel recovery on error isn't implemented?

Bunny's automatic recovery is for network failures, not connection exceptions (which are rare
and severe, and should be as visible as possible).

> I guess one thought I have had is to abstract away AMQP channels from the public API.

I don't think this is a good idea. First, it is part of the protocol
and developers may want to use that. Second, error handling for operations such as queue.declare
or basic.publish, is per-channel. Finally, basic.qos setting, delivery tags and publisher confirms
are all per-channel.

Bunny before 0.9 tried to completely hide channels in the API. I don't think it's gained it any
new users but it surely annoyed those who needed to use more than one channel. Shaving off one
line from "Hello, world" is as much as it did.

Bunny 0.9 requires opening channels explicitly and I haven't heard a single complaint about that.
--
MK


_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss

signature.asc (506 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: AMQP library APIs

Chris-3
In reply to this post by alan.antonuk
+1 for connection recovery and automatic fail-over in a cluster.  In my opinion, this should automagically connect/reconnect to the server, redeclare and/or rebind any non-durable objects, and reconsume from the queues.  It should also reconsume if a master queue dies and a slave queue is promoted.  

I'm working on a project that uses amqp libraries in Java, C#, C++, and node.js, and it's a pain to have to handle this ourselves in each (except node-amqp, which mostly works for this).  On the other hand, it's free, so I try not to complain too much. ;-)

-Chris



On Wed, Aug 7, 2013 at 5:27 PM, Alan Antonuk <[hidden email]> wrote:

On Wed, Aug 7, 2013 at 1:11 PM, Michael Klishin <[hidden email]> wrote:

amqp gem, Bunny serve as good examples of how it can be made much easier
for common cases. I hope to make HotBunnies and Langohr support the same
automatic recovery process fairly soon.

Interesting, whole connection recovery is something I haven't really considered.  Any reason why individual channel recovery on error isn't implemented?

I guess one thought I have had is to abstract away AMQP channels from the public API.

_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss



_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: AMQP library APIs

Daniel Pocock
In reply to this post by alan.antonuk


On 07/08/13 21:16, Alan Antonuk wrote:
> I've been doing a bit of thinking about building a more fully-featured
> native AMQP client library in C/C++, and have been pondering where to go
> with designing the public API for the library.
>
> A question for the authors of AMQP client libraries: if you could change
> the the public API to your library, what would you change? (and why?).
>

Here is some feedback from the non-AMQP world

As mentioned in the other thread, I'm working with things like SIP and
boost::asio (sometimes together)

asio and boost::asio are very compelling for asynchronous programming
and distributed systems.

Not so long ago I was looking for a WebSocket client library and I came
across websocketpp: it basically builds on asio to provide an async
WebSockets API.  It was really trivial for me to take that and then add
in SIP over WebSocket, you can see how it works in this example:

https://github.com/zaphoyd/websocketpp/tree/master/examples/sip_client

It would be really nice to have a similar paradigm for AMQP programming,
e.g. receiving callbacks for different types of events (message
receiving, failures, etc)

_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: AMQP library APIs

alan.antonuk
In reply to this post by Michael Klishin-2
On Wed, Aug 7, 2013 at 2:43 PM, Michael Klishin <[hidden email]> wrote:

Bunny's automatic recovery is for network failures, not connection exceptions (which are rare
and severe, and should be as visible as possible).

I agree with you on the connection exceptions, in that case it makes a lot of sense to be loud.  However I was thinking in the case of a channel exception being thrown.  E.g., you have a consumer associated with a channel, and you happen to publish to a non-existent exchange, whoops that consumer goes away. Obviously the client code should be notified of the failure, but from an API perspective its annoying to have something that is potentially unrelated be affected and have to be recreated.


I don't think this is a good idea. First, it is part of the protocol
and developers may want to use that. Second, error handling for operations such as queue.declare
or basic.publish, is per-channel. Finally, basic.qos setting, delivery tags and publisher confirms
are all per-channel.

That makes me wonder what is it that developers use channels for?  My understanding is they're were designed to be an 'error scope' at a protocol level. I don't see a lot of developers using them that way, and sometimes its the source of a bit of pain (having to recreate any resources that get destroyed as a result of a channel exception, or in bad cases, their whole app aborts because the channel dies).

My thinking is that an API could be created that internally uses channels intelligently to give more native feel to error handling. One example of changing the API might be: instead of apply basic.qos to a channel, you apply it to a consumer, and internally the library maps a consumer to a channel, so a BasicQos on a consumer.  The same can be done for other aspects of the protocol (this is what I ended up doing with SimpleAmqpClient).  A downside being the API may not map 1:1  to a lot of the documentation that is out there.


_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: AMQP library APIs

Michael Klishin-2
Alan Antonuk:

> That makes me wonder what is it that developers use channels for?  My understanding is they're were designed to be an 'error scope' at a protocol level. I don't see a lot of developers using them that way, and sometimes its the source of a bit of pain (having to recreate any resources that get destroyed as a result of a channel exception, or in bad cases, their whole app aborts because the channel dies).

For separating things that may require different error handling.

> My thinking is that an API could be created that internally uses channels intelligently to give more native feel to error handling. One example of changing the API might be: instead of apply basic.qos to a channel, you apply it to a consumer, and internally the library maps a consumer to a channel, so a BasicQos on a consumer.  The same can be done for other aspects of the protocol (this is what I ended up doing with SimpleAmqpClient).  A downside being the API may not map 1:1  to a lot of the documentation that is out there.

Nearly every other client exposes basic.qos on a channel, not a consumer. Not doing the same
will likely only cause more confusion at this point, not less.
--
MK


_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss

signature.asc (506 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: AMQP library APIs

alan.antonuk
Excellent points.  

Having had a chance to sleep on it - its seems to me you need to have an API that exposes the protocol at some level. A layer that abstracts away the concept of channels can always be layered over the top of that.


On Thu, Aug 8, 2013 at 6:58 AM, Michael Klishin <[hidden email]> wrote:
Alan Antonuk:

> That makes me wonder what is it that developers use channels for?  My understanding is they're were designed to be an 'error scope' at a protocol level. I don't see a lot of developers using them that way, and sometimes its the source of a bit of pain (having to recreate any resources that get destroyed as a result of a channel exception, or in bad cases, their whole app aborts because the channel dies).

For separating things that may require different error handling.

> My thinking is that an API could be created that internally uses channels intelligently to give more native feel to error handling. One example of changing the API might be: instead of apply basic.qos to a channel, you apply it to a consumer, and internally the library maps a consumer to a channel, so a BasicQos on a consumer.  The same can be done for other aspects of the protocol (this is what I ended up doing with SimpleAmqpClient).  A downside being the API may not map 1:1  to a lot of the documentation that is out there.

Nearly every other client exposes basic.qos on a channel, not a consumer. Not doing the same
will likely only cause more confusion at this point, not less.
--
MK


_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss



_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: AMQP library APIs

Gordon Sim-2
In reply to this post by alan.antonuk
On 08/07/2013 08:16 PM, Alan Antonuk wrote:
> I've been doing a bit of thinking about building a more fully-featured
> native AMQP client library in C/C++, and have been pondering where to go
> with designing the public API for the library.
>
> A question for the authors of AMQP client libraries: if you could change
> the the public API to your library, what would you change? (and why?).

I've been involved in the c++ AMQP client (qpid::messaging) at Apache
Qpid. The ability to integrate with external event loops and a more
complete model for asynchronous control flow would be the key things I
would change. Also providing more detailed information about the status
of outgoing messages.

_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: AMQP library APIs

alan.antonuk
On Fri, Aug 9, 2013 at 4:25 AM, Gordon Sim <[hidden email]> wrote:
I've been involved in the c++ AMQP client (qpid::messaging) at Apache Qpid. The ability to integrate with external event loops and a more complete model for asynchronous control flow would be the key things I would change. Also providing more detailed information about the status of outgoing messages.


Could you expand on what kind of information you would want about the status of outgoing messages? 


_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: AMQP library APIs

Gordon Sim-2
On 08/09/2013 07:54 PM, Alan Antonuk wrote:

> On Fri, Aug 9, 2013 at 4:25 AM, Gordon Sim <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>         I've been involved in the c++ AMQP client (qpid::messaging) at
>         Apache Qpid. The ability to integrate with external event loops
>         and a more complete model for asynchronous control flow would be
>         the key things I would change. Also providing more detailed
>         information about the status of outgoing messages.
>
>
>
> Could you expand on what kind of information you would want about the
> status of outgoing messages?

Basically the ability to determine which asynchronously sent messages
are accepted or rejected (or indeed potentially have the ability to
associate some other statuses with them). Right now qpid::messaging
allows you to track the acceptance of messages, but rejecting messages
can really only be handled be ending the session (whereas it would be
nice to merely indicate those that weren't accepted but allow the sender
to continue).

_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss