Librabbitmq-c API

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Librabbitmq-c API

Alex P-7
Hi,

We've started using librabbitmq-c on ARM device with 200MHZ and 64mb
of RAM, works very well, it's so nice to know that it's available for
embedding & stable enough to use in industrial projects. Actually,
librabbitmq together with v8 opens up a whole new world of
opportunities and eases embedded development a lot, thanks for the
hard work done on the lib!

So far we went with a toolset that's defined in ./examples and ./
tools, but, for instance in example for listenqueue framing is done
manually. Is there a high-level API for receiving a whole message in
case it's there and getting AMQP_BASIC_GET_EMPTY_METHOD or anything
else if body is empty / nothing received?

amqp_basic_get(conn, 1, queue, 1); seemed to be a good candidate for
that, but still in codegen.py it's marked as one of those methods that
should not be exposed as an API.

So, my question would be - what is the best way to receive messages?
Should we handle framing manually, as it's done in examples? If not,
what are the api methods that we should stick to?

Thanks
_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Librabbitmq-c API

David Wragg-4
Alex P <[hidden email]> writes:
> So far we went with a toolset that's defined in ./examples and ./
> tools, but, for instance in example for listenqueue framing is done
> manually. Is there a high-level API for receiving a whole message in
> case it's there and getting AMQP_BASIC_GET_EMPTY_METHOD or anything
> else if body is empty / nothing received?

You're right, the support for consuming messages is quite low-level.

There's no particularly good reason for that, although designing an
efficient general purpose high-level API is non-trivial.  You can write
easily write one that meets the needs of your application though, and I
would recommend you do so.

For example, the code in tools/ is able to directly output message
contents from the memory buffers managed by librabbitmq.  The copy_body
function in tools/common.c encapsulates this.

But other applications might want to malloc a single block of memory for
each incoming message body, and copy the body data from the frames into
that.  Etc.

> amqp_basic_get(conn, 1, queue, 1); seemed to be a good candidate for
> that, but still in codegen.py it's marked as one of those methods that
> should not be exposed as an API.

amqp_basic_get is exposed as an API.  It's defined in amq_api.c, and is
definitely intended for use by applications.

It is marked for special handling in codegen.py simply because the
autogenerated definition would not be correct (the comment there is
misleading; the real issue is that basic.get has two possible responses:
AMQP_BASIC_GET_OK_METHOD and AMQP_BASIC_GET_EMPTY_METHOD).

> So, my question would be - what is the best way to receive messages?
> Should we handle framing manually, as it's done in examples? If not,
> what are the api methods that we should stick to?

Do it manually, but by writing your own code to encapsulate the
low-level frame handling.  librabbitmq might get a higher-level API one
day, and by encapsulating the code that uses the current API, you'll
find it easier to transition.

David

--
David Wragg
Staff Engineer, RabbitMQ
VMware, Inc.
_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Librabbitmq-c API

Alex P-7
Thank you David.

Actually, that was exactly what we ended up doing, 

As far as I understood, that even though Librabbitmq is event-driven, it doesn't mean it's non-blocking, right?
For instance, wait_frame_inner is blocking until we recv() is responding with something useful. 

Are there ways to work it around? We actually need some non-blocking approach, since we want to be able to send heartbeats and do many other things in same thread, while we have no data available. 

Actually, just changing that function would change many things. As far as I can tell, Send() calls are currently blocking as well, even though they're unlikely to block cycle for long time.

Should we just use several threads and create a connection per thread, or there are better approaches that were built into the lib? 

Thanks for the help, again.
It seems that something is causing basic_get to eat out memory when ran in cycle. If i manage to reproduce it without app itself and have more info, i'll file a bug-report. For now it may very well be my mistake(  

_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Librabbitmq-c API

Alex P-7
Actually, seems that there are ways to do so..

I tried using non-blocking recv and then setting buffer limits, but that may have side effects because of framing and other things, that'd be very bad.

Another thing, it's still possible to use select(2) on socket in order to receive it's status, and allow or disallow starting amqp_simple_wait_frame(conn, &frame). 

Do you think that may have potential downsides?

Thank you!

_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Librabbitmq-c API

Alex P-7
Sorry for spamming, 

Just another thought, it'd be possible and very easy to create a method amqp_data_available that would tell us whether there's something on the socket before entering blocking call. This way it'll be possible to wait not inside of recv but inside of main loop, but i'm not sure how it would fit into the overall idea.

I sketched it, maybe you would find anything useful there.  https://gist.github.com/1189157

Thanks again!

_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Librabbitmq-c API

David Wragg-4
In reply to this post by Alex P-7
Alex P <[hidden email]> writes:
> As far as I understood, that even though Librabbitmq is event-driven, it
> doesn't mean it's non-blocking, right?

Correct.  librabbitmq does not really support non-blocking use.

> For instance, wait_frame_inner is blocking until we recv() is responding
> with something useful.
>
> Are there ways to work it around? We actually need some non-blocking
> approach, since we want to be able to send heartbeats and do many other
> things in same thread, while we have no data available.
>
> Actually, just changing that function would change many things. As far as I
> can tell, Send() calls are currently blocking as well, even though they're
> unlikely to block cycle for long time.
>
> Should we just use several threads and create a connection per thread, or
> there are better approaches that were built into the lib?

It's better to use threads that to try to persuade librabbitmq to do
non-blocking operation.

> Thanks for the help, again.
> It seems that something is causing basic_get to eat out memory when ran in
> cycle. If i manage to reproduce it without app itself and have more info,
> i'll file a bug-report. For now it may very well be my mistake(

Make sure you call amqp_maybe_release_buffers when you are finished with
each message, as shown at
<https://github.com/rabbitmq/rabbitmq-c/blob/master/tools/consume.c#L149>.

--
David Wragg
Staff Engineer, RabbitMQ
VMware, Inc.
_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss