RabbitMQ and Avro

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

RabbitMQ and Avro

Mark
Anyone use Avro with RabbitMQ for event aggregation. Is so, would you mind sharing you experiences?
_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: RabbitMQ and Avro

Jerry Kuch-2
We've certainly heard of people doing it...

On Thu, Jun 13, 2013 at 1:55 PM, Mark <[hidden email]> wrote:
Anyone use Avro with RabbitMQ for event aggregation. Is so, would you mind sharing you experiences?
_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss


_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: RabbitMQ and Avro

michaelklishin
In reply to this post by Mark

2013/6/14 Mark <[hidden email]>
Anyone use Avro with RabbitMQ for event aggregation. Is so, would you mind sharing you experiences?

Mark,

RabbitMQ is content agnostic so you can use any serialization library with it.

For event aggregation, protocol buffers worked well for me. Protocol buffers help enforce
schema to some extent. Because all events share some properties (e.g. timestamp, type),
having some structure enforced by the message format is helpful in the long term.

_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: RabbitMQ and Avro

Mark
Thanks Michael. Avro will also enforce a schema hence the whole reason I was asking over something more generic like JSON. The reason Im leaning towards Avro over PB is its tight integration with Hadoop. Some of these events will be shipped over for hadoop for further processing.

Just wanted to make sure I was going in the right direction.

On Jun 13, 2013, at 2:12 PM, Michael Klishin <[hidden email]> wrote:


2013/6/14 Mark <[hidden email]>
Anyone use Avro with RabbitMQ for event aggregation. Is so, would you mind sharing you experiences?

Mark,

RabbitMQ is content agnostic so you can use any serialization library with it.

For event aggregation, protocol buffers worked well for me. Protocol buffers help enforce
schema to some extent. Because all events share some properties (e.g. timestamp, type),
having some structure enforced by the message format is helpful in the long term.
_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss


_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Getting High Memory Watermark Status

Umutcan
In reply to this post by Mark
Hello,

I am trying to find a way to get high memory watermak alarm status via
rabbitmqctl or another tool. I couldn't find it in the documents.

How can I get status of any alarm (socket descriptors, diskspace, memory
etc.)?

Thanks,

Umutcan.
_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Getting High Memory Watermark Status

Brett Cameron
Entries written to the RabbitMQ log file are also published to the exchange "amq.rabbitmq.log". If you create a couple of queues and bind them to this exchange then you can consume the feed. Note that amq.rabbitmq.log is a topic exchange, so you can filter out the specific event types that you want (error, warning, ot info). See http://videlalvaro.github.io/2010/09/haskell-and-rabbitmq.html for a simple example in Haskell (but equally as simple with other client languages). I guess that alarms probably fall into the "warning" category, but don't quote me on it...
 
Brett

On Fri, Jun 14, 2013 at 9:46 AM, Umutcan <[hidden email]> wrote:
Hello,

I am trying to find a way to get high memory watermak alarm status via rabbitmqctl or another tool. I couldn't find it in the documents.

How can I get status of any alarm (socket descriptors, diskspace, memory etc.)?

Thanks,

Umutcan.
_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss


_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Getting High Memory Watermark Status

michaelklishin
In reply to this post by Umutcan
2013/6/14 Umutcan <[hidden email]>
I am trying to find a way to get high memory watermak alarm status via rabbitmqctl or another tool. I couldn't find it in the documents.

RabbitMQ HTTP API [1] provides this and related information.

See /api/nodes/{name}.

--
MK

http://github.com/michaelklishin
http://twitter.com/michaelklishin

_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Getting High Memory Watermark Status

Umutcan
On 06/14/2013 11:01 AM, Michael Klishin wrote:
2013/6/14 Umutcan <[hidden email]>
I am trying to find a way to get high memory watermak alarm status via rabbitmqctl or another tool. I couldn't find it in the documents.

RabbitMQ HTTP API [1] provides this and related information.

See /api/nodes/{name}.


Thanks. This is what I am looking for.

_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss