lager vs sasl error log

classic Classic list List threaded Threaded
35 messages Options
12
Reply | Threaded
Open this post in threaded view
|

lager vs sasl error log

Gavin M. Roy
Any plans to move rabbitmq.log to lager (and allow for lager-syslog) while keeping rabbit-sasl.log for crash dumps?

Would be nice to be able to get the plugable loggers and improvements to logging performance lager purports to provide, as well as a more modern, *NIXy logging output.

Cheers,

Gavin

_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: lager vs sasl error log

Gotthard, Petr

Correct me if I’m wrong, but I don’t think that moving to lager requires any changes inside rabbitmq. Since rabbitmq (as far as I know) uses standard Erlang’s logging and since standard logs are redirected to lager, all rabbitmq related logs are processed by lager. Wihout many filtering possibilities though.

 

Petr

 

From: [hidden email] [mailto:[hidden email]] On Behalf Of Gavin M. Roy
Sent: 18. března 2013 17:36
To: Discussions about RabbitMQ
Subject: [rabbitmq-discuss] lager vs sasl error log

 

Any plans to move rabbitmq.log to lager (and allow for lager-syslog) while keeping rabbit-sasl.log for crash dumps?

 

Would be nice to be able to get the plugable loggers and improvements to logging performance lager purports to provide, as well as a more modern, *NIXy logging output.

 

Cheers,


Gavin


_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: lager vs sasl error log

Tim Watson-6
Hi all,

We've discussed moving to a different (non-standard) logging provider before, and I'll raise the issue again. Afaik we've not run into the issues with error_logger very often (i.e., the reasons why lager was invented, viz massive memory consumption due to serialisation of large terms in logging output). The 'filtering possibilities' with lager don't necessarily require the parse_transforms to be used iirc - you can submit a logging request direct to lager and specify the log level at the call site. Again, I'll raise the discussion with the rest of the team and let's see what comes of that.

Cheers,
Tim

On 19 Mar 2013, at 08:48, Gotthard, Petr wrote:

> Correct me if I’m wrong, but I don’t think that moving to lager requires any changes inside rabbitmq. Since rabbitmq (as far as I know) uses standard Erlang’s logging and since standard logs are redirected to lager, all rabbitmq related logs are processed by lager. Wihout many filtering possibilities though.
>  
> Petr
>  
> From: [hidden email] [mailto:[hidden email]] On Behalf Of Gavin M. Roy
> Sent: 18. března 2013 17:36
> To: Discussions about RabbitMQ
> Subject: [rabbitmq-discuss] lager vs sasl error log
>  
> Any plans to move rabbitmq.log to lager (and allow for lager-syslog) while keeping rabbit-sasl.log for crash dumps?
>  
> Would be nice to be able to get the plugable loggers and improvements to logging performance lager purports to provide, as well as a more modern, *NIXy logging output.
>  
> Cheers,
>
> Gavin
> _______________________________________________
> rabbitmq-discuss mailing list
> [hidden email]
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss

_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: lager vs sasl error log

Ben Hood
Hey Tim,

I think a good start would be to implement Gavin's suggestion about having a more UNIXy output, since you could do this without having to change the transport.

If you are aggregating lots of Rabbit logs, this results in lots of double entries such as

Mar 21 12:05:30 a logger:  : =INFO REPORT==== 21-Mar-2013::12:05:29 === 
Mar 21 12:05:30 a logger:  : started TCP Listener on [::]:5672

If I understand Gavin correctly, you could improve the SNR by putting this on one line.

Cheers,

Ben

On Thursday, 21 March 2013 at 12:03, Tim Watson wrote:

Hi all,

We've discussed moving to a different (non-standard) logging provider before, and I'll raise the issue again. Afaik we've not run into the issues with error_logger very often (i.e., the reasons why lager was invented, viz massive memory consumption due to serialisation of large terms in logging output). The 'filtering possibilities' with lager don't necessarily require the parse_transforms to be used iirc - you can submit a logging request direct to lager and specify the log level at the call site. Again, I'll raise the discussion with the rest of the team and let's see what comes of that.

Cheers,
Tim

On 19 Mar 2013, at 08:48, Gotthard, Petr wrote:

Correct me if I’m wrong, but I don’t think that moving to lager requires any changes inside rabbitmq. Since rabbitmq (as far as I know) uses standard Erlang’s logging and since standard logs are redirected to lager, all rabbitmq related logs are processed by lager. Wihout many filtering possibilities though.
Petr
From: [hidden email] [[hidden email]] On Behalf Of Gavin M. Roy
Sent: 18. března 2013 17:36
To: Discussions about RabbitMQ
Subject: [rabbitmq-discuss] lager vs sasl error log
Any plans to move rabbitmq.log to lager (and allow for lager-syslog) while keeping rabbit-sasl.log for crash dumps?
Would be nice to be able to get the plugable loggers and improvements to logging performance lager purports to provide, as well as a more modern, *NIXy logging output.
Cheers,

Gavin
_______________________________________________
rabbitmq-discuss mailing list

_______________________________________________
rabbitmq-discuss mailing list


_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: lager vs sasl error log

Tim Watson-6
Hi Ben,

On 21 Mar 2013, at 12:13, Ben Hood wrote:
> I think a good start would be to implement Gavin's suggestion about having a more UNIXy output, since you could do this without having to change the transport.
>

That's an understand-able requirement, however...

> If you are aggregating lots of Rabbit logs, this results in lots of double entries such as
>
> Mar 21 12:05:30 a logger:  : =INFO REPORT==== 21-Mar-2013::12:05:29 ===
> Mar 21 12:05:30 a logger:  : started TCP Listener on [::]:5672
>

Sure - that multi-line output is what Erlang's built in error_logger generates.

> If I understand Gavin correctly, you could improve the SNR by putting this on one line.
>

To do that, we'd have to change from the built-in error_logger to something else, such as lager. That idea's been up for discussion before and as I said, I'll raise it again.

Cheers,
Tim

> Cheers,
>
> Ben
>
> On Thursday, 21 March 2013 at 12:03, Tim Watson wrote:
>
>> Hi all,
>>
>> We've discussed moving to a different (non-standard) logging provider before, and I'll raise the issue again. Afaik we've not run into the issues with error_logger very often (i.e., the reasons why lager was invented, viz massive memory consumption due to serialisation of large terms in logging output). The 'filtering possibilities' with lager don't necessarily require the parse_transforms to be used iirc - you can submit a logging request direct to lager and specify the log level at the call site. Again, I'll raise the discussion with the rest of the team and let's see what comes of that.
>>
>> Cheers,
>> Tim
>>
>> On 19 Mar 2013, at 08:48, Gotthard, Petr wrote:
>>
>>> Correct me if I’m wrong, but I don’t think that moving to lager requires any changes inside rabbitmq. Since rabbitmq (as far as I know) uses standard Erlang’s logging and since standard logs are redirected to lager, all rabbitmq related logs are processed by lager. Wihout many filtering possibilities though.
>>> Petr
>>> From: [hidden email] [mailto:[hidden email]] On Behalf Of Gavin M. Roy
>>> Sent: 18. března 2013 17:36
>>> To: Discussions about RabbitMQ
>>> Subject: [rabbitmq-discuss] lager vs sasl error log
>>> Any plans to move rabbitmq.log to lager (and allow for lager-syslog) while keeping rabbit-sasl.log for crash dumps?
>>> Would be nice to be able to get the plugable loggers and improvements to logging performance lager purports to provide, as well as a more modern, *NIXy logging output.
>>> Cheers,
>>>
>>> Gavin
>>> _______________________________________________
>>> rabbitmq-discuss mailing list
>>> [hidden email]
>>> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>>
>> _______________________________________________
>> rabbitmq-discuss mailing list
>> [hidden email]
>> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
> _______________________________________________
> rabbitmq-discuss mailing list
> [hidden email]
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss

_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: lager vs sasl error log

Ben Hood
Hey Tim,

Thanks for the heads up.

Cheers,

Ben

On Thursday, 21 March 2013 at 12:36, Tim Watson wrote:

Hi Ben,

On 21 Mar 2013, at 12:13, Ben Hood wrote:
I think a good start would be to implement Gavin's suggestion about having a more UNIXy output, since you could do this without having to change the transport.

That's an understand-able requirement, however...

If you are aggregating lots of Rabbit logs, this results in lots of double entries such as

Mar 21 12:05:30 a logger: : =INFO REPORT==== 21-Mar-2013::12:05:29 ===
Mar 21 12:05:30 a logger: : started TCP Listener on [::]:5672

Sure - that multi-line output is what Erlang's built in error_logger generates.

If I understand Gavin correctly, you could improve the SNR by putting this on one line.

To do that, we'd have to change from the built-in error_logger to something else, such as lager. That idea's been up for discussion before and as I said, I'll raise it again.

Cheers,
Tim

Cheers,

Ben

On Thursday, 21 March 2013 at 12:03, Tim Watson wrote:

Hi all,

We've discussed moving to a different (non-standard) logging provider before, and I'll raise the issue again. Afaik we've not run into the issues with error_logger very often (i.e., the reasons why lager was invented, viz massive memory consumption due to serialisation of large terms in logging output). The 'filtering possibilities' with lager don't necessarily require the parse_transforms to be used iirc - you can submit a logging request direct to lager and specify the log level at the call site. Again, I'll raise the discussion with the rest of the team and let's see what comes of that.

Cheers,
Tim

On 19 Mar 2013, at 08:48, Gotthard, Petr wrote:

Correct me if I’m wrong, but I don’t think that moving to lager requires any changes inside rabbitmq. Since rabbitmq (as far as I know) uses standard Erlang’s logging and since standard logs are redirected to lager, all rabbitmq related logs are processed by lager. Wihout many filtering possibilities though.
Petr
From: [hidden email] [[hidden email]] On Behalf Of Gavin M. Roy
Sent: 18. března 2013 17:36
To: Discussions about RabbitMQ
Subject: [rabbitmq-discuss] lager vs sasl error log
Any plans to move rabbitmq.log to lager (and allow for lager-syslog) while keeping rabbit-sasl.log for crash dumps?
Would be nice to be able to get the plugable loggers and improvements to logging performance lager purports to provide, as well as a more modern, *NIXy logging output.
Cheers,

Gavin
_______________________________________________
rabbitmq-discuss mailing list

_______________________________________________
rabbitmq-discuss mailing list

_______________________________________________
rabbitmq-discuss mailing list

_______________________________________________
rabbitmq-discuss mailing list


_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: lager vs sasl error log

Gavin M. Roy
Fwiw in some advocacy discussions, the logging output was a reason some people have discounted or not moved forward with Rabbit. From conversations I have had recently with people, logging and config file syntax are the bigger approachachability issues with Rabbit.

My goal is to be able to do log aggregation for our largish count of RabbitMQ nodes, which afaik is not possible with Rabbit out of the box. This makes it nearly impossible to stay on top of logs for anything but troubleshooting individual nodes when problems occur.

On Thursday, March 21, 2013, Ben Hood <[hidden email]> wrote:
Hey Tim,

Thanks for the heads up.

Cheers,

Ben

On Thursday, 21 March 2013 at 12:36, Tim Watson wrote:

Hi Ben,

On 21 Mar 2013, at 12:13, Ben Hood wrote:
I think a good start would be to implement Gavin's suggestion about having a more UNIXy output, since you could do this without having to change the transport.

That's an understand-able requirement, however...

If you are aggregating lots of Rabbit logs, this results in lots of double entries such as

Mar 21 12:05:30 a logger: : =INFO REPORT==== 21-Mar-2013::12:05:29 ===
Mar 21 12:05:30 a logger: : started TCP Listener on [::]:5672

Sure - that multi-line output is what Erlang's built in error_logger generates.

If I understand Gavin correctly, you could improve the SNR by putting this on one line.

To do that, we'd have to change from the built-in error_logger to something else, such as lager. That idea's been up for discussion before and as I said, I'll raise it again.

Cheers,
Tim

Cheers,

Ben

On Thursday, 21 March 2013 at 12:03, Tim Watson wrote:

Hi all,

We've discussed moving to a different (non-standard) logging provider before, and I'll raise the issue again. Afaik we've not run into the issues with error_logger very often (i.e., the reasons why lager was invented, viz massive memory consumption due to serialisation of large terms in logging output). The 'filtering possibilities' with lager don't necessarily require the parse_transforms to be used iirc - you can submit a logging request direct to lager and specify the log level at the call site. Again, I'll raise the discussion with the rest of the team and let's see what comes of that.

Cheers,
Tim

On 19 Mar 2013, at 08:48, Gotthard, Petr wrote:

Correct me if I’m wrong, but I don’t think that moving to lager requires any changes inside rabbitmq. Since rabbitmq (as far as I know) uses standard Erlang’s logging and since standard logs are redirected to lager, all rabbitmq related logs are processed by lager. Wihout many filtering possibilities though.
Petr
From: [hidden email] [mailto:[hidden email]] On Behalf Of Gavin M. Roy
Sent: 18. března 2013 17:36
To: Discussions about RabbitMQ
Subject: [rabbitmq-discuss] lager vs sasl error log
Any plans to move rabbitmq.log to lager (and allow for lager-syslog) while keeping rabbit-sasl.log for crash dumps?
Would be nice to be able to get the plugable loggers and improvements to logging performance lager purports to provide, as well as a more modern, *NIXy logging output.
Cheers,

Gavin
_______________________________________________
rabbitmq-discuss mailing list

_______________________________________________
rabbitmq-discuss mailing list

_______________________________________________
rabbitmq-discuss mailing list


--

Gavin M. Roy
Chief Technology Officer


100 Union Square Drive
New Hope, PA 18938
p. +1.215.862.1162 x263
f. +1.215.862.0465

      

The public market leader in social discovery. (NYSE MKT: MEET) 



_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: lager vs sasl error log

Tim Watson-5
Hi Gavin,

On 21 Mar 2013, at 21:02, "Gavin M. Roy" <[hidden email]> wrote:
Fwiw in some advocacy discussions, the logging output was a reason some people have discounted or not moved forward with Rabbit.
From conversations I have had recently with people, logging and config file syntax are the bigger approachachability issues with Rabbit.


That's a pity - the logging is something we could change. The config file syntax, not so much. I suppose we could bootstrap the prelaunch phase to parse and XML file (or whatever) and spew out the native format...

My goal is to be able to do log aggregation for our largish count of RabbitMQ nodes, which afaik is not possible with Rabbit out of the box. This makes it nearly impossible to stay on top of logs for anything but troubleshooting individual nodes when problems occur.

Yeah with the current implementation you'd need to install your own log handler to make that work. In terms of changing the output format though, allowe to play devil's advocate for a moment...

The info/progress/error reports have a lot of data in them, process stats, stack traces, etc. are you wanting to see all that plonked on one single long line? Because I can see how that'd make parsing/aggregating easier, but not reading them. Does the lager output fit exactly with what you want (I've not looks at it or a couple of years)?


On Thursday, March 21, 2013, Ben Hood <[hidden email]> wrote:
Hey Tim,

Thanks for the heads up.

Cheers,

Ben

On Thursday, 21 March 2013 at 12:36, Tim Watson wrote:

Hi Ben,

On 21 Mar 2013, at 12:13, Ben Hood wrote:
I think a good start would be to implement Gavin's suggestion about having a more UNIXy output, since you could do this without having to change the transport.

That's an understand-able requirement, however...

If you are aggregating lots of Rabbit logs, this results in lots of double entries such as

Mar 21 12:05:30 a logger: : =INFO REPORT==== 21-Mar-2013::12:05:29 ===
Mar 21 12:05:30 a logger: : started TCP Listener on [::]:5672

Sure - that multi-line output is what Erlang's built in error_logger generates.

If I understand Gavin correctly, you could improve the SNR by putting this on one line.

To do that, we'd have to change from the built-in error_logger to something else, such as lager. That idea's been up for discussion before and as I said, I'll raise it again.

Cheers,
Tim

Cheers,

Ben

On Thursday, 21 March 2013 at 12:03, Tim Watson wrote:

Hi all,

We've discussed moving to a different (non-standard) logging provider before, and I'll raise the issue again. Afaik we've not run into the issues with error_logger very often (i.e., the reasons why lager was invented, viz massive memory consumption due to serialisation of large terms in logging output). The 'filtering possibilities' with lager don't necessarily require the parse_transforms to be used iirc - you can submit a logging request direct to lager and specify the log level at the call site. Again, I'll raise the discussion with the rest of the team and let's see what comes of that.

Cheers,
Tim

On 19 Mar 2013, at 08:48, Gotthard, Petr wrote:

Correct me if I’m wrong, but I don’t think that moving to lager requires any changes inside rabbitmq. Since rabbitmq (as far as I know) uses standard Erlang’s logging and since standard logs are redirected to lager, all rabbitmq related logs are processed by lager. Wihout many filtering possibilities though.
Petr
From: [hidden email] [mailto:[hidden email]] On Behalf Of Gavin M. Roy
Sent: 18. března 2013 17:36
To: Discussions about RabbitMQ
Subject: [rabbitmq-discuss] lager vs sasl error log
Any plans to move rabbitmq.log to lager (and allow for lager-syslog) while keeping rabbit-sasl.log for crash dumps?
Would be nice to be able to get the plugable loggers and improvements to logging performance lager purports to provide, as well as a more modern, *NIXy logging output.
Cheers,

Gavin
_______________________________________________
rabbitmq-discuss mailing list

_______________________________________________
rabbitmq-discuss mailing list

_______________________________________________
rabbitmq-discuss mailing list


--

Gavin M. Roy
Chief Technology Officer


100 Union Square Drive
New Hope, PA 18938
p. +1.215.862.1162 x263
f. +1.215.862.0465

      

The public market leader in social discovery. (NYSE MKT: MEET) 


_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss

_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: lager vs sasl error log

Ben Hood
Hey Tim,

On Friday, 22 March 2013 at 07:55, Tim Watson wrote:

Yeah with the current implementation you'd need to install your own log handler to make that work. In terms of changing the output format though, allowe to play devil's advocate for a moment...

The info/progress/error reports have a lot of data in them, process stats, stack traces, etc. are you wanting to see all that plonked on one single long line? Because I can see how that'd make parsing/aggregating easier, but not reading them. Does the lager output fit exactly with what you want (I've not looks at it or a couple of years)?
I don't think that everything needs to fit on one line. Longer traces which provide good context should probably remain as multiline entries. However, info items, for example,

Mar 22 22:33:01 a rabbit3:  =INFO REPORT==== 22-Mar-2013::22:33:01 === 
Mar 22 22:33:01 a rabbit3:  Setting permissions for 'guest' in '/' to '.*', '.*', '.*' 

need not get split up over two lines.

With regard to longer traces, it would be nice if they could be handled in such a way that the generic portion of the call stack were trimmed down to something nearer the actual call site, rather that the entire generic call trace. As inspiration, logback mitigates the default "root cause comes last" behavior in Java with the reverse depth layout modifier (rEx{depth}). Lot's of hand waving here though.

Finally, I do have to concur with Gavin on people's perception about Rabbit due to logging - I've have many conversations with people over the years who ask why Rabbit has such bizare logging - I've generally answered that this is an Erlang-ism, and though it is strange, the server kind of serves its primary function well (i.e. delivering messages), so just put up with it.

Cheers,

Ben

_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: lager vs sasl error log

Tim Watson-6
Hi Ben,

Sorry for the delayed response.

On 23 Mar 2013, at 23:06, Ben Hood wrote:
> Hey Tim,
> I don't think that everything needs to fit on one line. Longer traces which provide good context should probably remain as multiline entries. However, info items, for example,
>
> Mar 22 22:33:01 a rabbit3:  =INFO REPORT==== 22-Mar-2013::22:33:01 ===
> Mar 22 22:33:01 a rabbit3:  Setting permissions for 'guest' in '/' to '.*', '.*', '.*'
>
> need not get split up over two lines.
>

Sure I understand the desire for that. We still need to decide whether having an external dependency is something we really want to do though.

> With regard to longer traces, it would be nice if they could be handled in such a way that the generic portion of the call stack were trimmed down to something nearer the actual call site, rather that the entire generic call trace. As inspiration, logback mitigates the default "root cause comes last" behavior in Java with the reverse depth layout modifier (rEx{depth}). Lot's of hand waving here though.
>

I agree, but that's something lager is responsible for so it's not exactly all in our hands.

> Finally, I do have to concur with Gavin on people's perception about Rabbit due to logging - I've have many conversations with people over the years who ask why Rabbit has such bizare logging - I've generally answered that this is an Erlang-ism, and though it is strange, the server kind of serves its primary function well (i.e. delivering messages), so just put up with it.
>

Would it be sufficient to simply rely on lager's error_logger replacement (handler) instead of switching completely over to lager's API? Can we get really specific about what exactly we want to be able to do? I mean, are people asking to be able to configure the logging output (format) or add additional appenders (which was the origin request) or both?

Cheers,
Tim

> Cheers,
>
> Ben
> _______________________________________________
> rabbitmq-discuss mailing list
> [hidden email]
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss

_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: lager vs sasl error log

Tim Watson-6
In reply to this post by Ben Hood
I have set up an experimental repository at https://github.com/hyperthunk/rabbitmq-lager that supports logging to lager via its built in error_logger redirection. You enable via `rabbitmq-plugins enable lager` and configure it in the RabbitMQ config file just like any other plugin, under the key 'lager'. In the morning I'll chat with the guys about integrating this as experimental plugin for the time being, so people can play around and see if it offers the required functionality.

Cheers,
Tim

On 23 Mar 2013, at 23:06, Ben Hood wrote:

> Hey Tim,
> On Friday, 22 March 2013 at 07:55, Tim Watson wrote:
>
>> Yeah with the current implementation you'd need to install your own log handler to make that work. In terms of changing the output format though, allowe to play devil's advocate for a moment...
>>
>> The info/progress/error reports have a lot of data in them, process stats, stack traces, etc. are you wanting to see all that plonked on one single long line? Because I can see how that'd make parsing/aggregating easier, but not reading them. Does the lager output fit exactly with what you want (I've not looks at it or a couple of years)?
> I don't think that everything needs to fit on one line. Longer traces which provide good context should probably remain as multiline entries. However, info items, for example,
>
> Mar 22 22:33:01 a rabbit3:  =INFO REPORT==== 22-Mar-2013::22:33:01 ===
> Mar 22 22:33:01 a rabbit3:  Setting permissions for 'guest' in '/' to '.*', '.*', '.*'
>
> need not get split up over two lines.
>
> With regard to longer traces, it would be nice if they could be handled in such a way that the generic portion of the call stack were trimmed down to something nearer the actual call site, rather that the entire generic call trace. As inspiration, logback mitigates the default "root cause comes last" behavior in Java with the reverse depth layout modifier (rEx{depth}). Lot's of hand waving here though.
>
> Finally, I do have to concur with Gavin on people's perception about Rabbit due to logging - I've have many conversations with people over the years who ask why Rabbit has such bizare logging - I've generally answered that this is an Erlang-ism, and though it is strange, the server kind of serves its primary function well (i.e. delivering messages), so just put up with it.
>
> Cheers,
>
> Ben
> _______________________________________________
> rabbitmq-discuss mailing list
> [hidden email]
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss

_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: lager vs sasl error log

Gavin M. Roy
That's awesome Tim, thanks!


On Tue, Mar 26, 2013 at 3:53 PM, Tim Watson <[hidden email]> wrote:
I have set up an experimental repository at https://github.com/hyperthunk/rabbitmq-lager that supports logging to lager via its built in error_logger redirection. You enable via `rabbitmq-plugins enable lager` and configure it in the RabbitMQ config file just like any other plugin, under the key 'lager'. In the morning I'll chat with the guys about integrating this as experimental plugin for the time being, so people can play around and see if it offers the required functionality.

Cheers,
Tim

On 23 Mar 2013, at 23:06, Ben Hood wrote:

> Hey Tim,
> On Friday, 22 March 2013 at 07:55, Tim Watson wrote:
>
>> Yeah with the current implementation you'd need to install your own log handler to make that work. In terms of changing the output format though, allowe to play devil's advocate for a moment...
>>
>> The info/progress/error reports have a lot of data in them, process stats, stack traces, etc. are you wanting to see all that plonked on one single long line? Because I can see how that'd make parsing/aggregating easier, but not reading them. Does the lager output fit exactly with what you want (I've not looks at it or a couple of years)?
> I don't think that everything needs to fit on one line. Longer traces which provide good context should probably remain as multiline entries. However, info items, for example,
>
> Mar 22 22:33:01 a rabbit3:  =INFO REPORT==== 22-Mar-2013::22:33:01 ===
> Mar 22 22:33:01 a rabbit3:  Setting permissions for 'guest' in '/' to '.*', '.*', '.*'
>
> need not get split up over two lines.
>
> With regard to longer traces, it would be nice if they could be handled in such a way that the generic portion of the call stack were trimmed down to something nearer the actual call site, rather that the entire generic call trace. As inspiration, logback mitigates the default "root cause comes last" behavior in Java with the reverse depth layout modifier (rEx{depth}). Lot's of hand waving here though.
>
> Finally, I do have to concur with Gavin on people's perception about Rabbit due to logging - I've have many conversations with people over the years who ask why Rabbit has such bizare logging - I've generally answered that this is an Erlang-ism, and though it is strange, the server kind of serves its primary function well (i.e. delivering messages), so just put up with it.
>
> Cheers,
>
> Ben
> _______________________________________________
> rabbitmq-discuss mailing list
> [hidden email]
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss

_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss



--

Gavin M. Roy
Chief Technology Officer


100 Union Square Drive
New Hope, PA 18938
p. +1.215.862.1162 x263
f. +1.215.862.0465

      

The public market leader in social discovery. (NYSE MKT: MEET) 


_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: lager vs sasl error log

Ben Hood
In reply to this post by Tim Watson-6
On Tuesday, 26 March 2013 at 15:53, Tim Watson wrote:
I have set up an experimental repository at https://github.com/hyperthunk/rabbitmq-lager that supports logging to lager via its built in error_logger redirection. You enable via `rabbitmq-plugins enable lager` and configure it in the RabbitMQ config file just like any other plugin, under the key 'lager'. In the morning I'll chat with the guys about integrating this as experimental plugin for the time being, so people can play around and see if it offers the required functionality.
Very cool - nice one Tim!

_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: lager vs sasl error log

Tim Watson-6
If anyone wants to try this plugin out, there are instructions in the README file on github now. You'll need at least R14B03 to use lager afaict. Please shout if you run into any difficulties.

Cheers,
Tim

On 27 Mar 2013, at 00:19, Ben Hood wrote:

> On Tuesday, 26 March 2013 at 15:53, Tim Watson wrote:
>> I have set up an experimental repository at https://github.com/hyperthunk/rabbitmq-lager that supports logging to lager via its built in error_logger redirection. You enable via `rabbitmq-plugins enable lager` and configure it in the RabbitMQ config file just like any other plugin, under the key 'lager'. In the morning I'll chat with the guys about integrating this as experimental plugin for the time being, so people can play around and see if it offers the required functionality.
> Very cool - nice one Tim!
> _______________________________________________
> rabbitmq-discuss mailing list
> [hidden email]
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss

_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Feedback On Logging Frameworks Please! (was lager vs sasl error log)

Tim Watson-6
Hello all!

I recently notified the mailing list that I've created a plugin providing a way to integrate the lager logging framework into RabbitMQ as a plugin: https://github.com/hyperthunk/rabbitmq-lager.

If you're interested in seeing improvements to rabbit's logging then please give this a try and give us some feedback. In particular, we'd like to know whether the approach I've taken (i.e., using lager's error_logger redirection) provides *enough* functionality for our users and/or what additional features you'd like to see. Getting decent feedback is a prerequisite to getting these changes integrated into rabbit in a permanent and supported manner, so please don't be shy!

If you're cc'ed on this email then you've piped up about logging recently/before on this list, and therefore I've kindly singled you out as a potential victim, erm... wait no... volunteer! That's right! ;)

So, feedback on a postcard please folks! We'd like to get this 'just right' so everyone has *nice* things to say about our gorgeous log files and configurable backends and whatnot.

Cheers,
Tim
_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: lager vs sasl error log / badarg?

Gotthard, Petr
In reply to this post by Tim Watson-6
Hi,
We used a custom lager plugin before without any troubles, but after switching to your plugin I'm getting 'badarg' when rabbit prints the name of the plugin. I disabled all plugins, except lager.

Stack trace:
   [{rabbit,'-print_plugin_info/1-lc$^0/1-0-',1,[]},
    {rabbit,'-print_plugin_info/1-fun-0-',1,[]},
    {rabbit_misc,with_local_io,1,[]},
    {rabbit,'-boot/0-fun-1-',0,[]},
    {rabbit,start_it,1,[]},
    {init,start_it,1,[]},
    {init,start_em,1,[]}]


Does it ring a bell to you?

Could it be caused by the unusual naming? The plugin is named "lager", while the directory is "rabbitmq_lager" and the .ez is called "rabbitmq_lager-2.0.0-rmq0.0.1-git9719370".


Petr

-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Tim Watson
Sent: 27. března 2013 15:04
To: Discussions about RabbitMQ
Subject: Re: [rabbitmq-discuss] lager vs sasl error log

If anyone wants to try this plugin out, there are instructions in the README file on github now. You'll need at least R14B03 to use lager afaict. Please shout if you run into any difficulties.

Cheers,
Tim

On 27 Mar 2013, at 00:19, Ben Hood wrote:

> On Tuesday, 26 March 2013 at 15:53, Tim Watson wrote:
>> I have set up an experimental repository at https://github.com/hyperthunk/rabbitmq-lager that supports logging to lager via its built in error_logger redirection. You enable via `rabbitmq-plugins enable lager` and configure it in the RabbitMQ config file just like any other plugin, under the key 'lager'. In the morning I'll chat with the guys about integrating this as experimental plugin for the time being, so people can play around and see if it offers the required functionality.
> Very cool - nice one Tim!
> _______________________________________________
> rabbitmq-discuss mailing list
> [hidden email]
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss

_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: lager vs sasl error log / badarg?

Tim Watson-5
Hi Petr,

That's probably a bug and you're probably right about the cause. I'll take a look at fixing that somehow. Other than puking on printing the plugin info, does the logging out behave as you'd expect though?

Cheers
Tim

On 27 Mar 2013, at 19:23, "Gotthard, Petr" <[hidden email]> wrote:

> Hi,
> We used a custom lager plugin before without any troubles, but after switching to your plugin I'm getting 'badarg' when rabbit prints the name of the plugin. I disabled all plugins, except lager.
>
> Stack trace:
>   [{rabbit,'-print_plugin_info/1-lc$^0/1-0-',1,[]},
>    {rabbit,'-print_plugin_info/1-fun-0-',1,[]},
>    {rabbit_misc,with_local_io,1,[]},
>    {rabbit,'-boot/0-fun-1-',0,[]},
>    {rabbit,start_it,1,[]},
>    {init,start_it,1,[]},
>    {init,start_em,1,[]}]
>
>
> Does it ring a bell to you?
>
> Could it be caused by the unusual naming? The plugin is named "lager", while the directory is "rabbitmq_lager" and the .ez is called "rabbitmq_lager-2.0.0-rmq0.0.1-git9719370".
>
>
> Petr
>
> -----Original Message-----
> From: [hidden email] [mailto:[hidden email]] On Behalf Of Tim Watson
> Sent: 27. března 2013 15:04
> To: Discussions about RabbitMQ
> Subject: Re: [rabbitmq-discuss] lager vs sasl error log
>
> If anyone wants to try this plugin out, there are instructions in the README file on github now. You'll need at least R14B03 to use lager afaict. Please shout if you run into any difficulties.
>
> Cheers,
> Tim
>
> On 27 Mar 2013, at 00:19, Ben Hood wrote:
>
>> On Tuesday, 26 March 2013 at 15:53, Tim Watson wrote:
>>> I have set up an experimental repository at https://github.com/hyperthunk/rabbitmq-lager that supports logging to lager via its built in error_logger redirection. You enable via `rabbitmq-plugins enable lager` and configure it in the RabbitMQ config file just like any other plugin, under the key 'lager'. In the morning I'll chat with the guys about integrating this as experimental plugin for the time being, so people can play around and see if it offers the required functionality.
>> Very cool - nice one Tim!
>> _______________________________________________
>> rabbitmq-discuss mailing list
>> [hidden email]
>> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
> _______________________________________________
> rabbitmq-discuss mailing list
> [hidden email]
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
> _______________________________________________
> rabbitmq-discuss mailing list
> [hidden email]
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: lager vs sasl error log

Ben Hood
In reply to this post by Tim Watson-6
Hey Tim,

I'm just kicking the tyres of your lager plugin with my rabbit umbrella.

Admittedly, I'm not an umbrella makefile jedi, so I just cloned your repo into a subdir of my umbrella (maybe this was the mistake). Running make results in the following issue:

[elided] generate deps
[elided] fix test deps
sed -e 's|build/deps.mk|$(DEPS_FILE)|' build/deps.mk > build/deps.mk.tmp && mv build/deps.mk.tmp build/deps.mk
rm -rf ./rabbitmq-lager-git
git clone http://github.com/basho/lager.git ./rabbitmq-lager-git
Cloning into './rabbitmq-lager-git'...
remote: Counting objects: 1822, done.
remote: Compressing objects: 100% (576/576), done.
remote: Total 1822 (delta 1330), reused 1709 (delta 1224)
Receiving objects: 100% (1822/1822), 713.54 KiB | 137 KiB/s, done.
Resolving deltas: 100% (1330/1330), done.
# Work around weird github breakage (bug 25264)
cd ./rabbitmq-lager-git && git pull
Already up-to-date.
cd ./rabbitmq-lager-git && git checkout 9719370eeaf8b64c1c3e79fb7052cb1ca29f16c8
Note: checking out '9719370eeaf8b64c1c3e79fb7052cb1ca29f16c8'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b new_branch_name

HEAD is now at 9719370... Merge pull request #120 from basho/adt-msg-has-datetime
touch rabbitmq-lager-git/.done
echo UPSTREAM_SHORT_HASH:=`git --git-dir=./rabbitmq-lager-git/.git log -n 1 HEAD | grep commit | cut -b 8-14` >hash.mk
sed -n -e 's|^.*{vsn, *"\([^"]*\)".*$|ORIGINAL_VERSION:=\1|p' <rabbitmq-lager-git/src/lager.app.src >build/version.mk
[elided] generate deps
/bin/sh: escript: command not found
rm -rf build/dep-ezs
mkdir -p build/dep-ezs
[elided] copy dependent ezs
touch build/dep-ezs/.done
rm -rf build/dep-apps
mkdir -p build/dep-apps
[elided] unzip ezs
touch build/dep-apps/.done
make: *** No rule to make target `build/deps.mk', needed by `ebin/error_logger_lager_h.beam'.  Stop.

Should I maybe have created a my own subdir that points to your repo by setting the UPSTREAM_GIT variable?

Cheers,

Ben



On Wed, Mar 27, 2013 at 10:03 AM, Tim Watson <[hidden email]> wrote:
If anyone wants to try this plugin out, there are instructions in the README file on github now. You'll need at least R14B03 to use lager afaict. Please shout if you run into any difficulties.

Cheers,
Tim

On 27 Mar 2013, at 00:19, Ben Hood wrote:

> On Tuesday, 26 March 2013 at 15:53, Tim Watson wrote:
>> I have set up an experimental repository at https://github.com/hyperthunk/rabbitmq-lager that supports logging to lager via its built in error_logger redirection. You enable via `rabbitmq-plugins enable lager` and configure it in the RabbitMQ config file just like any other plugin, under the key 'lager'. In the morning I'll chat with the guys about integrating this as experimental plugin for the time being, so people can play around and see if it offers the required functionality.
> Very cool - nice one Tim!
> _______________________________________________
> rabbitmq-discuss mailing list
> [hidden email]
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss

_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss


_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: lager vs sasl error log

Ben Hood
Hey Tim,

On Wed, Mar 27, 2013 at 10:03 PM, Ben Hood <[hidden email]> wrote:

I'm just kicking the tyres of your lager plugin with my rabbit umbrella.

Just to get things going, I tried using the published ez (md5: 9066ed797ca75519a68820271c79994a) but I ran into the following issue:

$ ./sbin/rabbitmq-plugins enable lager
Warning: Problem reading some plugins: [{"./sbin/../plugins/rabbitmq_lager-2.0.0.ez",
                                         {invalid_ez,bad_eocd}}]
Error: The following plugins could not be found:
  lager

Did I maybe grab the wrong thing?

Cheers,

Ben
 

_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: lager vs sasl error log

Gotthard, Petr

Ben,

Did you rename the .ez file? Because mine has an usual name that probably causes rabbit to crash (see my other post), but enabling works fine for me. Could you try using the .ez file with exactly the same name as created by the makefile?

 

Petr

 

From: [hidden email] [mailto:[hidden email]] On Behalf Of Ben Hood
Sent: 28. března 2013 3:21
To: Discussions about RabbitMQ
Subject: Re: [rabbitmq-discuss] lager vs sasl error log

 

Hey Tim,

 

On Wed, Mar 27, 2013 at 10:03 PM, Ben Hood <[hidden email]> wrote:

 

I'm just kicking the tyres of your lager plugin with my rabbit umbrella.

 

Just to get things going, I tried using the published ez (md5: 9066ed797ca75519a68820271c79994a) but I ran into the following issue:

 

$ ./sbin/rabbitmq-plugins enable lager

Warning: Problem reading some plugins: [{"./sbin/../plugins/rabbitmq_lager-2.0.0.ez",

                                         {invalid_ez,bad_eocd}}]

Error: The following plugins could not be found:

  lager

 

Did I maybe grab the wrong thing?

 

Cheers,

 

Ben

 


_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
12