a companion discussion area for blog.codinghorror.com

The Problem With Logging


#121

If it’s worth saving to a logfile, it’s worth showing in the user interface.

NO! Didn’t you read the books you yourself recommend? The Inmates are Running the Asylum, Design of Everyday Things…


#122

I agree with Gilbert.
In my (SHORT, of course) experience, I see that it’s better to have more logs than less logs.
The web-application runs - there’s an exception - if I don’t log the values of variables on begin and on end of methods I spent more time on debug for studyng if it’s a problem of bad coding or uncorrect behaviour of customer or other problems.
I think that the safe time on debug gives back the time on coding the log.debug/info/fatal/etcetera in the methods.


#123

Jesus freaking Christ, if I hear someone talk about logging ONE MORE TIME, I’m going to puke. It’s vi vs. EMACS, tabs vs spaces, KR vs Allman. There can be NO outcome to this discussion, no matter how much you argue.

Don’t you guys ever code anything complicated? I mean complicated as in can’t figure this out by just staring at it or holy shit this stuff is insane and I have to read whitepapers for a week just to understand what it does. That’s a rhetorical question, BTW, because if you did logging would be on the very bottom of your priority list.


#124

I work in the tax and accounting industry and, in our production environment, we have near zero visibility into what our apps are doing (beyond what a customer sees). We have some high level monitoring tools, but if not for detailed application-level logging, we’d have zero visibility.

I can’t remote desktop or otherwise log into the boxes, either. We have a proprietary viewer we use (which is only accessible through a single machine, I might add) to view the log messages even. having the log entries is a life and time saver. Given the constraints of our environment/industry, I can’t imagine NOT having mucho logging. More the better, I say.

In fact, it really bothers me when I take on someone else’s code and it doesn’t have logging. The first thing it gets is, of course, logging.


#125

Logging costs performance!

Sounds silly? Untill it really happens that your log database has gotten so incredibly huge that adding records to the log-db takes milliseconds… The DBA forgot to move+empty the logdb for a while :slight_smile:


#126

I would more prefer use higher log level is most case / use other filter to filter out logs, because, if you don’t log it, you lose the detail.

Even if the application build work is great that you can release to add logging very quick, it still take time to release. So all to have more logging by config is a good idea in most cases.


#127

There’s no right answer to logging - it depends entirely on the context your software is operating in.

The software we build for clients logs information that’s appropriate to the environment and context - sometimes that’s nothing more than a Started and Stopped event message. Other times it’s doing full logging of all messages from third party systems (eg: payment gateways).

What’s excessive for one environment might be nowhere near enough information for another.

What might be worth noting, however, is that you can still build in very high level bits of logging - but wrap compiler instructions around it.

eg:
#if DEBUG
Logging.Log(…whatever);
#endif

…if there’s some weird issues we can deploy the debug DLLs temporarily and see if we can find out more info. Once that’s fixed, we re-deploy Release Mode DLLs and we’re back to minimal logging. No unnecessary performance hits.


#128

The funniest thing about this post, imo, is that you will see the opposite advice quite frequently on highscalability.com. Some of the biggest sites out there log incredible amounts of data for debugging purposes.

They don’t do sill things like sending the logs directly to their main database on the same connections/txns that work their main data, though. :slight_smile:


#129

if you mentioned the log messages were (apparently) going to the database

The log messages were not going to the database. An action was taken which resulted in a) logging of the action and b) writing actual real data to the database.

and the object’s method for obtaining a string representation is doing complex stuff - maybe causing object state to be lazy-loaded from the database, that kind of thing.

Bingo.


#130

Don’t you guys ever code anything complicated? I mean complicated as in can’t figure this out by just staring at it or holy shit this stuff is insane and I have to read whitepapers for a week just to understand what it does. That’s a rhetorical question, BTW, because if you did logging would be on the very bottom of your priority list.

I agree. Most difficult scenarios that need logging lie in the application’s problem domain, that’s where the real fun is. Like discovering that the cherished and patented algorithm to do XYZ simply does not work in the real environment. After that the petty issues with logging become like the warm childhood memories.

But software engineers need to prove their right to exist, thus these discussions always pop up.


#131

Thank you for correcting me. I couldn’t see where the shared state was which would lead to a deadlock (short explanation: you only get deadlocks when there is some kind of locking – although it may not be operating system-level mutex locking ™ – and you don’t need locks unless there is some sort of shared state).


#132

and the object’s method for obtaining a string representation is doing complex stuff - maybe causing object state to be lazy-loaded from the database, that kind of thing.

Bingo.

If a read is causing a lock, this blog post should be titled The Problem With Database Software.

Or maybe The Problem With ToString() Referencing Lazy Loaded Data.

It would be an interesting excerise to try the same code on a database backend where readers don’t block writers. (Postgres/Oracle).


#133

Jeff discusses some interesting points, but clearly from the mind set of a developer. Supporting applications where transactions span multiple servers is not a trivial effort and root cause analysis of problems is time consuming and difficult at the best of times. I would like to address several of his points in detail.

  1. So what? Better than not having the information necessary to perform root cause analysis of a problem.
  2. Right. It’s always a trade off between application speed and ignorance of what the application is doing.
  3. Doesn’t make any sense. There is information necessary to audit a transaction (e.g. which server processed a transaction) which the user has no need to know.
  4. True. But I’d much rather use something like Microsoft log parser and mine for data than lamenting over data which is not available.
  5. Sounds like bad design. Be sure to involve operational support so logs designs reflect real world experiences. Logs are great during the development process, but their real work begins supporting the application.

#134

I agree fully, MOSS is a prime culprit for logging too much trash, making it hard to get at the errors when they occur, and MOSS also logs the crucial errors, and displays pretty errors to the screen.


#135

Good logging messages act as documentation

$self-log-info(Doing daily expire run on outdated foos);
for my $foo ($self-get_expired_foos){
$foo-set_expired;
}


#136

I think for web-based applications, it’s always a good idea to log how long it takes to service each request and also how long every database query takes. The data gives you a good picture of the frequency of each request/query and which request/query is the worst performing and may need tuning/caching.


#137

You can log all your method calls etc using interceptor (castle) so that your code isn’t filled with logger code.


#138

Now, that makes sense Jeff, but in this case, logging still isn’t at fault. The action of logging is creating side effects in the code, which it shouldn’t be doing.

(Hindsight, 20/20, et cetera.)


#139

Use ETW on Windows, DTrace on Unix. Just because you were doing it wrong doesn’t mean it’s a bad idea. The guy you’re quoting in the beginning has a pretty good strategy.


#140

Huh? You can’t set the log level on startup? How about ERROR for production and DEBUG or INFO for development/test? Isn’t that why there are different levels?