Comment by Zopieux
3 days ago
journalctl is mentioned once in the landing page and it seems to imply that journalctl is not supported per se, as logs need to be stored plaintext to legacy syslog (?).
I do not want to store plaintext logs and use ancient workarounds like logrotate. journald itself has the built-in ability to receive logs from remote hosts (journald remote & gateway) and search them using --merge.
That's true, as of today nerdlog doesn't use journalctl, and needs plain log files. There were a few reasons of that, primarily related to the sheer amount of logs that we were dealing with.
As mentioned in the article, my original use case was: having a fleet of hosts, each printing pretty sizeable amount of logs, e.g. having more than 1-2GB log file on every host on a single day was pretty common. My biggest problem with journalctl is that, during some intensive spikes of logs, it might drop logs; we were definitely observing this behavior that some messages are clearly missing from the journalctl output, but when we check the plain log files, the messages are there. I don't remember details now, but I've read about some kind of ratelimiting / buffer overflow going on there (and somehow the part which writes to the files enjoys not having these limits, or at least having more permissive limits). So that's the primary one; I definitely didn't want to deal with missing logs. Somehow, old school technology like plain log files keeps being more reliable.
Second, at least back then, journalctl was noticeably slower than simply using tail+head hacks to "select" the requested time range.
Third, having a dependency like journalctl it's just harder to test than plain log files.
Lastly, I wanted to be able to use any log files, not necessarily controlled by journalctl.
I think adding support for journalctl should be possible, but I still do have doubts on whether it's worth it. You mention that you don't want to store plaintext logs and using logrotate, but is it painful to simply install rsyslog? I think it takes care of all this without us having to worry about it.
I appreciate this response, and want to say I really like your tool's UI over something like lazyjournal. But like the above commentor, I would love to see journald support as well, just because it's the default these days on the distros I use, and seems like the direction the Linux system industry has headed in.
Fyi I've done a simple benchmark today, and journalctl is indeed a lot slower than simply reading log files. In case you're interested: https://github.com/dimonomid/nerdlog/issues/7#issuecomment-2...
Thanks for the feedback. I'll see what I can do. But for now, do you think the workaround of having to install rsyslog is not good enough?
8 replies →
can't you just read from stdin?
i use lnav in this way all the time: journalctl -f -u service | lnav
this is the ethos of unix tooling
Not really, at least not yet, because nerdlog's focus is very different than that of lnav. There is a section about it in the article as well.
In fact nerdlog doesn't even support anything like -f (realtime following) yet. The idea to implement it did cross my mind, but I never really needed it in practice, so I figured I'd spend my time on something else. Might do it some day if the demand is popular, but still, nerdlog in general is not about just reading a continuous stream of logs; it's rather about being able to query arbitrary time periods from remote logs, and being very fast at that.
The article makes it sound like it uses various command-line tools (bash/awk/head/tail) to process the logs. So, I imagine it's not a huge leap to extend support to using journalctl to do that work instead.
One small hitch I found is that this kind of tools are fixes in what to process, so for example I can't use them for structured logging. If it has an escape hatch where I can supply my own pipe (for example `process = 'vector ....'`) then it will be enough.