Kentaro Kuribayashi's blog

Software Engineering, Management, Books, and Daily Journal.

My pull request to #fluentd to support LTSV-formatted values was merged

Labeled Tab Separated Value (LTSV) has been such a hot word among Web developers these days. LTSV is yet another log format that Hatena, Inc. has been used for years to record their own httpds' access log. id:stanaka (CTO of Hatena, Inc.) explains its merits on his blog post:

As soon as I read the blog post, I proposed we switch the log format which we had used (Apache's combined or so) to LTSV-formatted one. Thank you ex-colleagues at Hatena, Inc. I was convinced it would become so popular.

Then, I sent a pull request to fluentd to support LTSV-formatted values by its in_tail plugin.

It was merged today and will be shipped as td-agent including the feature. I'm really looking forward to it because I wanted it indeed.

In one of our use-cases, in_tail plugin takes a large responsibility because we have to write everything into files, not directly sending to fluentd in the middle. I'll talk the reason why so at Fluentd Casual Talks #2, incidentally, in such a case, fluentd's LTSV support can work effectively.