Post
Fun with Grok and Logstash regexes
I have been using Logstash extensively lately. Along with ElasticSearch, it's a great tool to centralize the logs and simplify access to them.
I have been using Logstash extensively lately. Along with ElasticSearch, it's a great tool to centralize the logs and simplify access to them. The only difficulty I had was related to supporting multiline log messages, such as those printed by Java stacktraces. I have found some good examples online, but none seemed to work the way I wanted. In some cases, I also got my messages tagged as * _grokparsefailure*, which indicated that the parser failed to process the regex. I ended up with one that it's not so different after all but which did match exactly the way we log messages with log4j:
(^.+Exception.+)|(^\s+at .+)|(^\s+... \d+ more)|(^\s*Caused by:.+)
It's also worth mentioning the Grok Debugger website along with an adequate regex tutorial are two priceless resources to have at hand.