Import entire log as a single message

Follow

Justin Montgomery

• February 01, 2018 19:21

I have some logs generated by the Rackspace Recap tool: https://github.com/rackerlabs/recap

They are basically snapshots of system state every 10 minutes and I don't believe there is much value in parsing them as single messages, the whole log is really one congruous piece of information.

Here is an example of the logs content: https://gist.github.com/fxer/f9d15ee9597cb30e02789969fb3a1598

So the question is, is there some setting or boundary regex I can choose which will parse the entire log as a single message?

1

• 

  Monty Yao

  • February 06, 2018 21:11

  Hi Justin,

  You can parse the output of every 10 minutes as ONE message. You will need to setup a multi-line detection regular expression to match the first line of the whole multi-lines message.

  In your example the following looks like a message separator: 

  2018-01-31_00:00:01
  You should set up something like below in the source's multi-line detection. ^\d{4}-\d\d-\d\d_\d\d:\d\d:\d\d$ So that every time when Sumo sees that pattern, that's the beginning of a new message.   HTH, Monty

  
   

  1

  Comment actions Permalink
• 

  Justin Montgomery

  • February 06, 2018 22:04

  Excellent thanks Monty, I added the following to my config JSON and it all appears to be working now!

   

  "useAutolineMatching":false,
  "manualPrefixRegexp":"^\\d{4}-\\d{2}-\\d{2}_\\d{2}:\\d{2}:\\d{2}$"

  1

  Comment actions Permalink

Please sign in to leave a comment.