Entire log is JSON - how can I parse it?

Follow

Tom Ruggles

• February 05, 2015 23:06

We're using Bunyan for logging.  As such our logs are entirely JSON.  Here is an example of a single line:

{"name":"worker","hostname":"phjkhl","pid":13437,"message_id":"6fd07356-10ac-4666-8a75-381f4692304f","message_type":"pending","worker_id":"1","level":20,"msg":"Build successful for 6fd07356-10ac-4666-8a75-381f4692304f","time":"2015-02-05T22:55:50.454Z","v":0}

 

Sumologic shows this entire JSON  as the "message" value.  I have found the JSON operator but haven't gotten it to work.  The examples seem to be for logs that contain JSON but aren't entirely JSON.  Any suggestions for searches that would do each of these two things?

• find messages where message_type == "pending"

• parse the entire line to split based on the JSON fields

Thanks.

2

• 

  Christian

  • February 06, 2015 01:03

  try this (and i agree, this could be easier/more obvious):

  | json field=_raw "name", "hostname", "pid", "message_id","message_type", "worker_id", "level", "msg", "time", "v"

  2

  Comment actions Permalink
• 

  Tom Ruggles

  • February 06, 2015 22:08

  Thank you Christian.  I finally got something working last night that was similar but yours is nicer.  I had 

  | parse "*" as parsed | json field=parsed "name","hostname","level","message_id","message_type","msg" nodrop | fields -parsed 

  And it looks like the where part works like this:

  | where message_type = "pending"

  0

  Comment actions Permalink
• 

  David Han

  • May 22, 2015 19:07

  I have a similar follow up question. If I also have a json but a nested json object that looks like -

  
  
  
  
  {"csp-report":{"document-uri":"value","referrer":"value","violated-directive":"value","effective-directive":"value","original-policy":"value","blocked-uri":"www.blocked.com","status-code":200}}
  
  
  
  
  

  would this query work?

  csp-report | json field=_raw "csp-report.document-uri", "csp-report.referrer", "csp-report.violated-directive"

  0

  Comment actions Permalink
• 

  David Han

  • May 29, 2015 19:01

  Hi, are there any updates on this?

   

  I checked the JSON Operator docs, and I'm not sure how to parse this json.. Any help would be much appreciated.

  0

  Comment actions Permalink
• 

  David Han

  • May 29, 2015 19:50

  I figured it out.

  Here is the query that I used.

  *
  
  | parse "{\"csp-report\":*" as jsonobject
  
  | json field=jsonobject "blocked-uri", "document-uri", "referrer", "violated-directive", "effective-directive", "status-code", "source-file" nodrop
  
  | fields -jsonobject

  1

  Comment actions Permalink

Please sign in to leave a comment.