Parsing Non-Structured Fields

Follow

Mario Sanchez

• May 27, 2017 00:33
• Edited

Sometimes you might need to parse fields that are not well structured within the message. In this example we are counting hits by browser. Since there is no browser field in the message, we simply search for the browser name and store it in its own field for future aggregation.

_sourceCategory=Apache/Access 
| if (agent matches "*MSIE*",1,0) as ie
| if (agent matches "*Firefox*",1,0) as firefox 
| if (agent matches "*Safari*",1,0) as safari 
| if (agent matches "*Chrome*",1,0) as chrome 
| sum(ie) as ie, sum(firefox) as firefox, sum(safari) as safari, sum(chrome) as chrome

0

• 

  Arjit Goswami

  • July 27, 2017 06:49
  • Edited

  Hi Mario, 

  Sorry ! Had some confusion over this query. If we already have "Agent" as a field, why do we need a "IF" query to get the count.  Cant we get desired result by "count' operator? 

   

  Thanks and regards,

  Arjit Goswami. 

  0

  Comment actions Permalink
• 

  Mario Sanchez

  • July 27, 2017 13:45

  You're exactly right Arjit. If the field already exists, a simple count will take care of it. For example, 

  _sourceCategory=Apache/Access
  | count by user_agent

  However, this would provide a large list that has all variations of browsers, operating systems, versions, etc. But if you want to get a count just by browser, then this query above helps you count occurrences for each browser, regardless of the other variables.

  Hope this helps.

  Mario

   

   

  0

  Comment actions Permalink
• 

  Dudu Sakharovich

  • August 07, 2017 12:11

  Hey Mario,

   

  When trying to run this query I receive an error that field "agent" is not found.
  I'm trying to run this on source category of IIS logs.

   

  Regards,

  Dudu

  0

  Comment actions Permalink

Please sign in to leave a comment.