Use logging date rather than receipt date



  • Avatar
    Peter Scalise

    Figured it out...for some background, I'm collecting historic logs to find queries performed by various applications which all connect to the same API -- our team needed to see if any particular application was overloading the system.

    Log messages look like this:

    2019-03-06 17:49:47,822  datetimeofquery="2016-05-24 19:44:31+0000" queryperformed="<some query string>" programquerying="<some application name>"


    Here is the search with some [annotations]:

    *         [regular searching]
    | keyvalue "datetimeofquery" as datetimeofquery      [extract the date when query was run]
    | keyvalue "queryperformed" as queryperformed       [extract query]

    [read docs carefully, converting a date-string to a date requires a two stage process...]

    [first, the field datetimeofquery must be converted to milliseconds using parseDate()] 
    | parseDate(datetimeofquery, "yyyy-MM-dd HH:mm:ssZZZZ") as p

    [next, convert from milliseconds to an actual date using formatDate().  The date format dictates the width of buckets for the stacked bar chart created at the end.  I used a one hour bucket size, converting to a one minute bucket size would entail using the forma "yyyy-MM-dd HH:mm"]

    | formatDate(p, "yyyy-MM-dd HH") as formattedDate

    | count by formattedDate, queryperformed
    | transpose row formattedDate column queryperformed




    Comment actions Permalink
  • Avatar
    Benj Kamm

    This solution is great but is missing one of the great features of timeslice -- filling in empty buckets for empty spots in the timeslice.  Is there a way to do that with the formatDate approach?

    Comment actions Permalink

Please sign in to leave a comment.