Merge and limit number of raw message

Comments

3 comments

  • Avatar
    Caleb Fogleman
    Hey Michael, I believe I've found a solution to this, as follows: _sourceCategory=travelweb| parse regex "(?[0-9]+\.[0-9]+\.[0-9]+\.[0-9]) - "| transactionize ip maxlogs=5 (merge ip takeFirst, _raw join with "\n\n\n")| transactionize ip (merge ip takeFirst, _raw takeFirst) The first transactionize statement puts the logs into groups of 5 messages by IP address, merging 5 _raw messages per log entry.The second transactionize statement then transactionizes those groups of 5, keeping only the first one (effectively removing any groups that are not the most recent 5 messages for a given IP). I hope this helps! Caleb F.
    0
    Comment actions Permalink
  • Avatar
    Michael Sabin
    I tried that out, but the problem I am seeing is that I am getting 5 random messages instead of the most recent 5
    0
    Comment actions Permalink
  • Avatar
    Caleb Fogleman
    Hmm. On the data I am testing, the log entries are well-ordered, and so the above query does return the 5 most recent messages for an IP address. Maybe try with the 'sort' operator on the timestamp that is most accurate for your logs (either _messagetime, _receipttime, or a parsed timestamp extracted from your logs). For example: _sourceCategory=travelweb| parse regex "(?[0-9]+\.[0-9]+\.[0-9]+\.[0-9]) - "| sort by _messagetime| transactionize ip maxlogs=5 (merge ip takeFirst, _raw join with "\n\n\n")| transactionize ip (merge ip takeFirst, _raw takeFirst) This sort is just a way of explicitly writing how Sumo implicitly sorts your logs to begin with, but if you have an alternate timestamp that is more accurate, this method may help return only the 5 most recent results. Thanks!
    0
    Comment actions Permalink

Please sign in to leave a comment.