Histogram of the elements in a list in a JSON log

Comments

5 comments

  • Avatar
    Mark Drummond

    Guessing I need some regex magic to make this work. What I have below is counting each unique set of triggered rules, but not counting the individual rules within each set, which is what I am shooting for.

    .

    .

    .

     

    Uh, for some reason I cannot copy & paste into this dialog box. MacOS 10.14.5, Chrome 75. When I paste, all I get are a bunch of blank lines. Tried copying into a text document first, and then copying here, but same results. :(

    0
    Comment actions Permalink
  • Avatar
    Mario Sanchez

    Mark,

    Fortunately there is an App (out-of-the-box content) that is already doing this. Check out this link for info on the App and how to install it:

    https://help.sumologic.com/07Sumo-Logic-Apps/18SAAS_and_Cloud_Apps/Cloudflare/Install_the_Cloudflare_App_and_view_the_Dashboards

    In particular, look for the "Cloudflare - Security (WAF)" Dashboard which gives you info on WAF rules triggered. Keep in mind that you can always edit Panels in that dashboard (go to the query behind each Panel) to tweak it for your specific needs.

    Cheers,

    Mario

    0
    Comment actions Permalink
  • Avatar
    Mark Drummond

    Hi Mario. I am using the Cloudflare app, but the WAF Rules Triggered query is just pulling the WAFRuleID field, which is not what I am looking for. The most common WAFRuleID is "981176", which is an aggregate rule triggered as a result of a number of other rules being triggered by the source.

    The actual rules behind a 981176 event are listed in the FirewallMatchesRuleIDs field. That field is a JSON list object. What I would like to do is get a histogram of all the rule IDs in the list object across all the logs in a given time period.

    {
    .
    .
    .
    "ClientRequestUserAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.100 Safari/537.36",
    .
    .
    .
    "EdgeResponseStatus":403,
    .
    .
    .
    "FirewallMatchesRuleIDs":["07","01","900001","900002","900004","900006","900012","900018","900021","960024","981133","950901","981311","959073","981257","981248","981245","981247B","981243","981136","973300","973335","973334","973333","973332","2000001","2000003","2000004","2000006","981176"],
    .
    .
    .
    "WAFRuleID":"981176",
    .
    .
    .
    }
    0
    Comment actions Permalink
  • Avatar
    Mario Sanchez

    Got it. 

    OK, here's an idea. Since we do not know the number of rules you will have in each message, we can use the parse multi option to have Sumo find a given pattern as many times as it exists in the message.

    By the way, my Regex skills are not the best, so you might be able to optimize this, but here's a thought:

    _sourceCategory=<your_cloudflare_logs>
    // Let's make a list of all the rules (call it all_my_rules)
    | parse "\"FirewallMatchesRuleIDs\":*"as all_my_rules
    // Use multi to parse out all rules that are between 1 and 9 digits)
    | parse regex field=all_my_rules "(?<rule>\d{1,9})" multi
    //This previous command creates a new message/line for each rule
    //A count by rule with the histogram can identify trends
    | count by rule

     

    Let me know how this works.

    1
    Comment actions Permalink
  • Avatar
    Mark Drummond

    That put me on the right path. Here is what I ended up with (apologies for the link ... cut and paste is not working for me and I'm not going to type it out here): https://pastebin.com/DKkJQZAv

    I was going to post a link to screen recording of me trying to cut and paste the log query code here, but of course cut and paste is not working for me, so I included it in the pastebin link.

    0
    Comment actions Permalink

Please sign in to leave a comment.