Counting JSON fields inside array of logs

Comments

5 comments

  • Avatar
    Piotr Woch

    Hi Mark! Did you try parsing your data not as JSON, but as plain text and just parse out like this:

    parse regex (<event_type>event: \".*\") multi

    Please let me know if that helped get you the results you were expecting. 

  • Avatar
    Mark Drummond

    This seems to have put me on the right path. I am able to extract just the event type text with the following:

    _sourceCategory=sendgrid/notify
    | parse regex "\"event\":\"(?<event_type>.*?)\"" multi

    But multi is not working for me. It only grabs the event field from one log where there are "multiple logs in a log", and, kinda odd, which one it grabs seems to be random. Sample log message below. Using this as an example, in some cases the above regex grabs "processed" and in some cases "delivered".

    [
        {
            "email": "a@foo.com",
            "timestamp": 1523709521,
            "smtp-id": "<A@B.sendgrid.net>",
            "sg_event_id": "STRING",
            "sg_message_id": "STRING",
            "event": "processed"
        },
        {
            "ip": "1.2.3.4",
            "response": "250 2.6.0 <A@B.sendgrid.net> [InternalId=INT, Hostname=foo.com] 14998 bytes in 0.180, 81.337 KB/sec Queued mail for delivery",
            "sg_event_id": "STRING",
            "sg_message_id": "STRING",
            "tls": 1,
            "event": "delivered",
            "email": "b@foo.com",
            "timestamp": 1523709523,
            "smtp-id": "<A@B.sendgrid.net>"
        }
    ]

     

  • Avatar
    Mark Drummond

    Small change:

    _sourceCategory=sendgrid/notify
    | parse regex "\"event\":\"(?<event_type>\w*)\"" multi

    Using '\w*' allows me to drop the trailing '?' and the regex still works as before, but multi still not working for me. First two results below have single sendgrid log messages, the second two have two each, but only one 'event_type' value is being pulled:

  • Avatar
    Mark Drummond

    Aside: I've been testing my regex's on regex101.com and they are working as expected there.

  • Avatar
    Latimer Luis

    Hi Mark - 

    I tested this myself on Regex101 and I had to explicitly define the whitespace (\s) after the colon following "event".   Perhaps what you're seeing is actually something that was extracted in a field extraction rule and not by this parse statement? 

    | parse regex "\"event\":\s\"(?<event_type>\w*)\"" multi

     

Please sign in to leave a comment.