Control where sumo logic splits data for large message log

I have app pool data streaming from an F5 load balancer that gets sent in the form of a large JSON object. The JSON object is larger than the maximum message size for Sumo Logic and gets split between multiple messages.

Is there a way to control where Sumo Logic splits these messages?

For example, I would like the message to be cut at the last occurrence of "}}}" it finds because this is the end of a natural object in my dataset and would be much easier to work with than splitting in the middle of an app pool name.

The start of the new object always looks like this:
"/SomeEnv/some_pool.name_various_chars_and_digits.company.com.http.8080":{"availabilityState":

The end of the object always looks like this:
}}},

I've tried to do multiline processing with boundary regex but I'm not sure how to do that when it's all just one big line. The regex that I've tried don't seem to do anything. I'm not sure how to match it with the multi-line requirement that the regex must match the entire first line. This is an issue because there aren't multiple lines. It's all just a one line json string of text.

Brandon Free

June 28, 2022 16:10

Updated with some more detail about the JSON object and issues I've found when trying to do multi-line processing.

June 28, 2022 19:40

I've found this regex will group the objects I'm intersted in, however it's not accepted as a boundary regex as it is "invalid or contains restricted patterns"

("\/([^"]*)":{"availabilityState":([^}{3}]).*?}{3})

June 28, 2022 21:16

Looks like it accepts that if I escape the brackets, but it still doesn't do anything:

("\/([^"]*)":\{"availabilityState":([^\}{3}]).*?\}{3})

June 30, 2022 15:44

Getting some indication that we may be able to use multi-line processing without the new lines if the regex matches up with everything in the message using | or conditions. Tried making a regex for that as follows but still doesn't seem to work:

\{"system":\{"systemTimestamp":".*?\}|,"pools":|\{"\/.{0,125}":\{"a.*?\}{3}|,"\/.{0,125}":\{"a.*?\}{3,5}

Control where sumo logic splits data for large message log

Comments

Didn't find what you were looking for?