Regex for multiline processing

I'm exploring a sumo logic and trying to regex for logs where I want to remove duplicate n number of records while counting.
I tried "dedup" and "count_distinct" but could not get an exact unique number while counting. Log example -
{
timestamp:1657351704433,
log:{
url_path:"/v1/test/user",
transaction_id:"ABCD-EFGH-IJKLM",
account_id:"JDFJDJF4595J4494594",
account_name:"testaccount",
},stream:"stdout",
time:"2000-07-09T07:28:24.433902928Z"
},
{
timestamp:1657351704433,
log:{
url_path:"/v1/test/user",
transaction_id:"ABCD-EFGH-IJKLM",
account_id:"JDFJDJF4595J4494594",
account_name:"testaccount",
},stream:"stdout",
time:"2000-07-09T07:28:24.433902928Z"
},
{
timestamp:1657351704433,
log:{
url_path:"/v1/test/user",
transaction_id:"DFJD-DFDD-DlDDD",
account_id:"55454884983JFJJ4JLKK989",
account_name:"acctest",
},stream:"stdout",
time:"2000-07-09T07:28:24.433902928Z"
}

Default count we see

No transaction_id account_name /v1/test/user
1 ABCD-EFGH-IJKLM testaccount 2
2 DFJD-DFDD-DlDDD acctest 1

Expected restuls with regex
No transaction_id account_name /v1/test/user
1 ABCD-EFGH-IJKLM testaccount 1
2 DFJD-DFDD-DlDDD acctest 1

Please help

Thanks

Eli Jang

July 14, 2022 11:31

Can you post your query here?

Pradeep Lokhande

July 15, 2022 11:47

Thanks for the reply!

I'm trying with this regex query -
| parse regex "(?<transaction_id>[A-Z0-9]+-[A-Z0-0]+-[A-Z0-9]+)"
Here I want to remove duplicate values.

July 19, 2022 04:10

use dedup operator. here

July 19, 2022 04:13

I already tried that though let me know if we do it through regex.

Brandon Free

July 19, 2022 21:23

Is this all one message? I think dedup or count distinct would require these to be separate, which may be able to be done using multi line processing on the source.

If not I think you'd have to separate these out to separate results somehow using something like parse regex multi or something.
https://help.sumologic.com/05Search/Search-Query-Language/01-Parse-Operators/02-Parse-Variable-Patterns-Using-Regex#parse-multi

Regex for multiline processing

Comments

Didn't find what you were looking for?