Mask rules for multiline log messages
Hello,
I'm trying to write a mask rule which will redact a multiline log message. My use case is we have two sets of developers, one who has full access to logs and can see sensitive data and another group which is not allowed to see sensitive data. The idea is to create a separate partition for the group which cannot see sensitive data and duplicate the log sources, but in the duplicate sources which flow to the new partition apply mask rules which redact all sensitive data.
I've encountered an issue with a log message which logs out a SOAP XML envelope which I would like to redact. Here is an example message:
2021-02-25 14:06:25,810 [http-bio-8080-exec-70] INFO com.example.SoapWriter.27 - [7fbe6d227aa053b9 7fbe6d227aa053b9] - - <?xml version="1.0" encoding="UTF-8"?><soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><soapenv:Body><processSync xmlns="urn:soap"><in0><?xml version="1.0" encoding="UTF-8"?><Order xmlns="http://www.example.com/" xmlns:icp="http://www.atis.org/obf/wir/wicis5/schemas"> <OrderDetails> <RequestDetails> <CustomerName>example</CustomerName> <SupplierName>example</SupplierName> <OrderType>example</OrderType> <OrderSubtype>example</OrderSubtype> ...
... example shortened for brevity
... </in0></processSync></soapenv:Body></soapenv:Envelope>
Here are the mask rules I attempted:
{
"filterType": "Mask",
"name": "SoapWriter",
"regexp": "SoapWriter\\.\\d+ (?:- )?\\[[0-9a-z]* [0-9a-z]*\\] - (?:[0-9a-f]{8}-[0-9a-f]{4}-[1-5][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12})? - (.*)",
"mask": "REDACTED"
}
This rule only redacts the first line of the log message.
Example redacted message:
2021-02-25 14:06:25,810 [http-bio-8080-exec-70] INFO com.example.SoapWriter.27 - [7fbe6d227aa053b9 7fbe6d227aa053b9] - - REDACTED
<OrderDetails>
<RequestDetails>
...
... entire message contents follow
I also tried this rule to match more, but this mask does not seem to work at all:
{
"filterType": "Mask",
"name": "SoapWriter 2",
"regexp": "SoapWriter\\.\\d+ (?:- )?\\[[0-9a-z]* [0-9a-z]*\\] - (?:[0-9a-f]{8}-[0-9a-f]{4}-[1-5][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12})? - ((?:.|\\n|\\r)*)",
"mask": "REDACTED"
}
Here is an example of what I'm trying to accomplish: https://regex101.com/r/2zKNqK/1
Any tips for applying a mask rule to a multiline selection?
-
Hi Nathan,
Please try below. I tested this for the sample message you have posted.
"filters":[{ "filterType":"Mask", "name":"Maskrule", "regexp":"SoapWriter\\.\\d+ - \\[[0-9a-z]* [0-9a-z]*\\] - ([0-9a-f]{8}-[0-9a-f]{4}-[1-5][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12})?\\s+-\\s+((?s).*)", "mask":"Redact" }],
Regards
-
Thank you Shobhit Garg for your response. This solution worked perfectly for me.
Please sign in to leave a comment.
Comments
2 comments