Collect logs from GCP
Hello,
I've diligently followed the instructions at https://help.sumologic.com/07Sumo-Logic-Apps/06Google/Google_Cloud_Storage/Collect_Logs_for_Google_Cloud_Storage and have been able to (allowing for the various GCP differences) export logs from GCP and have them arrived into Sumologic.
However, I want to automate this process and started looking at the Sumologic terraform provider.
I've got all of the basic plumbing working using this provider, the only I have is that in the sumologic UI, my source shows as a HTTP source, and all of the messages that duly arrive are pubsub (base64) formatted.
ie.
{"message":{"attributes":{"logging.googleapis.com/timestamp":"2020-02-18T10:19:38.199542Z"},"data":"base64_encoded_string","messageId":"995588409306531","message_id":"995588409306531","publishTime":"2020-02-18T10:19:38.639Z","publish_time":"2020-02-18T10:19:38.639Z"},"subscription":"projects/rl-dev-vpc/subscriptions/rl-fw-shared-firewall-sumologic-subscription"}
When I configure the source through the UI and correctly choose the 'Google Cloud Log' - then sumologic seems to magically unencode the pubsub messages and display the actual log message.
ie.
{ "message":{ "attributes":{ "logging.googleapis.com/timestamp":"2020-02-18T10:24:18.018532Z" }, "data":{ "insertId":"ABC123", "jsonPayload":{ "actor":{ "user":"-SNIP-@-SNIP-.iam.gserviceaccount.com" }, "event_subtype":"compute.firewalls.insert", "event_timestamp_us":"1582021458018532", "event_type":"GCE_OPERATION_DONE", "operation":{ "global":true, "id":"-SNIP-", "name":"operation-1582021452565-59ed712b50c45-377ec48d-1a7c84f2", "type":"operation" }, "resource":{ "global":true, "id":"1234567890", "name":"earl-allow-home-pmiles", "type":"firewall" }, "trace_id":"operation-1582021452565-59ed712b50c45-377ec48d-1a7c84f2", "version":"1.2" }, "labels":{ "compute.googleapis.com/resource_id":"-SNIP-", "compute.googleapis.com/resource_name":"some-message", "compute.googleapis.com/resource_type":"firewall" }, "logName":"projects/-SNIP-/logs/compute.googleapis.com%2Factivity_log", "receiveTimestamp":"2020-02-18T10:24:18.058392077Z", "resource":{ "labels":{ "firewall_rule_id":"-SNIP-", "project_id":"-SNIP-" }, "type":"gce_firewall_rule" }, "severity":"INFO", "timestamp":"2020-02-18T10:24:18.018532Z" }, "messageId":"1234567890", "message_id":"1234567890", "publishTime":"2020-02-18T10:24:19.193Z", "publish_time":"2020-02-18T10:24:19.193Z" }, "subscription":"projects/-SNIP-/subscriptions/rl-fw-shared-firewall-sumologic-subscription" }
From what I can see, there is no way via the terraform provider to tell sumologic to create a 'Google Cloud Log'
Can anyone point me in the right direction ?
Many thanks
Please sign in to leave a comment.
Comments
0 comments