Sumo-ECS integration

Follow
Avatar
Vidhya Krishnamoorthy

Do you have any support for ECS? I am tryingt  to use sumo File Source Type but I want to better categorize the messages with Image Id and Container Id. I am not sure how to do it. Docker Labels dont work either I am using default JsonLogDriver

 

Any help is appreciated.

0

Comments

4 comments

Sort by Date Votes
  • Avatar
    Rahul Choudhary

    Hi Vidhya,

    Yes you can ingest ECS events into Sumo Logic using the steps mentioned in below KB article.

    Also I have highlighted the same just for your reference.

    https://help.sumologic.com/Send-Data/Data-Types-and-Applications/Amazon-EC2-Container-Service-(ECS)/01-Collect-ECS-Logs-and-Metrics#Collect_ECS_Events_using_CloudTrail

    Collect Metrics for Amazon ECS

    1. Configure a Hosted Collector.
    2. Configure an Amazon CloudWatch Source for Metrics.
      1. Name. Enter a name to display for the new Source.
      2. Description. Enter an optional description.
      3. Regions. Select your Amazon Regions for ECS.
      4. Namespaces. Select AWS/ECS.
      5. Source Category. Enter ecs_metrics.
      6. Access Key ID and Secret Access Key. Enter your Amazon Access Key ID and Secret Access Key.
      7. Scan Interval. Use the default of 5 minutes, or enter the frequency Sumo Logic will scan your CloudWatch Sources for new data.
    3. Click Save.

    Collect ECS Events using CloudTrail

    1. To your Hosted Collector, add an AWS CloudTrail Source.
      1. Name. Enter a name to display for the new Source.
      2. Description. Enter an optional description.
      3. S3 Region. Select the Amazon Region for your ECS S3 bucket.
      4. Bucket Name. Enter the exact name of your ECS S3 bucket.
      5. Path Expression. Enter the string that matches the S3 objects you'd like to collect. You can use a wildcard (*) in this string. (DO NOT use a leading forward slash. See Amazon Path Expressions.) The S3 bucket name is not part of the path. Don’t include the bucket name when you are setting the Path Expression.
      6. Source Category. Enter ecs_event.
      7. Access Key ID and Secret Access Key. Enter your Amazon Access Key ID and Secret Access Key.
      8. Scan Interval. Use the default of 5 The steps to perform the same has been put forth very well in below doc link. Also I have highlighted the same just for your reference.
      9. https://help.sumologic.com/Send-Data/Data-Types-and-Applications/Amazon-EC2-Container-Service-(ECS)/01-Collect-ECS-Logs-and-Metrics#Collect_ECS_Events_using_CloudTrail
      10. Collect Metrics for Amazon ECS
      11. Configure a Hosted Collector.
      12. Configure an Amazon CloudWatch Source for Metrics.
      13. Name. Enter a name to display for the new Source.
      14. Description. Enter an optional description.
      15. Regions. Select your Amazon Regions for ECS.
      16. Namespaces. Select AWS/ECS.
      17. Source Category. Enter ecs_metrics.
      18. Access Key ID and Secret Access Key. Enter your Amazon Access Key ID and Secret Access Key.
      19. Scan Interval. Use the default of 5 minutes, or enter the frequency Sumo Logic will scan your CloudWatch Sources for new data.
      20. Click Save.
      21. Collect ECS Events using CloudTrail
      22. To your Hosted Collector, add an AWS CloudTrail Source.
      23. Name. Enter a name to display for the new Source.
      24. Description. Enter an optional description.
      25. S3 Region. Select the Amazon Region for your ECS S3 bucket.
      26. Bucket Name. Enter the exact name of your ECS S3 bucket.
      27. Path Expression. Enter the string that matches the S3 objects you'd like to collect. You can use a wildcard (*) in this string. (DO NOT use a leading forward slash. See Amazon Path Expressions.) The S3 bucket name is not part of the path. Don’t include the bucket name when you are setting the Path Expression.
      28. Source Category. Enter ecs_event.
      29. Access Key ID and Secret Access Key. Enter your Amazon Access Key ID and Secret Access Key.
      30. Scan Interval. Use the default of 5 minutes. Alternately, enter the frequency Sumo Logic will scan your S3 bucket for new data.
      31. Enable Timestamp Parsing. Select the check box.
      32. Time Zone. Select Ignore time zone from log file and instead use, and select UTC.
      33. Timestamp Format. Select Automatically detect the format.
      34. Enable Multiline Processing. Select the check box, and select Infer Boundaries.
      35. Click Save.
      36. minutes. Alternately, enter the frequency Sumo Logic will scan your S3 bucket for new data.
      37. Enable Timestamp Parsing. Select the check box.
      38. Time Zone. Select Ignore time zone from log file and instead use, and select UTC.
      39. Timestamp Format. Select Automatically detect the format.
      40. Enable Multiline Processing. Select the check box, and select Infer Boundaries.
    2. Click Save.

    Hope above provided information helps. Let me know for any further question and I can help you out with that.

     

    Best,

    -Rahul

    0
    Comment actions Permalink
  • Avatar
    Vidhya Krishnamoorthy

    Hi Rahul:

    Thanks for your response.

     

    #1. As I researched furthur, I found another new docker logging option - https://github.com/SumoLogic/sumologic-docker-logging-driver

    Since this seems like a relatively new driver, is there any document in the sumo website about it? I also found that ECS agent supports this as of Sep29th. Can you tell me more about this?

     

    #2. The above option requires a hosted collector. Is there any extra pricing associated with the hosted collector?

    Thanks
    Vidhya

    Ano

     

    0
    Comment actions Permalink
  • Avatar
    Rahul Choudhary

    Hi Vidhya,

    Please find the answers to your follow-up questions as below:

     

    #1. As I researched furthur, I found another new docker logging option - https://github.com/SumoLogic/sumologic-docker-logging-driver

    Since this seems like a relatively new driver, is there any document in the sumo website about it? I also found that ECS agent supports this as of Sep29th. Can you tell me more about this?

    Answer: Please find the below documentation link that point towards the same GitHub community link for collector docker sources logs and stats.

    https://help.sumologic.com/Send-Data/Applications-and-Other-Data-Sources/Docker

    #2. The above option requires a hosted collector. Is there any extra pricing associated with the hosted collector?

    Answer: This is no extra pricing associated with hosted collector. 

     

    -Rahul

     

     

     

    0
    Comment actions Permalink
  • Avatar
    Vidhya Krishnamoorthy

    Thanks Rahul

     

    Are there any pitfalls using a hosted collector that we should be aware of? Since it is hosted in sumo cloud, I am curious to understand the SLA (if the service might go down, etc) - any loss of messages,  queuing , etc. Can you point me to any docs about this?

     

    Vidhya

    0
    Comment actions Permalink

Please sign in to leave a comment.

Didn't find what you were looking for?

New post