sourceCategory for multiple AWS Lambda Applications

Follow

Chris Slack

• November 07, 2018 21:03

Hi all,

I'm new to SumoLogic and trying to determine the best way of aggregating CloudWatch logs for our AWS needs. In my case I have have numerous AWS Lambda functions generating CloudWatch logs and these functions group into several independent applications. My thought would be to organize the applications by sourceCategory (i.e. - <DEPT>/App1/Prod, <DEPT>/App2/Prod, etc.), as appears to be the norm for organization in SumoLogic. However, the SumoLogic provided Lambda functions for collecting CloudWatch logs (https://help.sumologic.com/03Send-Data/Collect-from-Other-Data-Sources/Amazon-CloudWatch-Logs) essentially send all CloudWatch logs to SumoLogic in one chunk so they can only be assigned a single sourceCategory.

Is there a best practice for this? I could replicate the SumoLogic Lambdas, rename them, and specify different environment variable for each and achieve what I'm looking for. However, it seems a bit wasteful to replicate everything simply to change the environment var. I'm probably missing something obvious.

If anyone can give me any suggestions or best practices I would greatly appreciate it. I have the opportunity to implement this from scratch at this point and would love to do it the right way from the beginning if possible.

Thanks,

Chris

0

• 

  Himanshu Pal

  • November 16, 2018 19:11

  Currently in our lambda functions

  sourceName is mapped to logStream
  sourceHost is mapped to logGroup 
  and there is single sourceCategory.

  One way is to use these fields to build your dashboards but I am assuming you want to have your own custom organization  <DEPT>/App2/Prod

  If one wants to have multiple sourceCategories then there are two ways of doing it both involve changing function's code

  I am using you can create <DEPT>/App2/Prod string using the messages and context information present in lambda environment.

  Our functions use a common utils library sumo-dlq-function-utils which exports SumoLogsClient object.
  1> Before passing it to SumoLogsClient you can add _metadata along with the message with category = <DEPT>/App2/Prod
  2> You can directly modify the common library's code by going to generateHeaders function in sumologsclient.js  and changing the sourcecategory there.

  Both these methods works because Sumo Logic supports passing source categories, source name and source host in request header in following way X-Sumo-Name:sourceName, X-Sumo-Category:sourceCategory,
  X-Sumo-Host:sourceHost

  0

  Permalink
• 

  Himanshu Pal

  • November 16, 2018 19:38

  There is also another way of doing this without changing the lambda code. One can use FER to overwrite _sourceCategory

  0

  Permalink

Please sign in to leave a comment.