HTTP source/compressed logs
Hi there,
I'm trying to create a AWS Config dashboard from data ingested via an HTTP source. I can see data being collected through the 3 sources I have configured, however, this data does not display in the default AWS Config dashboard.
I have a feeling the reason behind this that the logs are compressed. But this shouldn't matter should it?
If you have any other suggestions on what might be causing this please let me know.
Cheers,
Ben
-
Hey Ben,
Most of the Sumo apps will expect your data to come in with a very specific SourceCategory name. The first thing I'd do is check what sourceCategory the app is expecting by clicking through one of the panels to open the query in a search, and looking at the sourceCategory it's searching for.
Then, look at your collectors/sources to see if they're using that same sourceCategory. Does that make sense?
Let us know what you find. If they match, then we can dig in a bit further. But to your original hypothesis, it shouldn't be because those logs are compressed...
-
Further to this.
What I have noticed today after analysing the log data in more detail is that one of my sources is rendering JSON (green text) and the other 2 sources are not. As a result I am successfully able to create a dashboard from the JSON source.
It's almost as if the 2 sources that are not displaying the JSON are not being un-compressed?
One thing worth pointing out, the source that generates the dashboard successfully has the S3 bucket, SNS and AWS Config services on the same account. The other 2 sources are on different AWS accounts. I'm not sure if this makes a difference but thought it worth mentioning.
Ben
Please sign in to leave a comment.
Comments
2 comments