Collector Progress
I have setup a new collector source for AWS CloudFront logs from 30 days ago via an AWS S3 Bucket. Our data usage has sky rocketed to over 100 gb / day for the past 3 days (just for this single source). Pushing above our account's daily data limit.
How can I view the import progress of the collector so I can confirm if this is just a peak in our usage due to 30 days of historical data, or this is our current daily usage?
-
Hi Dallas,
the best place to start is the data volume app. Provided you enabled the data volume index: https://help.sumologic.com/Manage/Ingestion-and-Volume/Data_Volume_Index data volume has dashboards showing ingested GB by various dimensions (category, collector etc).
To look at what's coming in today you could run a search for say -60m using the 'use receipt time' option in the UI. That means you will see what sumo is ingesting from the source right now (rather than searching by timestamp). If you are seeing old timestamped logs in the recent receipt time range it means you have an ingest issue most likely.
It might be a good idea to log a support ticket about the source also. If the number of objects in the bucket if very large you can have issues of re-ingestion and the support team can help you fix that.
Please sign in to leave a comment.
Comments
1 comment