Sumo Logic Collector causing High Write IOPS (Input/Output Operations Per Second)

Problem:

As part of the general behavior, the Collector writes to disk after every read to update the "last read" signature to note where the Collector left off with the reading of a file or events. By default, every second the Collector saves the last read file (pointer) location in a source config file under the Collector. So, for N number of Sources, every second there will be N number of updates to these files. If you have a long list of sources configured under a collector these updates could result in higher than expected IOPS.

Resolution:

The following Collector properties can be updated within the /<sumo_home>/config/collector.properties file and control how often the Collector will write to the Source json files under the Collector, which can help reduce IOPS. 

For example, to increase the interval of these writes from 1 second to every 30 seconds, with a backup of every 3 minutes:

1. Stop the current Collector process.
2. Open the following file for edit:
   /<sumo_home>/config/collector.properties
   
3. Update the following parameters or add them to the end of the file.
   bladeManager.localPersist.interval.sec = 30
   bladeManager.fallback.localPersist.interval.sec = 180
   
4. Restart the Collector process to apply the changes. 

NOTE: Increasing these values may result in duplicated log ingest when an unexpected shutdown and subsequent restart event occurs due to the Collector not having the most up to date last read pointer for a Source. Any duplication due to an unexpected shutdown should be limited to the interval set within these parameters.