Performance drop, waste of CPU Credits
Hello,
We have Windows EC2 t2.medium with installed SumoLogic Collector. When it is active, java.exe process takes lot of processor time and eats up all the CPU credits that results in the machine slow down.
What we have tried so far:
1. Reduce size of the logs analyzed. Updated path patterns so that only most recent log files are analyzed. That reduced size to around 140 MB. CPU credits usage doesn't seem do be different to the one when we analyze all logs (~ 2GB). Is 140 MB still too much for Sumo?
2. In user.properties set wrapper.java.maxmemory=256. Still memory used by java.exe floats around 190 MB, doesn't increase and still uses lot of CPU. wrapper.conf contains following lines as advised:
# Overwrite the defaults with any properties in user.properties
#include ./config/user.properties
3. Added to collector.properties line
local.win.event.collection.flag = 0
as advised in https://ianpaullin.com/2014/11/17/from-the-frontline-cpu-utilization-issue-on-windows-server/
4. Tried to set cpuTarget in collector.properties but it seems that 20 is a minimum and this value is reverted back to 20 each time when service is restarted.
5. Collector version is 19.182-25.
6. Collection status shows 13,629 message count.
Can somebody help with that?
-
Mykola,
Although I am the Product Manager for data ingest, I am not the expert at determining the optimal settings for your collector configuration. I assume that although this is a Windows Server, the Source you are configuring is a file source? Usually, high CPU usage is related to folder traversal, and the amount of regex processing that is occurring. Another factor is the size of the file that has to be parsed through. Others can chime in on the optimal settings.
The "local.win.event.collection.flag = 0" property that you mention is no longer relevant. That applies to Windows Event collection. It told the collector to use the Legacy API. We totally refactored the design of that Source. It is now much more performant, and we removed the support for the old, legacy Windows API.
Regarding the CPU Target setting. That is a best effort setting to try and keep the CPU utilized by the Sumo Collector process below that target. It only applies to the File Source. It will slow the rate of directory scans and decrease parallel file processing, to a minimum level, until the target is reached. However, if you have large file directories, are configured for lots of recursion, or have very large individual files, these will limit the impact of that setting.
Please sign in to leave a comment.
Comments
1 comment