Updating cutoffTimestamp does not trigger Sumo to update the source
We want to be able to temporarily disable collectors. We see that the sources have a parameter called cutoffTimestamp that we could set to be in the future, which would make it stop collecting for a while. It seems to work as expected, except for one thing. The only way I can get the collector to pickup the new value for cutoffTimestamp is to modify some other attribute. At first we did not have cutoffTimestamp set, so it was equal to 0. Then I changed it to 1527260653000. I did a GET request and the source still had 0. Then I changed the "pathExpression". It immediately picked up both the changes to "pathExpression" and "cutoffTimestamp". The same thing happened if I changed the description. It like updating that field on it's own is not enough to trigger an update to the source.
Am I doing something wrong or is that the expected behavior?
Here are some version details:
"collectorVersion": "19.155-13",
"osVersion": "3.2.0-69-virtual",
"osName": "Linux",
"osArch": "amd64",
-
Official comment
Clever use of cutoffTimestamp. The normal use case for this property is to prevent too many old logs from ingesting when deploying a new source. Confess I didn't test it myself, but trust your findings since it makes sense conceptually. Modifying this property on a source is normally a "no op" by design because the backdated logs would have already been ingested and the field is effectively replaced by the collector's cached value(s) for date/time of lasted ingested logs. seems you worked around in a reasonable way but if you think this worth changing the product, you might pop over to ideas.sumologic.com to add it there.
-
If you are JSON managed, then as you point out, there is no changing sources via API or UI after the fact. JSON managed collectors just look at the local files for updates, so cleanest approach is probably to remove the source JSON, and re-add later, making sure the cutoffTimestamp is set correctly and not overlapping with what was already collected. If by "nodes" you mean sources, you can sync at the directory level, and use 1 well named JSON file per source, and that might make things easier to maintain.
-
I am attempting to do a very similar thing. I have been able to setting cutoffTimestamp in the event that a particular source exceeds its budget, however when I set the cutoffTimestamp, the logs stop as expected....however once the cutoffTimestamp is reached it appears the collector pulls in from last log message rather than the cutoffTimestamp.
Please sign in to leave a comment.
Comments
5 comments