Local File Source is missing the last few lines on rotated files

Problem:

We have noticed that on one of our Local File Sources that a few of the log lines are not appearing to be sent to Sumo Logic. My log file has a rotation policy applied that will rotate the file every 500KB and will keep up to 10 rotated files. Is there a way to ensure the Collector gets the last bit of log messages that occurred just prior to the rotation of the file?

The logs look something like the following.

/var/log/mylog.log
/var/log/mylog.log.1
/var/log/mylog.log.2

And the path expression in our Source is configured as:

/var/log/mylog.log

Cause:

The Sumo Logic Collector will periodically scan the provided path expression looking for any new files that match that path and name. When a file is found it is opened for reading for a short period of time before the Collector will close the connection and then rescan the path expression. The Sumo Logic Collector keeps track of where it left off within a file by using a fingerprint that matches the first 2KB of a log file, as well as the last known byte position, read from the file. When the Collector scans the provided path and locates any files that match the expression it will then open and compare the first 2KB of those files to known fingerprints it has on record and if it finds a matching fingerprint it will then continue to read the file from the last known byte position. If a file rotation happens to occur at a time where the Collector does not have an open connection to the file it is possible the Collector may not get the last few messages, since that file has now been rotated to a different name which is not found given the path expression supplied.

Solution:

To address this problem it is recommended that within your Source you configure a path expression that will also match with the rotated versions of the file. This will ensure that the Collector opens the rotated file to get the last bit of data it was unable to read prior to the rotation. So for the above sample logs you would provide a path expression set like the following.

/var/log/mylog.log*

The wildcard added to the end of the file name will ensure that the Collector will find the rotated version of the file during the scanning process. The Collector will then open that rotated file, compare that to the fingerprint it had on record for the original version of the file, and will then read any new data from the last known byte position. This will ensure the Collector captures any messages that were added to the file just prior to the rotation