Adding older log data with epoch timestamps, however, Time always matches Receipt Time

Follow

Aric Whitaker

• August 06, 2019 15:25

I am POSTing older log data to an HTTP collector using an EPOCH timestamp, however, Sumo populates the Time field with the same time as the Receipt Time.

The collector is configured with:

• Advanced Options for Logs ->
• [checked] Extract timestamp information from the log file entries
• [checked] Ignore time zone from log file and instead use: (UTC) Etc/UTC
• [checked] Automatically detect the format

I have also tested it with the configuration:

• Advanced Options for Logs ->
• [checked] Extract timestamp information from the log file entries
• [checked] Use time zone from log file. If none is detected use: (UTC) Etc/UTC
• [checked] Automatically detect the format

The format of the log messages:

[1512086400] {'keyA':'valueA', 'keyB':'valueB'}
[1512086400] {'keyA':'valueA', 'keyB':'valueB'}
...

(In this case, the epoch time equates to 12-01-2017 12:00:00 AM GMT)

However, when querying the logs, the Receipt Time and Time are identical, showing the time they were added to Sumo, and not the epoch time specified in the log file.

Expected: The Time for these messages should be December 1, 2017 12:00:00 AM, not the Receipt Time of when the data was POSTed.

Any help is appreciated!

1

• 

  Kevin Keech

  • August 16, 2019 20:37

  Hi Aric,

  Sumo Logic assumes that all log messages coming from a particular Source will have timestamps that are within a window of -1 year through +2 days compared to the current time. Any log message with a parsed timestamp outside of that window is automatically re-stamped with the current receipt time.
  
  See the following help for more information on ingesting historical data. 

  0

  Comment actions Permalink

Please sign in to leave a comment.