parseDate bug?

Follow

Corbin Phelps

• October 06, 2017 15:38

I think I may have found a bug with parseDate not correctly parsing a given date.

I'm diffing two date strings

Date 1: 2017-10-06T14:36:48.243177Z
Date 2: 2017-10-06T14:36:48.24416Z

Using the following logic:
toMillis(parseDate(date_str, "yyyy-MM-dd'T'HH:mm:ss.SSSSSS'Z'")) as date_millis

I'm getting the following milliseconds for each

Date 1 ms: 1,507,326,051,177
Date 2 ms: 1,507,325,832,416

I'm not sure how Date 2 clearly happening after Date 1 has a smaller millisecond value than Date 1.  Am I missing something or could this be a bug?

0

• Official comment

  

  Nick Wilson

  • October 09, 2017 19:30

  Hey Corbin,

  This is because Sumo's datetime parsing supports down to milliseconds (ss.SSS), not to the microsecond (ss.SSSSSS) as you have written in your query. So for example, here is a query that would truncate your dates and return an appropriate number:
  
  *
  | limit 1
  | formatDate(toMillis(parseDate("2017-10-06T14:36:48.243Z", "yyyy-MM-dd'T'HH:mm:ss.SSSSSS'Z'")), "MM/dd/yyyy hh:mm:ss.SSSSSS") as date_millis
  | formatDate(toMillis(parseDate("2017-10-06T14:36:48.244Z", "yyyy-MM-dd'T'HH:mm:ss.SSSSS'Z'")), "MM/dd/yyyy hh:mm:ss.SSSSSS") as date_millis2
  
  Or, more similarly to yours:
  
  *
  | limit 1
  | toMillis(parseDate("2017-10-06T14:36:48.243Z", "yyyy-MM-dd'T'HH:mm:ss.SSSSSS'Z'")) as date_millis
  | toMillis(parseDate("2017-10-06T14:36:48.244Z", "yyyy-MM-dd'T'HH:mm:ss.SSSSS'Z'")) as date_millis2
  
  Which returns 

  1,507,325,808,243

  1,507,325,808,244
  
  You would either have to truncate or round your milliseconds to fit them into the supported precision level.
  
  Let me know if this helps.
  
  Thanks,
  Nick

  Comment actions Permalink
• 

  Corbin Phelps

  • October 10, 2017 11:57
  • Edited

  Thanks Nick, that makes total sense. I'll adjust my query to handle that.
  
  I don't know if it's ideal behavior to give a completely wrong number back if the date I put in is past the millisecond precision that is supported. I think an error would be more ideal as right now it's not apparent that things are not working correctly without inspecting each conversion.

  0

  Comment actions Permalink
• 

  Nick Wilson

  • October 11, 2017 14:07

  Hi Corbin,

  I agree - I plan on filing a Jira for this and will post back here once I do that.

  Thanks,

  Nick

  0

  Comment actions Permalink
• 

  Nick Wilson

  • October 11, 2017 15:31

  Hi Corbin - I've gone ahead and entered a ticket for this bug. Thank you for identifying this!

  0

  Comment actions Permalink
• 

  Corbin Phelps

  • October 11, 2017 15:37

  Thanks for following up!

  0

  Comment actions Permalink

Please sign in to leave a comment.