Percentile (PCT) operation does not generate expected results.

Problem:

Percentile operation does not generate expected results. When I compare the percentile results to other tools. Please let me know how Sumo Logic calculates the percentile?

Resolution:

Sumo Logic uses actual universal formula of percentile. It maps to the actual values and uses the approximate algorithm if the dataset have very large number of values.

There is bunch of information regarding it in the below documentation link

https://help.sumologic.com/05Search/Search-Query-Language/aaGroup/percentile-(pct)

The operator works in two ways:

1. The operator returns exact percentiles at under 1,000 data points.
2. At over 1,000 data points, the pct operator automatically switches to the t-digest algorithm for approximate results. This approximation is more accurate near the extremes (e.g., 99th and 1st percentiles) and less accurate closer to the median.

The other tools might interpolates the percentile value if it lies between two values in the dataset and don't necessarily map to actual values.

For example:
Lets say if you have below twelve values
Values
44,58,59,71,59,55,52,27,60,34,47,39

We want to find out below percentile
98 percentile
95 percentile

As per the global percentile formula (which Sumo Logic follow) the calculation would be like

  98th percentile = 71

  Solution:
  Step 1. Arrange the data in ascending order: 27, 34, 39, 44, 47, 52, 55, 58,
  59, 59, 60, 71

  Step 2. Compute the position of the pth percentile (index i):
  i = (p / 100) * n), where p = 98 and n = 12
  i = (98 / 100) * 12 = 11.76

  Step 3. The index i is not an integer, round up. (i = 12) ⇒ the 98th percentile
  is the value in 12th position, or 71

  Answer: the 98th percentile is 71

Similarly for 95th percentile

  Results
  95th percentile = 71
  
  Solution: 
  Step 1. Arrange the data in ascending order: 27, 34, 39, 44, 47, 52, 55, 58,
  59, 59, 60, 71

  Step 2. Compute the position of the pth percentile (index i):
  i = (p / 100) * n), where p = 95 and n = 12
  i = (95 / 100) * 12 = 11.4

  Step 3. The index i is not an integer, round up. (i = 12) ⇒ the 95th percentile
  is the value in 12th position, or 71

  Answer: the 95th percentile is 71

Below is the calculation in Sumo Logic, for same set of data