Sumo Logic Kubernetes node metric (metric=node:node_cpu_utilisation:avg1m") missing data when Changing Scrape Interval

Problem:

When configuring Sumo Logic Kubernetes solution with Prometheus for metrics collection using HELM CLI command, and setting prometheus-operator.prometheus.prometheusSpec.scrapeInterval to be greater than or equal to 1m:

Ex.

helm install sumologic/sumologic --name collection --namespace sumologic 
--set sumologic.accessId=<SUMO_ACCESS_ID> 
--set sumologic.accessKey=<SUMO_ACCESS_KEY>  
--set sumologic.clusterName="<MY_CLUSTER_NAME>" 
--set prometheus-operator.prometheus.prometheusSpec.scrapeInterval=1m

An alternate method of implementing the same configuration is to use the prometheus-overrides.yaml file:

Note: This issue is specific to "node:node_cpu_utilisation:avg1m" metrics, other metrics will still work.

Cause:

The default setting for scrapeInterval is 30s as shown below.

prometheusSpec:
 ## Prometheus default scrape interval, default from upstream Prometheus Operator Helm chart
 ## NOTE changing the scrape interval to be >1m can result in metrics from recording rules to be missing and empty panels in Sumo Logic Kubernetes apps.
              scrapeInterval: "30s"

When the scrape interval is increased to 1m then metric for the node:node_cpu_utilization should average at a multiple of the scrape interval.

Resolution:

Option:1 

Undo the customization and set the Scrape interval back to the default of 30s.

Option: 2

To retain the increased scrape interval setting and avoid the empty panels in Sumo Logic Kubernetes apps., you will need to add the following setting to compensate for above stated scrapeInterval increase from 30s to 1m to the helm upgrade command

--set kube-prometheus-stack.nodeExporter.serviceMonitor.interval="30s"