How to determine sum of data usage (storage) for parsed searches
How would I determine the sum of the storage/volume usage based on the following search data (sum(mbytes) as mbytes):
_sourceCategory=app_service | parse "* * * * *+* [*] * * - *" as iso_time,hostname,log_type,dater,time,timezone,service_type,syslog_level,java_app,other | count by hostname,java_app | sort by _count
example data:
2014-07-01T22:17:18.190368+00:00 server-service-prod-i-6e0xxxxx service_log 2014-07-01 22:17:17.358+0000 [qtp123456-123456] INFO c.e.c.resource.UserFileResource - upsert; userId:89074xxxxxxxxxxxxxxxxx, uniqueFilename:ServerSaveSlotX_0.bin, filename:ServerSaveSlotX_0.bin
2014-07-01T22:17:18.190363+00:00 server-service-prod-i-6f7xxxxx service_log 2014-07-01 22:17:17.334+0000 [jetty-stats] INFO c.e.common.service.StandardServer - qtp1234567890{8<=7<=12/700,0}
required aggregate data:
|
#
| hostname
| java_app
| _count
|
|
1
| launch-service-prod-i-6e0xxxxx
| c.e.auth.oauth.OAuthAuthenticator
| 3,156
|
|
2
| launch-service-prod-i-a7exxxxx
| c.e.auth.oauth.OAuthAuthenticator
| 3,133
|
|
3
| serverinfo-service-prod-i-04axxxxx
| c.e.auth.oauth.OAuthAuthenticator
| 1,666
|
|
4
| serverinfo-service-prod-i-d55xxxxx
| c.e.auth.oauth.OAuthAuthenticator
| 1,119
|
|
5
| server-service-prod-i-6f7xxxxx
| c.e.c.resource.UserFileResource
| 731
|
|
6
| server-service-prod-i-41dxxxxx
| c.e.c.resource.UserFileResource
| 669
|
|
7
| server-service-prod-i-549xxxxx
| c.e.c.resource.UserFileResource
| 641
|
|
8
| server-service-prod-i-32cxxxxx
| c.e.c.resource.UserFileResource
| 636
|
|
9
| server-service-prod-i-6f7xxxxx
| c.e.common.service.StandardServer
|
603
|
...with the added column of gbytes or mbytes usage for the aggregate data. currently we can only view by sourceCategory using the built in sumologic volume usage search that uses the sourcecategory volume usage.
-
try this:
_sourceCategory=app_service | parse "* * * * *+* [*] * * - *" as iso_time,hostname,log_type,dater,time,timezone,service_type,syslog_level,java_app,other | count, sum(_size) as sum_size by hostname,java_app | sort by sum_size | (sum_size / 1024 / 1024) as sum_size_mb | (sum_size_mb / 1024) as sum_size_gb
_size has the size in bytes of the raw message.
if you need to run this over larger periods of time and/or more often, consider turning the query into a scheduled view.
chr.
Please sign in to leave a comment.
Comments
1 comment