i have my application data in csv format and how to use sumo logic to query on that data
How sumologic collector determines dimension and metrics from csv files?
Is there any specific format the file should be:-
E.g of my file data.
time , action , location, urlcategory (All columns)
oct2013,allowed,india,im
nov2013,blocked,new york,social networking
-
If you have a Collector that is reading a .csv file each row of that file will be considered an individual log message. The Collector will not auto determine the column fields, this would need to be done via an initial parse statement in your queries to get the field name values. Note you will need to denote each field with a * and a comma as the separator.
ex 1.
| parse "*,*,*" as time, action, location
ex 2.
| parse "*,*,*,*" as time, action, location, urlcategory
This should result in an output like below.
time action location urlcategory
oct2013 allowed india im
nov2013 blocked new york social networking
You can then aggregate on these fields. For example to count the number of message lines by action
| parse "*,*,*,*" as time, action, location, urlcategory | count by action
-
Collector is not showing any output that it has received data
What could be the reason for this ?
i have selected source data as Local file:-
C:\Users\abhishek\trysumologic
My search query also doesnt show anything?
_sourceCategory=cstat|parse "*,*,*,*,*,*,*" as Time,companyid,locationid,latitude,longitude,socialnetworking,generalbrowsing
Sample Data:-
Time,companyid,locationid,latitude,longitude,socialnetworking,generalbrowsing
2013-11-27 10:22:12,1266529,9742086,31.5,34.75,0,2
2013-11-27 10:22:12,1266529,11260369,14.6042,120.982201,1,187
2013-11-27 10:22:12,1266529,6170740,10,8,0,39 -
Abhishek,
Make sure to select to select the right timezone, since you don't have it in the csv file. That could be the reason no data is showing up: https://service.sumologic.com/ui/help/Default.htm#Understanding_Time_Stamps.htm
I am guessing that if you put "now +14h" in the time window of the search, you will get results, since you seem to be about 12 hours ahead of the default Pacific Standard Time timezone. - Ben
-
We do not have access to our customers' accounts without going a support account process.
Look at the Status window (under the "Manage" menu up top) - https://service.sumologic.com/ui/help/Default.htm#Status.htm#Status.htm
Change the timescale to 24 hours and see if you ingested any data. If you did, it is most the timezone issue. I would add new lines to the file to see if they get ingested correctly. If not, then the path you put in the source may be wrong, or the collector does not have rights to the file.
-
Are you running the collector as a different user? Are you sure the collector has read access to the file? Since you set up that file in your personal directory, the collector probably does not have access. I would move the file to another directory, and make sure that the collector has read access to it.
-
What its the name of your source .csv file? I saw you have the path expression defined as C:\Users\abhishek\trysumologic but I do not see a file name as part of this. Is the file name "trysumologic" without a file extension? You need to specify the full path and name of the file in order for the Collector to find the file.
Please sign in to leave a comment.
Comments
10 comments