Lookup against CSV that's stored within SumoLogic itself?
My search results for log data return IP addresses of hosts which I need to be resolved to hostnames. I have a CSV hosted on a web server and I can get SumoLogic to look up each IP in the CSV from the external web server and return the hostname for inclusion in the results. I'd like to stop using the web server and instead have the data held directly in SumoLogic. Is this possible? None of the IP address and hostname data is held anywhere in my logs and I don't want to create a collector simply to look up the contents of a file to get the data into SumoLogic. Is there any way I can upload data like a CSV and use it?
Thanks
-
Official comment
Hi Kenneth!
Historically I would recommend one of the following:
- Externally HTTPS hosted CSV (what you've been doing)
- Dedicated collector to monitor and upload the file (what you'd rather not do), or;
- Combining the data from either step 1 or 2, and apply a unnecessarily complex search query to save this to a lookup stored in the platform (what no-one should have to do)
Now the (soon to be) good news is that we'll be announcing a major uplift of our lookup capabilities at our user conference 'Illuminate' next month which will address your requirements (and then some!). So while I don't have a nice solution for you today, I can assure you than help is on the way :-)
Cheers!
- RJ
Comment actions -
Hi Guys,
I wanted to give a quick update about the improvements to a lookup table. Here are a couple of things we are going to address.
1.) Users will be able to manage lookup tables using UI. They will be able to perform CRUD operations from UI, including uploading data into Lookup file from a CSV file directly.
2.) Fine-grain RBAC control on lookups. Users will be to grant View/Edit & manage access to various users/groups within their organization. This will help in managing lookup tables.
3.) Lookup size limit will be increased from 8MB to 100MB
4.) Lookups will perform 10x faster, so you should be able to use them in a non-aggregate query without any problems.
Currently, our plan is to have these ready for beta by the end of this year. if you are interested in trying out the functionality in beta, reach out to the rep associated with the account so they can get you into the beta.
Angad
PM, Sumo -
Is the combination of collector + "save" query the only way to generate/update the local lookup file?
It sounds like the beta UI allows you to upload CSV directly. What about via API?
We need to update the lookup table on a daily basis in an automated manner. So direct access and update to the look up table would be ideal (no collectors / scheduled save query).
Please sign in to leave a comment.
Comments
6 comments