Populating dashboard panel with results of most recent scheduled search
Is it possible to configure a dashboard panel to populate with the results of the most recently executed scheduled search?
I have a complex search query that takes a few minutes to execute. I want the results of this search to populate a dashboard panel. Because the query takes a few minutes to execute, I am trying to avoid the scenario where this search gets submitted each time I navigate to the dashboard.
Ideally I would like to have the search execute once a day at 1AM and have the dashboard panel show the results of this 1AM search execution (rather than resubmit the search).
Is this possible?
Are you looking to simply prevent re-running the complicated search regularly, or are you looking specifically to display the results of the most recent Scheduled Search?
If you want to prevent having to re-run a complex search for the dashboard, check out Scheduled Views. Once you turn your query into a Scheduled View, you can add it to a dashboard. It'll essentially be running in the background regularly, and it'll store the results in the InternalCollector index. Your dashboard query will search across the aggregate results from the SV rather than the log messages.
If you want to specifically display the results of the Scheduled Search, you can set it to "Save to Index" then have your dashboard query off that index.
"Save to Index" almost works, except there is a limit of 512 results that can be saved in the index. Unfortunately, I have thousands of results that need to be saved from this search.
I'll explain my use case. I am sifting through roughly 1 million log events per day to get a total count of distinct users during the past day. I have sliced the query as much as possible but there isn't a way to get a smaller set of log events to sift through. The count_distinct function is not allowed to be used in the Scheduled View search, so I am not exactly sure how to proceed.
There's a workaround you can do to mimic count_distinct() without the limitations. If you do a normal count then count the results, it will give you the distinct number. For example:
| parse " src_ip: *," as src_ip
| count src_ip | count
What we're doing here is grouping all the src_ip's in the query and giving the count as normal. So now if have 15000 rows, when you run the next "count" against this it will produce the single value of 15000.
Let me know if this helps!
Interesting. This looks like it solves my problem. The results of the searches don't match up though. I am trying this search:
* | count user_id | count
* | count_distinct(user_id)
The results of the two searches are not the same. The value returned from the count_distinct search is about 5% less. I see a warning message on the count_distinct search that reads "count_distinct saw more than 100 values, results may be approximate". Are results from the "count user_id | count" considered accurate while count_distinct is approximate?
Yes, the count_distinct() will be more of an approximation after 100 results.
Please sign in to leave a comment.