Is it possible to exclude partial keywords from search

Comments

8 comments

  • Avatar
    Ryan

    Hi Tony,

    Due to the way data is indexed and how keyword searches are performed, this isn't possible in the way you are looking to do this. An overview of this may be found here: https://help.sumologic.com/Search/Search-FAQs/Searching-by-keyword-returns-no-results-even-though-the-keyword-exists-in-messages

    Another caveat is that we don't allow the mix of the wildcard with special characters. When attempting to do so you will receive the following error:

    Unquoted search term can't mix wildcard, punctuation, and alphanumeric.

    However back to your question, a way you can do this is by using the inherent filtering capability of our parsing operators. For example, using your data, the following will exclude events where there is no value for the workerthread:

    | parse regex "workerthread:\s+(?<workerthread_details>.+)"
    0
    Comment actions Permalink
  • Avatar
    Tony Petronelli

    I responded to your comment via email a few days ago.  I am going to copy what I sent here...

     

     
    How would I incorporate the parse regex into a query.  I have tried the following (trying to exclude empty workerthreads form being returned in the results by using “- fields” and it is not working:  I do know that he query has grammar errors as well.
     
    (_sourceCategory=log/logmessages AND "workerthread”) 
    AND !"Internal Error"
    | parse regex "kernel:\s+(?<workerthread_details>.+)"
    | if(isEmpty(workerthread_details), 1, 0) as nada2 
    | parse “47-* " as id
    | count by id
    | sort by _count
    | where _count > 25
    | fields - nada2
     
    Thank you!
     
    -Tony P.
     
    0
    Comment actions Permalink
  • Avatar
    Ryan

    Hi Tony,

    Stepping through the example query you've sent through (and understanding there may be some errors in what you've posted)

    (_sourceCategory=log/logmessages AND "workerthread”) AND !"Internal Error"
    | parse regex "kernel:\s+(?<workerthread_details>.+)"

    I can see in the parse statement you've changed workerthread to kernel. This may be an issue as on the very first line you have specifically asked for only events that contain the text workerthread.

    Assuming your base search *is* returning results, at this point, only events that have a value for workerthread_details will be available for use. This is due to the filtering/dropping nature of the parse regex statement, specifically any event that does not contain the text kernel:, followed by one-or-more spaces, and then followed by one-or-more of *any* character (apart from spaces or newlines) will be dropped.

    | if(isEmpty(workerthread_details), 1, 0) as nada2

    As per the previous note, at this point you should only have events with a non-empty value for workerthread_details (i.e. nada2 will only evaluate to 0 here). Also, seeing as you don't use nada2 for aggregation or filtering later in this query, this statement doesn't actually need to be included.

    | parse “47-* " as id
    | count by id

    This will further filter the events you are looking at to those containing the string 47- followed by a string (followed by a space). Assuming there is matching data, the output of this would be a table containing values for id and a corresponding value for _count.

    | sort by _count
    | where _count > 25

    Yep, no problem here. Standard sort with a conditional statement.

    | fields - nada2

    At this point, there would not be a field called nada2 to filter out due to the previous aggregation, so this is asking Sumo to not display a field it doesn't actually have to display in the first place.

    I hope this is helpful :-)

    RE: the email, unfortunately email responses back to community threads aren't forwarded through, so please feel free to either reply back within the community forum, or alternatively join our public Slack channel at http://slack.sumologic.com where you can get assistance from myself, other Sumo Logic engineers, and other customers as well in real-time.

    Cheers!

    0
    Comment actions Permalink
  • Avatar
    Tony Petronelli

    I did not realize that the parse regex "kernel:\s+(?<workerthread_details>.+)" actually took care of omitting the results. I thought that I had to do something with the workerthread_details in order to omit them.

    THANK YOU!!!

    This was blocking some major work.  

    Regards!

    -Tony Petronelli

    0
    Comment actions Permalink
  • Avatar
    Ryan

    No problems! Just to round this conversation out, in the event you did need to keep the events that didn't match the parse statement (e.g. for additional parsing or aggregation), then you would use the nodrop qualifier in the parse statement.

    | parse regex "kernel:\s+(?<workerthread_details>.+)" nodrop
    0
    Comment actions Permalink
  • Avatar
    Tony Petronelli

    Yes, I was just going to send you an email regarding that...

    However, the regex statement catches anything after "kernel:\s+ and stores it in workerthread_details, right?  Because I am only looking to drop workerthread_details that are blank.  There are many others that I have yet to parse and don't want dropped and wasn't able to drop by using a keyword search expression and thus must use a regex against.

     

    0
    Comment actions Permalink
  • Avatar
    Tony Petronelli

    Hi Ryan,

    Any chance we can setup a call?  I must be missing something with the way parse regex works, because I am unable to get multiple various parse regex statements to work in sequence for example:

    | parse regex "workerthread:\s+(?<blank_messages>.+)"
    | parse regex "workerthread:\s5[a-z0-9]{3}:(?<non_blank_5>.+)"
    | parse regex "workerthread:\sb[a-z0-9]{3}:(?<non_blank_b>.+)"

     

    Thank you!

    -Tony P.

    0
    Comment actions Permalink
  • Avatar
    Ryan

    Hi Tony - I've reached out to you via our Public Slack channel at http://slack.sumologic.com. Will post any solution we come to in here for the benefit of anyone else who may be have a similar issue in future.

    0
    Comment actions Permalink

Please sign in to leave a comment.