collector reports sourcename as underlying filename rather than name of symlink

Comments

3 comments

  • Avatar
    Steve Adams
    Attached Code Snippet : https://community.sumologic.com/s/rich_text_post/a090L00001QvG2iQAF
    0
    Comment actions Permalink
  • Avatar
    Mario Sanchez
    Scott, I'm reporting @Steve Adams's – response as it was posted as a Code Snippet before. Here you go. --------- Hi Scott, My approach to collecting logs from Kubernetes was to deploy a SumoCollector as a Daemonset with the docker source pre-configured, this will use the docker API to stream the logs. Sumokube is similar expect it gets the logs from the host /var/log/containers directory. The issue, which you described, is all the logs fall under the same sourceCategory making it very hard to search and build dashboards. My solution to this was by using Field Extraction Rules (FER) (http://help.sumologic.com/Manage/Search_Optimization_Tools/Manage_Field_Extractions/Create_a_Field_Extraction_Rule) Using FER's we can dynmically override the fields on ingestion. The below FER will extract the namespace, pod and container and rename _source, _sourceName and _sourceCategory. It will also add a field called namespace. NOTE: I've only tested this FER within my k8s cluster so it might need some tuning depending you name your pods. _sourceCategory=kubernetes | parse regex field=_sourceName "k8s_(?<_source>[^\.]+)\.[^_]+_(?<_sourcename>[^_0-9]+?)(?:-(?:ip|\d)[^_]*)?_(?[^_]+)" | format("%s.%s.%s", _source, _sourceHost, namespace) as _sourceName | replace(_source, "-", "/") as name | format("%s/%s", namespace, name) as _sourceCategory | fields - name Hope that helps?
    0
    Comment actions Permalink
  • Avatar
    Scott Bessler
    > Daemonset with the docker source pre-configured, Mind sharing how you did this? Its unclear what to set the URI to.
    0
    Comment actions Permalink

Please sign in to leave a comment.