When collecting more complex application logs, like those generated by Ruby on Rails (RoR) applications, you must complete a few essential steps to make sure that your logs are ingested and parsed correctly.
Set up a Ruby on Rails (RoR) Log Source
RoR generates multiline log entries, which require a few extra steps to configure. The essential elements are the log file path, the time zone, and the multiline setup.
1. Create a new Source. Then, under File Path, input the correct log path.
2. Enable timestamp parsing. If your logs do not contain a time zone as part of the timestamp, you will need to specify the time zone of the messages. Make sure to set the time zone correctly. Misconfigured time zones can cause problems. For example, in searches, data will appear to be in the future. And in Dashboards, no data will be displayed.
3. Enable multiline processing. To do this, under Boundary Regex, enter what the first line of your RoR log looks like. Typically (depending on your configuration and platform) it will begin with the words "Starting" or "Processing". Check your logs to confirm. Enter the string ^Processing.* or ^Starting.*.
If you have configured your RoR logs to output something like the date before "Starting" or "Processing", you will need to match that as well (e.g., "^\[[0-9/:\s]+\]\s+Processing.*" See the screenshot for an example:
4. Click Save.
5. In the Sumo Logic Web Application, go to Manage > Status to verify that the logs are being ingested. If you don't see any data coming in after 2-3 minutes, check that your file path is correct, that the Sumo Logic Collector has read access to the logs, and that your time zone is configured correctly.
Parsing RoR Logs
Once you have ingested the logs, parsing is straightforward. Your logs might look something like this:
[29514] [07-10-13 12:54] :
Processing SomeComponent#show (for 111.111.111.111 at 2013-07-10 12:54:19) [GET]
[29514] [07-10-13 12:54] : Parameters: {"slug"=>"2013/7/7/4502012/someurl", "action"=>"show", "controller"=>"somecomponent"}
[29514] [07-10-13 12:54] : Rendering template within layouts/somecomponent
[29514] [07-10-13 12:54] : Rendering somecomponent/article
[29514] [07-10-13 12:54] : Completed in 934ms (View: 850, DB: 75) | 200 OK [http://www.somecompany.com/someurl]
[31594] [07-10-13 12:54] :
For that format, you could do a search query like this:
_sourceCategory=*RoR*
| parse "Completed in *ms (View: *, DB: *)
| * OK" as totalTime,viewTime,dbTime,returnCode
| timeslice by 1m
| avg(totalTime) as avgTotalTime by _timeslice
For the latest information on Ruby on Rails, see https://rubyonrails.org/
Comments
2 comments
Just wanted to add a caveat for those running Rails 3.2 on multi-process app servers like Passenger. You will need to implement the following monkey patch or else your logs will be interleaved between the different processes making the pretty useless for parsing out with the typical regex.
Here is a stack overflow article describing the issue: http://stackoverflow.com/questions/11561846/rails-3-2-2-log-files-unordered-requests-intertwined
Here is a git with our monkey patch which addressed this in our rails 3.2 apps: https://gist.github.com/rayrod2030/7139408
A
The regular expression in the screenshot is not working for Rails 4 or 5.
I believe it should be /^.*?Started.*?$/
Please sign in to leave a comment.