How to ingest and parse Ruby on Rails logs

When collecting more complex application logs, like those generated by Ruby on Rails (RoR) applications, you must complete a few essential steps to make sure that your logs are ingested and parsed correctly.

Set up a Ruby on Rails (RoR) Log Source

RoR generates multiline log entries, which require a few extra steps to configure. The essential elements are: the log file path, the time zone, and the multiline setup.

1. Create a new Source. Then, under File Path, input the correct log path. 

2. Enable timestamp parsing. If your logs do not contain a time zone as part of the timestamp, you will need to specify the time zone of the messages. Make sure to set the time zone correctly. Misconfigured time zones can cause problems. For example, in searches, data will appear to be in the future. And in Dashboards, no data will be displayed.

3. Enable multiline processing. To do this, under Boundary Regex, enter what the first line of your RoR log looks like. Typically (depending on your configuration and platform) it will begin with with the words "Starting" or "Processing". Check your logs to confirm. Enter the string ^Processing.*  or ^Starting.*.

If you have configured your RoR logs to output something like the date before "Starting" or "Processing", you will need to match that as well (e.g., "^\[[0-9/:\s]+\]\s+Processing.*" See the screenshot for an example:

4. Click Save. 

5. In the Sumo Logic Web Application, go to Manage > Status to verify that the logs are being ingested. If you don't see any data coming in after 2-3 minutes, check that your filepath is correct, that the Sumo Logic Collector has read access to the logs, and that your time zone is configured correctly.

Parsing RoR Logs

Once you have ingested the logs, parsing is straightforward. Your logs might look something like this:

[29514] [07-10-13 12:54] : 

Processing SomeComponent#show (for 111.111.111.111 at 2013-07-10 12:54:19) [GET]

[29514] [07-10-13 12:54] :   Parameters: {"slug"=>"2013/7/7/4502012/someurl", "action"=>"show", "controller"=>"somecomponent"}

[29514] [07-10-13 12:54] : Rendering template within layouts/somecomponent

[29514] [07-10-13 12:54] : Rendering somecomponent/article

[29514] [07-10-13 12:54] : Completed in 934ms (View: 850, DB: 75) | 200 OK [http://www.somecompany.com/someurl]

[31594] [07-10-13 12:54] : 

For that format, you could do a search query like this:

_sourceCategory=*RoR*
| parse "Completed in *ms (View: *, DB: *)
| * OK" as totalTime,viewTime,dbTime,returnCode
| timeslice by 1m
| avg(totalTime) as avgTotalTime by _timeslice

For the latest information on Ruby on Rails, see http://rubyonrails.org/documentation/.