Problem:
We've been having trouble with some hosts not using the source configuration JSON file pointed by the syncSources parameter defining sources for the host.
Resolution:
Sources supplied via a json file may not be applied to a Collector for a few reasons:
- The Collector is not in Local Configuration Mode
- syncSources parameter is missing from the user.properties file or the path and file name are incorrect
- The JSON file is not in UTF-8 encoding
- The JSON within your file is not in valid JSON format
- The JSON within your file may contain invalid or missing keys
Solution 1 - Collector Not in Local Configuration Mode
The collector operating in JSON or local configuration mode can be verified by going to Manage Data -> Collection -> find the specific collector by inserting it's name in the search box -> Edit Collector and check in the Advanced section (see screenshot below) if the radio button has "Local Configuration File" selected, you will not see an Edit option on the right-hand side of the source, only the "Show" option
If the collector is in "Cloud Based Management" mode, toggle the above setting to "Local Configuration File" and then restart the Collector for the changes to take effect.
- Mac/Linux:
sudo ./collector restart
- Windows: use Task Manager restart sumo-collector service
Solution 2 - syncSources missing from user.properties
When a Collector starts it reads the syncSources parameter from the user.properties configuration file to determine the file path of the JSON file or folder with the Source configurations. If the syncSources parameter is missing or the path to the JSON file/folder is incorrect the Collector will be unable to read and apply your Sources.
- Open or create the user.properties file located at /<CollectorInstallationFolder>/config and verify the syncSources parameter and path to your JSON file is present.
syncSources=/path/to/sources.json
On Windows systems, you will need to use double-backslash within the file path:
syncSources=C:\\path\\to\\sources.json
- Save and close the file.
- Restart the Collector for the changes to take effect.
- Mac/Linux: sudo ./collector restart
- Windows: use Task Manager restart sumo-collector service
Solution 3 - JSON file is not UTF-8 encoded
Check the encoding of the json config file is in UTF-8 encoding. If your JSON file is saved under a different encoding, such as UTF-16, the Collector will be unable to properly read the contents of this file.
Solution 4 - JSON file is not in valid JSON format
If the JSON provided within your file is not in a valid JSON format the Collector will fail to read the contents of this file. If there is any formatting issues within the JSON file you may see the following message presented within the /logs/collector.log file.
com.sumologic.scala.collector.auth.UpdateFailedException: * Could not synchronize blade, failed to read the content of the json file
- To help verify the format of your JSON you can copy and paste the content into a JSON validation tool such as jsonlint.com. jsonlint.com will highlight any potential formatting errors such as missing commas, quotes, curly braces, brackets, etc. You may then correct your JSON as necessary
- We also recommend using the JSON view of an existing Collector to get a valid JSON configuration for your list of Sources. If you have an existing Collector with the proper Source configurations you need you can:
- In Sumo Logic select Manage Data > Collection > Collection.
- Click the
icon to the right of the Collector or Source.
- A dialog box opens to display the JSON configuration.
- Copy this JSON and place within your json file.
Solution 5 - JSON file contains incorrect or missing keys or values
Sumo Logic requires certain key/values be present within your JSON file. If the values supplied for a required key is not in the proper casing or of the proper type the Collector may fail to read and apply the Source information. A few common problems which may prevent a JSON Source from being applied are:
- Duplicate source names
Check that your JSON file does not contain any duplicate Source "names." Duplicate names will result in a failure of the Collector to read the content from your Source JSON and the following error message may be presented within the /logs/collector.log file.2017-05-16 20:30:25,084 +0000 [WrapperSimpleAppMain] INFO com.sumologic.scala.collector.auth.CollectorRegistrationManager - [main] Response code is 200 with warnings: List(Key: collector.error, Message: Error creating source: Server returned undeclared exception from cocoa-soa/CollectorStore: org.apache.avro.AvroRuntimeException: Unknown datum type com.sumologic.cocoa.api.DuplicateNameException: com.sumologic.cocoa.api.DuplicateNameException: Error creating sources List(ticker, ticker); a source already exists with that name.), errors: List()
- Invalid Key names
If a required key name is misspelled or not within the proper camel casing the Collector may fail to read and apply the Source information for that Source. For example, if you have supplied a key name of "sourcetype" instead of "sourceType" within a Source JSON. When an invalid key has been supplied you may see the following type of error message presented within the /logs/collector.log file.
2018-12-07 14:29:03,881 -0800 [JsonSync Manager] INFO com.sumologic.scala.collector.auth.CollectorRegistrationManager - [main] Response code is 200 with warnings: List(Key: source.type.invalid, Message: Please specify a source type), errors: List()
Note: See the following help link for proper key/value syntax.
- Invalid Values
If the values supplied for a required key is misspelled, not in the proper casing, or of the proper type the Collector may fail to read and apply the Source information. For example, the value for the "sourceType" key has been supplied as "localfile" instead of "LocalFile." When an invalid value is found for a required key you may see the following type of error message presented within the /logs/collector.log file.
2018-12-07 14:32:01,898 -0800 [JsonSync Manager] INFO com.sumologic.scala.collector.auth.CollectorRegistrationManager - [main] Response code is 200 with warnings: List(Key: source.type.invalid, Message: Invalid source type: 'Localfile'), errors: List()
Note: See the following help link for proper key/value syntax. - Duplicate port numbers in Sources
Under a Collector you can only have one Source that uses a specific port and protocol combination. If you have multiple Syslog and/or Metric Sources that specify the same port and protocol you may see the following type of error message presented within the /logs/collector.log file.
2018-12-06 10:52:08,896 -0800 [JsonSync Manager] INFO com.sumologic.scala.collector.auth.CollectorRegistrationManager - [main] Response code is 200 with warnings: List(Key: collector.error, Message: Error creating source: Error validating source.), errors: List()
Comments
0 comments
Please sign in to leave a comment.