Complex queries via client library/REST API
[Posting this for Roman Vottner as we transition to the new Community platform]
We log a number of online/offline events for certain FTP endpoints we have to send data to. In order to query for all currently as offline marked FTP endpoints we came up with a query like this:
_sourceCategory=app/prod/ftp EventEntity
| parse "appType='FTP', connectorUuid='*', cId='*', cName='*', ts=*, remoteIp='*', event='*', eventText='*', uId='*', uName='*'" as(connectorUuid,companyId,companyName,timestamp,ipAddress,event,eventText,userId,userName)
| sort by _receipttime asc
| formatDate(_receipttime, "yyyy-MM-dd'T'HH:mm:ss") as timestamp
| last(event) as event, last(timestamp) as timestamp by companyId, companyName
| fields event,companyId,companyName,timestamp
| where event = "OFFLINE"
An endpoint is marked as offline until a further entry for the same companyId is found that was issued at a later time point and explicitly defined the event as ONLINE.
This seems to work in the query testfield if entered directly in the sumo portal. However, if we issue the query via the client java library (https://github.com/SumoLogic/sumo-java-client) we don't receive the same results.
The corresponding Java code does look like this (simplified for this example):
public GetMessagesForSearchJobResponse queryOfflineFTPEndpoints(int offset, int limit) {
HttpUtils utils = new HttpUtils();
SearchJobClient client = new SearchJobClient(utils);
Credentials credentials = new Credentials(accessID, accessKey);
ConnectionConfig config = new ConnectionConfig("https", "api.eu.sumologic.com", 44
3, credentials);
StringBuilder query = new StringBuilder();
query.append("_sourceCategory=app/prod/ftp EventEntity ")
.append("| parse \"appType='FTP', connectorUuid='*', cId='*', cName='*', ts=*, remoteIp='*', event='*', eventText='*', uId='*', uName='*'\" as(connectorUuid,companyId,companyName,timestamp,ipAddress,event,eventText,userId,userName) ")
.append("| sort by _receipttime asc ")
.append("| formatDate(_receipttime, \"yyyy-MM-dd'T'HH:mm:ss\") as timestamp ")
.append("| last(event) as event, last(timestamp) as timestamp by companyId,companyName ")
.append("| fields event,companyId,companyName,timestamp | where event = \"OFFLINE\"");
LocalDateTime today = LocalDate.now().atStartOfDay();
LocalDateTime now = LocalDateTime.now();
LOG.info("Looking up offline FTP connectors from {} till {}",
dateFormatter.format(today), dateFormatter.format(now));
String searchId = client.createSearchJob(config, new CreateSearchJobRequest(query.toString(), dateFormatter.format(today), dateFormatter.format(now), "Etc/UTC"));
GetMessagesForSearchJobRequest jobRequest = new GetMessagesForSearchJobRequest(searchId, offset, limit);
return client.getMessagesForSearchJob(config, jobRequest);
}
If I invoke this method with an offset of 0 and a limit of 10 I get 10 online endpoints but not the offline ones.
How do I have to modify the Java code in order to return correct results?
TIA
-
[Posting this for Roman Vottner as we transition to the new Community platform]
Thanks to Raghu Murthy from the Sumo support I was able to fix this issue. The main problem was that we were using GetMessagesForSearchJobRequest/Response rather than GetRecordsForSearchJobRequest/Response though the latter one is needed in case of aggregator functions (i.e. grouping by last(), first(), ...)
Please sign in to leave a comment.
Comments
1 comment