File too big. Cannot lookup from

Follow

Rey Padua

• July 12, 2018 14:33

Creating a lookup file seems it is getting larger and becoming a road block, is there a work around for this issue?

0

• 

  Monty Yao

  • July 21, 2018 17:44

  Lookup file file has a 8MB right now and there was a few votes out there to increase that soft cap.

  For customers who has more than 8MB, what I recommend is break that down into multiple files. You an chain up lookups to achieve the same request.

  Obviously, there are a couple of options to break it down. 1. by slice the existing large files into multiple of the same pieces but by different names, ie: blah_p1, blah_p2, etc. When you lookup, you can do something like this.

  _view=user_session_logs 

  | where !(onlineid ="" or onlineid="null") 

  | lookup loyalty from /shared/online2loyalty_0 on onlineid=online

  | lookup loyalty from /shared/online2loyalty_1 on onlineid=online

  | lookup loyalty from /shared/online2loyalty_2 on onlineid=online

  | (isNull(loyalty) ? "Not buying" : "Bought with loyal card") as outcome

  | count by outcome

  Or option 2, you could do cascade lookups.. ie: first lookup is a million of 'keys', then use the 'keys' to lookup for more "keys", then goes on and on.

  HTH,

  Monty

  0

  Permalink

Please sign in to leave a comment.