File too big. Cannot lookup from
Creating a lookup file seems it is getting larger and becoming a road block, is there a work around for this issue?
-
Lookup file file has a 8MB right now and there was a few votes out there to increase that soft cap.
For customers who has more than 8MB, what I recommend is break that down into multiple files. You an chain up lookups to achieve the same request.
Obviously, there are a couple of options to break it down. 1. by slice the existing large files into multiple of the same pieces but by different names, ie: blah_p1, blah_p2, etc. When you lookup, you can do something like this.
_view=user_session_logs| where !(onlineid ="" or onlineid="null")| lookup loyalty from /shared/online2loyalty_0 on onlineid=online| lookup loyalty from /shared/online2loyalty_1 on onlineid=online| lookup loyalty from /shared/online2loyalty_2 on onlineid=online| (isNull(loyalty) ? "Not buying" : "Bought with loyal card") as outcome| count by outcomeOr option 2, you could do cascade lookups.. ie: first lookup is a million of 'keys', then use the 'keys' to lookup for more "keys", then goes on and on.
HTH,
Monty
Please sign in to leave a comment.
Comments
1 comment