Overcoming the IN LIST limit

Question

Hi,
I've recently posted a question about this but right now, I'm to understand the concept. I'm going to give a bit of background in order to make it more understandable.
We have an application in which an object (A) contains a list of objects (B). Each of these objects in turn may contain objects (C). Each of these is a different type of object
Currently, we have designed a very simplistic pattern that follows:
- Load all Objects A according to user criteria
- Build a list of comma-separated IDs of Object A and retrieve Objects B using SELECT * FROM B WHERE ID IN (1, 2, 3, etc...)
- Use the B objects returned to build a comma-separated list of IDs of Object B and retrieve Objects C using this list as: SELECT * FROM C WHERE ID IN (4, 5, 6, etc...)
Now, this works, but only in very simple conditions.
Obviously my main concern is that it just won't work if you return more than 1000 Objects A or B at any point in time. The main problem I am facing now is regarding the way to make this work better, in terms of coding pattern and SQL optimization. IN is not that great with large sets as I understand it since it will use conditions such as 1 OR 2 OR 3 etc...
I've checked out Tom Kyte's website and the table type solution (http://asktom.oracle.com/pls/ask/f?p=4950:8:::::F4950_P8_DISPLAYID:110612348061). It works, but my string parameter passed in could be more than 4,000 bytes for large reports... Soooo, I kept on reading and saw the BULK INSERT idea.
My question is the following: In an application where many users may run reports at the same time, is it even valid to consider doing this?
- LOAD --> BULK INSERT --> LOAD USING BULK INSERTED VALUES --> CLEAN UP CUSTOM TABLE
Hereby preventing other users from running their reports until this one is finished (unless I have one custom table per type of object)?
Or is the simple: LOAD OBJECTS A --> LOAD OBJECTS B USING SUBQUERY TO LOAD OBJECTS A better from the beginning? My main thing against it is that we run a query again after we just ran it so we already have the data we need but we have no choice but to re-write it.
I don't have anything against the BULK INSERT method etc... but for index usage and maintenance, the other solution just seems more complex for a performance that just would not be that great.
What do you think about it? And if you have the same problem as I do right now, how do you go around it?
Greg

Answer 1

Hi,
No no it's my bad, I actually did not realize that I mixing up some programming and Oracle concepts without being specific.
I'm assuming the problem with the 1000 list limit for an IN operator is understood though.
We perform several SELECT statements to retrieve records from our database. These records are used to instantiate java objects (this is in a J2EE application server, not in the Oracle JVM or java pool at all).
As an example (I apologize if it's a bad one), on the java side, we have a motherboard. It can contain one or more RAM slots. Each ram slot may contain a RAM module in it (for the sake of the example, I'll say it can contain one or more modules, just like old sockets...). On the database side, we have a MOTHERBOARD table, a RAMSLOT table, and a RAMMODULE table.
My question was the following: a user may run a report saying that he/she wants to get all motherboards' information in the database. Right now, the way it's been made is that we select all motherboards, get their unique identifier and build a comma-separated list of these ids (in the java code), and then do a
SELECT * FROM RAMSLOT WHERE MOTHERBOARD_ID IN (1, 2, 3 etc...).
And the same is done between RAMSLOT and RAMMODULE. The problem is that if more than 1000 motherboards exist, it just won't work.
Now, when I was looking at the ways to overcome this problem, I went to the Tom Kyte article mentioned. One solution is to create a function that will return a custom table type when passed in the string '1, 2, 3, etc...'. While this works with a small number of motherboards, well, if we loaded a lot of them and went over 4,000 bytes, we get a 'String Literal too long' error. This was mentioned on the AskTom article, to which the followup was: 'use BULK INSERT'. This is why I mention it. I guess what was meant is to load all the data in the custom type table and then perform as before.
This is why I was wondering how this could possibly be better than just re-writing the query as a subquery later (as undesirable as it might be).
Thanks for the help and please answer if anything isn't clear.
Greg