Analytical Functions: Parent/child bucketing (distribution)

Question

OK here is an interest problem for the analytical functions hardcore users. I have a parent and child tables. A parent record can have N number of child records. The child table has a FK to the parent table so all child records have a parent. I need to distribute my parents into an N number of buckets based on the number of children they have. The number of buckets will be variable and user-defined. Each bucket should have a similar number of child records but the number of parents is not important. Here a simple output example with 6 parents using 3 buckets of 10 children each:
Parent   Number_Of_Children           Bucket
1        5                            1
2        5                            1
3        6                            2
4        3                            2
5        1                            2
6        4                            3At first this looks like an easy job for the NTILE analytical function to bucket the child records. Another solution could be to use the ROW_NUMBER analytical function to number every child record and then use the WIDTH_BUCKET analytical function to bucket the parents based on their children ROW_NUMBER's value. But there is an additional requirement that makes these 2 approaches unusable. When doing the bucket distribution I need to guarantee that all children from the same parent are bucketed into the same bucket. This obviously makes it much more difficult. The distribution will obviously leave non-equiheight buckets since there will be no guarantee I can fill all my buckets with the same number of child records. Small differences are aceptable however. The distribution can be random and doesn't need to be the best possible distribution (the one that will cause the buckets to be as equiheight as possible). Finally this needs to be a SQL query not PL/SQL. Thanks

Answer 1

Thanks Frank, that was very helpful. I did some research on bin packing and indeed there doesn't seem to be a SQL solution for this kind of problem. Lucky for me some of my requirements are slightly different to the bin packing problem so I was able to come up with something that works. My requirements are that I need to bucket the data by the number of bins rather than by the size of the buckets (unlike most bin packing problems where the size of the bin is fixed). So the bin's size will based on the total number of children divided by the number of desired buckets. The buckets don't have to (and probably won't be) of the same size given that parents can have N number of children. Furthermore we are not lucking to fill the buckets as much as possible, small differences are OK. Based on the above I came up with the following solution.
1) I calculate the size of each bin based on the total childs and the number of bins.
2) I then adjust this size finding the Top N parents by child where N = number of bins. This is done since I am going to bucket parents by simple order (Parent ID and Child ID). This means I will most likely end up with parents that have children in two buckets which obviously is not desired. My approach is to move these parents to the last bucket hence I have to increase the bucket size. In the worst case scenario I will have the parents with most childs overflowing into the next bucket for all buckets. So I find out which are the Top N parents and then increase my total child universe by that much.
3) I then bucket the data using the WIDTH_BUCKET function and ROW_NUMBER by simple order (Parent ID and Child ID)
4) I then find out the parents that have children in two buckets.
5) And finally I move the "broken" parents to the last bucket.
This approach seems to work well. I ran it for ~130,000 parents 500,000 childs and the query returns the bucketed data in 4 seconds. The number of buckets does not affect the performance of the query.
WITH total_bins AS
     -- Load the number of bins
     SELECT 5 AS BIN_COUNT
       FROM DUAL
count_childs AS
     -- Calculate the sum of child records for top N parents based on the number of bins and the total number of child records
     SELECT SUM(CASE WHEN ranked.RANK <= total_bins.BIN_COUNT THEN ranked.CHILD_COUNT ELSE NULL END) AS TOP_N_CHILDS,
            SUM(CHILD_COUNT) AS TOTAL_CHILDS 
       FROM
               -- Rank records by their child count
               SELECT CHILD_COUNT, ROW_NUMBER() OVER (ORDER BY CHILD_COUNT DESC) AS RANK FROM 
                      -- Count all child records for each parent
                      SELECT PARENT_ID, COUNT(1) AS CHILD_COUNT FROM PARENT_CHILD GROUP BY PARENT_ID ORDER BY 2 DESC
          ) ranked
          CROSS JOIN
          total_bins
bins AS
     -- Calculate each bin's size based on the number of childs and bins
     -- top_n_childs is used to increase the bins in case of childs from 1
     -- parent falling into 2 different bins
     SELECT CEIL((TOTAL_CHILDS + TOP_N_CHILDS) / total_bins.BIN_COUNT) AS SIZE_EACH,
            CEIL((TOTAL_CHILDS + TOP_N_CHILDS) / total_bins.BIN_COUNT) * total_bins.BIN_COUNT AS SIZE_ALL
       FROM count_childs
            CROSS JOIN
            total_bins
bucket_data AS
     -- Bucket data using WIDTH_BUCKET data function, most likely child records will end in 2 buckets
     SELECT PARENT_ID,
            CHILD_ID,
            WIDTH_BUCKET(ORDER_NUMBER, 1, (SELECT bins.SIZE_ALL FROM bins), (SELECT total_bins.BIN_COUNT FROM total_bins)) AS BUCKET
       FROM
            SELECT PARENT_ID, CHILD_ID, ROW_NUMBER() OVER (ORDER BY PARENT_ID, CHILD_ID) ORDER_NUMBER FROM PARENT_CHILD
broken_parents AS
     -- This finds out which parents have their child records on more than 1 bucket so they can be fixed
     SELECT PARENT_ID FROM bucket_data GROUP BY PARENT_ID HAVING COUNT(DISTINCT BUCKET) > 1
fixed_data AS
     -- Join bucket_data and broken_parents and moved broken_parents to the last bucket
     SELECT bucket_data.PARENT_ID,
            bucket_data.CHILD_ID,
            CASE WHEN broken_parents.PARENT_ID IS NOT NULL THEN (SELECT total_bins.BIN_COUNT FROM total_bins) ELSE bucket_data.BUCKET END AS BUCKET
       FROM bucket_data,
            broken_parents
      WHERE bucket_data.PARENT_ID = broken_parents.PARENT_ID (+)
SELECT PARENT_ID,
       CHILD_ID,
       BUCKET
  FROM fixed_data
-- Check number of childs per bucket
-- SELECT BUCKET, COUNT(1) FROM fixed_data GROUP BY BUCKET ORDER BY 1
-- Check all parents have their childs on the same bucket
-- SELECT PARENT_ID FROM fixed_data GROUP BY PARENT_ID HAVING COUNT(DISTINCT BUCKET) > 1