Update rows in portions

Question

Hello,
I have a batch that creates output files. One file for each id. In a first step it only asigns a file_id to each record. The generation of the file is a second step after approval by a controller and not part of this problem.
DROP TABLE test_update;
CREATE TABLE test_update (
     processed VARCHAR2(1)
    ,amount    NUMBER
    ,file_id   VARCHAR2(3)
    ,id        NUMBER
INSERT INTO test_update (processed,amount,id) VALUES ('n',10,8);
INSERT INTO test_update (processed,amount,id) VALUES ('n',20,8);
INSERT INTO test_update (processed,amount,id) VALUES ('n',30,8);
INSERT INTO test_update (processed,amount,id) VALUES ('n',40,8);
INSERT INTO test_update (processed,amount,id) VALUES ('n',10,9);
INSERT INTO test_update (processed,amount,id) VALUES ('n',20,9);
INSERT INTO test_update (processed,amount,id) VALUES ('n',30,9);
INSERT INTO test_update (processed,amount,id) VALUES ('n',40,7);
COMMIT;Expected result:
id 7 file_id 1
id 8 file_id 2
id 9 file_id 3The current implementation opens a cursor to select distinct id's with unprocessed rows and loops.
For each id it calls a procedure that generates a new file_id and loops in another cursor through each record, inserts the file_id, adds up the amount and counts the rows.
I would like to do the second part in a single update and use
DECLARE
    x           NUMBER;
    y           NUMBER;
    new_file_id NUMBER;
BEGIN
    FOR r IN (
            SELECT  DISTINCT id
            FROM    test_update
            WHERE   processed = 'n'
    LOOP
        SELECT  NVL(MAX(file_id),0) + 1
        INTO    new_file_id
        FROM    test_update;
        UPDATE  test_update
        SET     processed = 'y'
               ,file_id = new_file_id
        WHERE   processed = 'n'
        AND     id = r.id
        --AND     ROWNUM < 3
        RETURNING SUM(amount)
                ,COUNT(*)
        INTO x,y;
        -- The following information will be stored in another table for approval
        dbms_output.put_line('id '||r.id||'#new_file_id '||new_file_id||'#SUM '||TO_CHAR(x)||'# COUNT '||TO_CHAR(y));
    END LOOP;
END;
id 7#new_file_id 1#SUM 40# COUNT 1
id 8#new_file_id 2#SUM 100# COUNT 4
id 9#new_file_id 3#SUM 60# COUNT 3It works, but now there is an additional requirement, that there is a limit of lines in each file, assume it is 3, then I need two files for id 8.
id 7#new_file_id 1#SUM 40# COUNT 1
id 8#new_file_id 2#SUM 60# COUNT 3
id 8#new_file_id 3#SUM 40# COUNT 1
id 9#new_file_id 4#SUM 60# COUNT 3I could put the content of the loop in a PROCEDURE, restrict the update with rownum and call it again and again until COUNT is less than the limit. Yet the finding of the rows to be updated is expensive because the actual table has millions of rows, the file limit is 50,000 rows and there can be up to 400,000 rows for an id.This would imply to call the procedure eight times. Therefore I would like to find a way to do it in one call.
Any idea?
Regards
Marcus

Answer 1

How about :
SQL> var last_file_id number
SQL> exec :last_file_id := 0
PL/SQL procedure successfully completed.
SQL> var row_per_file number
SQL> exec :row_per_file := 3
PL/SQL procedure successfully completed.
SQL> merge into test_update t
  2  using (
  3    select rid, dense_rank() over(order by id, bucket_id) as file_id
  4    from (
  5      select rowid as rid
  6           , id
  7           , trunc(
  8               (row_number() over(partition by id order by null) - 1) / :row_per_file
  9             ) as bucket_id
10      from test_update
11      where processed = 'n'
12    )
13  ) v
14  on ( t.rowid = v.rowid )
15  when matched then update
16   set t.file_id = :last_file_id + v.file_id
17     , t.processed = 'y'
18  ;
8 rows merged.
SQL> select * from test_update order by id, file_id;
P     AMOUNT FIL         ID
y         40 1            7
y         10 2            8
y         20 2            8
y         30 2            8
y         40 3            8
y         20 4            9
y         30 4            9
y         10 4            9
8 rows selected.
SQL> select id, file_id, sum(amount), count(*)
  2  from test_update
  3  where file_id > :last_file_id
  4  group by id, file_id
  5  order by id
  6  ;
        ID FIL SUM(AMOUNT)   COUNT(*)
         7 1            40          1
         8 2            60          3
         8 3            40          1
         9 4            60          3