Incremental file load with agent scheduler

Question

Hello.
I have a problem with agent scheduler. At first Ill describe what I do:
I wanna load data from files named xx_20100101, xx_20100102, xx_20100103, and so on..
- I created interface which loads data into oracle table.
- File Data store (resource name) is called #file_name. #file_name is Variable computed from metadata table.
- This metadata table contains data about files and statuses. E.g. columns (File name,File date, Status) data (xx_20100101,20100101,READY).
- I created Procedure which sets Status from READY to DONE after successfull load from every file into table.
- All these steps are in Package. So when I run this package (with steps: Declare variable -> run Interface -> Run package -> Refresh variable).
- Variable is computed as record with min(File Date) and Status = 'READY'
After execution of package everything looks ok.
But when I execute Package via agent scheduler only step with Interface is correctly runned. Packege which changes status of file load does no update and variable is not refreshed. So that next run of Package computes with sam data as first run.
Where can be a problem?
Thanks.

Answer 1

Always use the #<PROJECTCODE>.<VARIABLENAME> syntax.
Where you want to use the variable in the file name it is best practice to run each iteration of the file load in a separate session. You should take the approach to invoke a scenario with the variable passed in as a parameter for each file to be processed.
Inside the "child" scenario you should declare the variable (making sure it is a non-persistent type) before you execute the load.
If you run the scenario synchronously you won't have to change the KMs to make them run in parallel by modifying the temporary table names - conversely, if you change the KMs to use unique table names for the temporary tables (C$, I$) then you will be able to run the scenarios asynchronously and in parallel.