XMII/BLS/Join Action block-Performance

Question

I am doing outer join between  2 xml output, and size the outputs 1 MB and 2MB. It takes 4 minutes.
Is there any aleternative way of joining which will minimize the duration. Duirng this operation CPU also peaks.

Answer 1

I have to say, those are very large xml files. In my experience, XML's structure doesn't really lend itself to high performance when using large files, period. On any platform, and I have dealt with it on many. Custom code may not get you the improvements in performance you are looking for. You should really revisit your process.  You could try looping on the smaller set and pulling the data you need out of the other source in a loop. Multiple SQL queries may be faster than the system traversing a large xml dataset. I have seen it. It works. You could then unwind the loop etc. Is cahing the data an option? Is this a situation where you can churn the bulk in the morning and then only deal with deltas through the day?
It might even be better to just dump the data into a common database.
have you determined how long it is taking to load the data from your queries?
Message was edited by:
        christian libich
Message was edited by:
        christian libich