Is it a queue managing the BULK jobs?

Question

Hi all,
When creating several jobs to import several batches data to Eloqua by Bulk APIs, would the jobs run one by one? Are the execute by the creating order?
Thanks,
Biao

Answer 1

Hi Biao,
Based on my own observation, jobs are processed one by one, and the order is generally the order in which they were added to the queue. A data transfer service runs first to parse the amount of rows on the file, and then puts it back into the queue again for loading into the db. You might see this in your sync logs as .json files being converted to .csv once you trigger the sync.
Depending on the amount of data present after parsing, it can affect what order it is processed in due to some advanced load balancing logic in place. For example, if you push one really big file and a few smaller files, chances are, the system will more frequently have small chunks of time/resources free to process a smaller file. The worker looks at the oldest file in the queue, sees that it is 'too big' to handle in the time it has, skips it and then grabs the next file which may be smaller. In most scenarios however, you don't see the system skip over large files that often or for very long. The net effect of this behaviour is that if you load a massive file into the system that it can't process right away, it's not going to cause a few minutes of unnecesssary blockage for other smaller files it can manage right away.
Again, this is based on observation and such backend logic is subject to change/tweaking at any time.
Regards,
Bojan