Suggestions For Handling Bulk Updates Without Blocking Local User Updates

Question

Hi,
This is a request for general implementation suggestions.
We have a CRM database that is used by a call center application to allow reps to update customer info during business hours.  Outside of business hours we receive data feeds from another source that are bulk uploaded into the database to refresh the
data. This has been working fine for now, but we are expanding the use of the app to offices in other countries and are beginning to encounter more blocking during the bulk upload because now the app is being used outside our local business hours because of
the time difference.
It seems this would be a common problem, but I haven't been able to identify a good source of information on methods to overcome this. 
What suggestions do people have to complete bulk loads while still allowing updates by local users?
Ideas I have been considering include duplicating the database and performing merge replication, using service broker to queue updates during the bulk load, using snapshot isolation or isolation levels with row versioning....
Any ideas would be greatly appreciated.
Thanks,
Reinis

Answer 1

I have considered trying to break the update into chunks, but my fear as you said is it will take a lot longer.
Quite a few years ago, I rewrote a process in our system to make it a set-based update for better performance. But I heard as late as today from our customer with the biggest volumes, that they are still running the old process which updates one by one,
because when they do all at once the blocks other operations. (Which admittedly is due to other shortcomings in the system.)
Anyway, I would recommend you to look into that, and particularly make the chunks size configurable. Maybe you are able to find a sweet spot where blocking is not a problem, but the chunks are till big enough to be efficient.
Both of those resolutions are feasible and I certainly know there will be significant changes we may have to undertake.  I was just thinking that this must be a really common issue now with the global reach of data and there must be people who have
handled it in different ways.
It's not really a simple problem, and the solution is likely to depend on the current architecure you have. What fits in one shop, may not fit in another. And most of all, one solution may be a lot less costly to implement than another.
Erland Sommarskog, SQL Server MVP, [email protected]