Achieving Parallelism and avoiding Concurrency - Need Design Advice

Requirement:
Extract Data from the database using SSIS.
Design Details:
There are two tables
ReportExtract and ReportExtractQueue. ReportExtract is a reference table and contains all the details related to the extracts.
ReportExtractQueue is a Queue Table designed for SSIS Package to process the extracts as per priority.
2. The SSIS Package, at any point of time picks up only one ReportExtractID to process from ReportExtractQueue where the ProcessingStatus = ‘Q’ and the Priority is High (in ascending order). This is achieved
by calling a stored procedure which has logic as below.
DECLARE @ReportExtractID int;
SELECT @ReportExtractID = (SELECT TOP 1 ReportExtractID
FROM ReportExtractQueue
WHERE ProcessingStatus = 'Q' -- InQueue
ORDER BY Priority);
IF @ReportExtractID IS NOT NULL
BEGIN
UPDATE ReportExtractQueue
SET ProcessingStatus = 'P' --InProgress
WHERE ReportExtractID = @ReportExtractID
SELECT @ReportExtractID AS ReportExtractID
END;
3. Once the package picks up the ReportExtractID from the Queue, the ProcessingStatus is changed to ‘P’ (InProgress) and then processes the extract. After the Processing is complete the ProcessingStatus is
updated again to either ‘S’ (Success) / ‘F’ (Failed).
4. Scheduling for Parallel Execution: 
Four copies of the same SSIS Package are copied to four different folders.
Four Jobs are created, each referring to a copy of the SSIS Package.
The Jobs are scheduled to run every 15 minutes to poll the Queue table through SSIS package.
The Start Time for all the Jobs is same.
Problem: Since all the Jobs start at the same time and run every 15 minutes. How do I ensure each job picks up a unique ReportExtractID from the Queue table. So far, from what i tested in the test
environments I have not faced this issue. But I want to avoid Concurrency Issues on the Queue table.
If I use the rowlock, readpast hints on the Queue table, will it resolve the Concurrency Issues (if any).      
If yes, then should I use these hints even while updating the Processing Status? Should I follow any other design approach?.
Please advise
-- Praveen

The jobs will clash.
And the design is not scalabale.
SSIS is not the best solution here in general.
You better have the stored procedure pick records up and then run the processing package - many times, even in parallel - given this package
can tolerate multiple copies running. SQL Agent runs them in async mode, the package in Agent can start with sp_start_job
Arthur My Blog

Similar Messages

  • Dynamic Forms and Flowing subforms -- Need help/Advice

    Hello All -
    I am creating a form for instructors at our college to complete. It has 3 pages, some static text, some required drop-down boxes, and three "comment" type areas which I would like to create so that they "flow" and display the entire contents when printed.
    I've read everything I could find so far, but I think I am missing something quite basic.
    I've created master pages, with a content subform on each one. I've placed all my content into subforms under that main content subform, named and ordered them the way they need to be. Most of it should appear as positioned. Only where there is one of the comment sections do I want/need the "flow content" option.
    I tried following the directions I found in another thread. If I set the subform with the comment area to flow, then I never get the page break option (and it is needed) - it's always grayed out. When I try to set the underlying subform on the page to 'flow content', ALL of the content from all three pages ends up on one page, everything on top of each other.
    What am I doing wrong?
    Can anyone give me some tips as to where at least to start looking for what I've not done or done wrong?
    Thanks in advance for any information -- it will be soooo appreciated!
    Lauren

    Hi Lauren,
    You only need one Master page, and there's no need to put anything on it unless you want a header/footer that repeats on each page.
    If you want to have things flow from page to page, create one body page object, and set its content type to Flowed (the body page is itself a subform). In it you put subforms with text, fields, and so on in them. Since your form will spill from page to page based on how much text the instructors type into the expandable comments fields, break up the forms content into sections (subforms) such that the layout can break between the subforms without taking away from the readability. These subforms should have a content type of Position. Even though there's only one body page object in your design, since you have three pages worth of stuff, it will spill onto three pages.
    Now for the comments fields, which you want to wrap and expand. Make sure that each of your comments fields is in its own subform. Set the field to allow multiple lines and, in the layout tab, click the box to Expand to Fit (height). For the subform that contains the comments field, set the height to auto-fit (layout tab). Also for the subforms that contain comments fields, experiment with the "Allow Page Breaks Within Content" checkbox on the Subform tab. If you expect an instructor to blab on for more than an entire page of text, you will have to select this box, but otherwise I suggest leaving it unselected so that the entire comment remains together.
    Jared Langdon
    www.jlangdon.ca

  • Session/jstl  and load balancing - need expert advice

    After user loggs in, I am getting locale from db and storing it inside userSession along with userLogin. Inside jsp using <fmt> tag to display locale based resource property.
    HttpSession userSession = request.getSession(true);
    userSession.setAttribute("UserLogin", oRS.getString("user_login"));
    Config.set(userSession, Config.FMT_LOCALE, currentLocale);Also using filter after login page to check for User Session. Currently all working fine .
    But if i want to make use of load balancing, whether still it will work?
    1)I understand that if i depend on network config i.e. session request sticks to one server then i believe shld not be a problem
    2)But if that is not the choice then what else ? Client want to make use of serialization technique. I do not know much about it, how to implement it, any sample is greatly appreciated.
    3)what about jstl Config.set(..) whether any changes needed here?
    4)Do i need to change filter code.
    Pls pls help me, i really need expert advice on this.
    regards
    Manisha

    1)If i want cluster server to replicate the session
    then all attributes saved in the session have to be
    serializable.yes.
    2)In docs it is mentioned
    Use setAttribute to Change Session State
    In an HTTP servlet that implements
    javax.servlet.http.HttpSession, use
    HttpSession.setAttribute (which replaces the
    deprecated putValue) to change attributes in a
    session object. If you set attributes in a session
    object with setAttribute, the object and its
    attributes are replicated in a cluster using
    in-memory replication. If you use other set methods
    to change objects within a session, WebLogic Server
    does not replicate those changes. Every time a change
    is made to an object that is in the session,
    setAttribute() should be called to update that object
    across the cluster.
    Whether it means that if I want to store a object
    (which is serializable) e.g. 'String' inside session
    and use setAttribute() method then I do not need to
    make that object implicitly serializable?No, you still need the objects to be serializable.

  • Combined bar and line graph: need some advice

    Hi,
    I'm builing a small app for a customer. The basic functionality is showing the bar chart and the line (see the figure attached), and now I want to show
    the area above the line in another color. This is the cost saving for the customer. Can I do this? And how would you suggest me to achieve this!
    Can I also show the big dot as seen in the figure?
    Thanks!

    _Faust wrote:
    Hi,
    I'm builing a small app for a customer. The basic functionality is showing the bar chart and the line (see the figure attached), and now I want to show
    the area above the line in another color. This is the cost saving for the customer. Can I do this? And how would you suggest me to achieve this!
    Can I also show the big dot as seen in the figure?
    Thanks!
    Although I believe that graph would be possible by banging on the graphs hard enough, but it would pretty easy to render that image using a picture control since rectangles, lines, text, and circles are supported.
    The link I included above will take you to a thread devoted to Picture related threads.
    Ben
    Ben Rayner
    I am currently active on.. MainStream Preppers
    Rayner's Ridge is under construction

  • Installed Parallels and XP but need help...

    I saw a post yesterday mentioning to change the version of Windows you are installing to other and then after installing change it back to XP. How do I change it back to XP?
    Thanks much

    I also am having problems with the same type of printer you have but I can't even get my printer to come on. No matter how many times I press the on button nothing happens. All the ub cords are hooked up right, I inserted the CD it came with and have it as my main printer but I am getting no where with it. And the sad thing about it I was so convinced I could hook it up as I have hooked up other printers and got them to work that I opened the box got everything out and then threw the box in the dump. So now I can't take it back to get my money back. Could you please tell me how you got yours to at least come on????
      P.S. Mine is a printer, copier and scanner but I can't get it to come on.
    Thanks
    Jeanne

  • Hi, i am a littl confused as I logged into creative cloud and bough the in design plan for a year but i cant seem to donwload it... there is a window that pops up and it says its downloading but its taking forever? any advice?

    hi, i am a littl confused as I logged into creative cloud and bough the in design plan for a year but i cant seem to donwload it... there is a window that pops up and it says its downloading but its taking forever? any advice?

    Hi Dima,
    Please refer to the help documents below:
    Troubleshoot Creative Cloud download and install issues
    Error downloading, installing, or updating Creative Cloud applications
    Regards,
    Sheena

  • HT4623 I tried to download the app of the day and I was told I was in the US store and that I needed to go the the Canada store. This is the first time I have encountered this. As far as I know I set up my account with the US store. Any advice? Thanks

    I tried to download the " app of the day " and was told that I was in the US store and that I needed to go to the Canada store. I never encountered this before .
    As far as I know I registered in the US store. Does anyone have any advice.
    thank you

    See "Change your iTunes Store country" > http://support.apple.com/kb/HT1311

  • I need some advice about the macbook pro and iPhone 5. I took a video on my iPhone and tried to email it it said it was too big to send? So i downloaded it to my macbook pro and tried to mail it to no avail? The macbook tells me the server won't let it th

    I need some advice about the macbook pro and iPhone 5.
    I took a video on my iPhone and tried to email it it said it was too big to send? So i downloaded it to my macbook pro and tried to mail it to no avail? The macbook tells me the server won't let it through other mail goes through any ideas how to resize it or what it might take to send it?

    I agree with LowLister, the best option for you to share the video online is to upload it to your online storage account for example : Box, Dropbox, SkyDrive (All of them provide free storage beginning from 2GB).
    You can upload the files which you want to share in this online storage and then they have sharing options in which you'll will get the link of the file to be sent and send the email. You're good to go!
    Tip : You can store multiple files for backup purposes.

  • Plan on going to Manila for vacation and going to bring my iPad 4. Need your  advice what to do.

    Plan on going to Manila for vacation and going to bring my iPad 4. Need your  advice what to do.

    Hello Aongtan051,
    Thanks for using Apple Support Communities.
    For more information on this, take a look at:
    About Apple World Travel Adapter Kit
    http://support.apple.com/kb/HT4341
    The World Travel Adapter Kit includes a USB power adapter, a 30-pin to USB cable, and a set of six AC plugs with prongs that fit different electrical outlets around the world. For world travelers, this is the perfect kit to ensure power connectivity in most countries you may travel to.
    Best of luck,
    Mario

  • When I burn a disc from a playlist in iTunes, the track names and artist info does not display on the disc when friends view/listen to it. This probably is an easy fix, but I need some advice.

    When I burn a disc from a playlist in iTunes on my MacBook, the track names and artist info does not display on the disc when friends view/listen to it. This probably is an easy fix, but I need some advice.

    No, this standard has been implemented for several decades.
    How iTunes remembers audio CDs - http://support.apple.com/kb/TA27785
    It is just conceivable you burned a CD that exactly matched one already in the Gracenote database.  Gracenote uses track count and duration to decide what CD it is you have in a computer.  If you have one that matches that it will label it.  You could conceivably even have one that matches it that is not the correct one.
    This script will only work on Macs:
    http://dougscripts.com/itunes/scripts/ss.php?sp=cdtexttocdinfo - script to use CD-text file on CD to label CD in iTunes

  • Why we need Design Repository and Runtime repository seperately?

    Hi All,
    Anybody please tell me why we need design time repository and runtime repository seperately? And also I want to know the total OWB Architecture.
    Thanks,
    Siv
    Edited by: Siv on Feb 10, 2010 10:40 PM

    Hi Siv:
    Concerning Repository:
    Since OWB 10.2 there is only one Repository storing Design and Runtime-Components.
    excerpt from http://download.oracle.com/docs/cd/E10926_01/doc/owb.101/b12146/over.htm (OWB 10.1.):
    The Design Repository, installed in an Oracle database, stores the metadata definitions for all of the objects used in Warehouse Builder. This is where all of the design information is stored for the target systems you are creating. You can access metadata stored here using the client user-interface, or through OMB Plus, the Warehouse Builder scripting utility.
    The runtime repository, installed in an Oracle database, stores all of the deployment data as well as data from executed mappings and process flows. This is where the target environment information is stored for the business intelligence systems you create. This includes connection information for all of the deployment locations.
    Which version of OWB are you using?
    Greetings
    Guenther Herzog

  • Having some serious issues with my iPhone and need some advice

    I'm going to these two issues which have been occuring with me for the past weeks-
    1.) I am facing a problem with my iPhone 5's lightning port or the cable. This started happening all of a sudden; one second my phone is charging, and the other it is not. Even if it is slightly moved from its position, it would stop charging, and further shaking starts it again, so on and so forth. At first I thought the issue might be with my cable, so I asked my friend to bring his lightning cable to my house so we could test that. It happened again- one shake the phone will charge, another shake stops it. (Note that any kind of movement does this, not only shaking) I'm a bit skeptical about his (my friend's) cable too, though- it shows NOT CHARGING on his iPad while it is in fact actually charging. His cable is only 3-4 weeks old though, so this confuses me even more. I think the issue might be with my port, but as of now I cannot draw a valid conclusion to my problem.
    2.) The second problem is that I've noticed my battery drains a **** lot faster now- ever since the first issue started. I manage to get my phone charging by placing it very carefully on the table and it charges overnight. However, when I pick it up in the morning, and read some news, it drains from 100% to 99% in ~60 seconds. This never happened before. Before this, it used to be at 100% for at least 10-15 minutes (might be even more). If I reboot my phone, it shows 100% again, but it still drains a lot faster now. I know because I've been using this phone for the past 4 months now, and I know when it drains and when it does nots
    Please, help! I need some advice as to how I should approach this problem. My phone is in the warranty period, however, I would like to have an expert's opinion first.

    Hello,
    First: I've experienced the same problem with my iPhone 5. I've bought a new charger and it still did the same thing. I went to the Apple Store to get my iPhone checked out and they said that the pins inside my iPhone we're damaged, and I had to get a replacement.
    ~ Josh

  • Need Powerbook G4 and Logic 7.1 Optimization advice

    Hello Logic/Mac Users,
    I need optimization advice for my portable logic setup.
    I've got a PB G4 with a gig of ram, a Firewire 410, and some soft synths.
    My goals are basically to come up with a few tracks of ideas, nothing too grand.
    I basically want to use one track for Ultrabeat,
    another two for a couple of soft synths like the G-Force stuff.
    and Finally to plug in some bass guitar stuff via a Bass Pod Xt into the firewire device.
    any tips on how to get the most out of this existing setup would be very appreciated.
    I'm reading the manuals and playing with the tutorial tunes but any tips to expedite would be cool also.
    Also any suggestions on how to max out performance on the powerbook would be great.
    Regards
    Adrian
    1.5 ghz, 1 gig ram Mac OS X (10.3.9)
    1.5 ghz, 1 gig ram Mac OS X (10.3.9)

    Hi Stereovibe,
    Your powerbook G4 could record some audio tracks but don't count on
    adding plug-ins and instruments on top. A G4 processor is simply too slow
    to process and run Logic Pro 8 smoothly. Don't put too much money upgrading since
    you will not see a major speed bump doing so. I would keep my money and put towards
    a Macbook Pro dual core, used or new.
    My 2 cents,
    Marc

  • Hi all! I need to buy iPad tomorrow. Need your advice which iPad to buy? For almost the same money I can buy iPad 2   3g or iPad 4 wireless only. Both of them are 16gb and price difference is 70€ for iPad 4

    Hi all! I need to buy iPad tomorrow. Need your advice which iPad to buy? For almost the same money I can buy iPad 2 + 3g or iPad 4 wireless only. Both of them are 16gb and price difference is 70€ for iPad 4

    16GB vs 32GB vs 64GB: Which new iPad storage capacity should you get?
    http://www.imore.com/2012/03/08/16gb-32gb-64gb-ipad-capacity/
    How much content will fit on my iPod or iPhone?
    http://support.apple.com/kb/HT1867
    iPad Mini vs. iPad 2 vs. iPad 4: Which iPad Should You Buy?
    http://www.padgadget.com/2012/10/25/ipad-mini-vs-ipad-2-vs-ipad-4-which-ipad-sho uld-you-buy/
     Cheers, Tom

  • When making a circle, illustrator automaticly makes lots and lots of more circles... How to resolve this problem? Really need quick advice!!

    Hi there,
    I have used illustrator now for a long period of time. Recently I purchased Adobe CC. Had no problems with it untill now.
    It's like this. Whenever I try to make a circle (or, square, line or other forms) Illustrator makes lots and lots of more circles within it.
    From the moment I click on the page untill I release my mouse, it keeps making circles. (see enclosed picture)
    I have really no idea whatsoever how to undo this. I have used Illiustrator CC before for another project, and it worked fine. Now however, it doesn't.
    Can anyone help me out here? I have payed projects to deliver, and I really need this to work really soon...
    Thnx for your help!
    btw, I work on a PC with Windows 8. (don't really think it's relevant, but still..)

    Thnx!
    I didn't know that could cause it. Which is actually weird since I've recently bought a new laptop. >,<U
    Thnx for your help!

Maybe you are looking for