SQL Union all VS SSIS Union All

I have 4 source SQL queries which is pointing different db's in the same server. All 4 queries results need to stored in single SQLServer table of different server. which one is best approach mentioned in below ?
1) Writing SP using union all option with all four source queries and use this sp in single data flow task[single source and destination tasks].
2) 4 source tasks to execute the 4 sql queries and then use Union all transformation to club the result and then one destination.
3) 4 different data flow task. Each data flow task point the different souurce queries and the same destination.
Please suggest in all aspects.

Prabu,
You are right in your observations from your previous post. Based on my experience, number 2 would be the best option.
First of all, it might be best to parallelize the extraction because the time taken to read from all sources equals the time to read from the source with the highest volume of data, whereas reading from sources serially would result in a sum of the time
taken to read from each of them individually.
Secondly, BULK LOADS with table locks are very fast, but you can only have one process writting to the same destination at the same time, because a table level lock is placed on the object. This will probably offset the performance gains of a parallel insert
into the same destination, but it is worth testing.
IF, and only IF, your process uses a lot of CPU time for some reason (transformations are performed), then there is a chance option number 3 will outperform option number 2 because CPU workload would generally be split between multiple processors, but I
dont think this is your case.
Just because there are clouds in the sky it doesn't mean it isn't blue. Some people would disagree.

Similar Messages

  • Execute Process Task showing all the SSIS Config variables in SQL Job History

    Hi,
    Am using an Execute Process task to execute my child package. And executing the Parent package from a SQL JOB.
    I am using the same config file for both Parent and Child packages.
    After the Job execution was Successful / Failure, in Job history it is showing all the variables from Config file which are not using in Child package. With the below message.
    "The package path referenced an object that cannot be found"
    I don't want to catch all the Variables information in Job History. Instead I need only the Success / Failure message.
    PFA Screen.
    Thanks,
    Sailaja

    see
    http://www.mssqltips.com/sqlservertip/1417/custom-logging-in-sql-server-integration-services-ssis/
    https://www.simple-talk.com/sql/ssis/ssis-event-handlers-basics/
    Please Mark This As Answer if it solved your issue
    Please Vote This As Helpful if it helps to solve your issue
    Visakh
    My Wiki User Page
    My MSDN Page
    My Personal Blog
    My Facebook Page

  • How to set SQL Server Login MUST_CHANGE, CHECK_POLICY, CHECK_EXPIRATION all to OFF with T-SQL

    How to set SQL Server Login MUST_CHANGE, CHECK_POLICY, CHECK_EXPIRATION all to OFF with T-SQL.  SSMS will not allow me to change policies to OFF. Error: 15128 - The CHECK_POLICY and CHECK_EXPIRATION options cannot be turned OFF when MUST_CHANGE is
    on.  I am attempting to change these options in a test environment. Thanks, Richard

    It appears you have to change the password first(it could the same password but an alter command needs to be done)
    check this : http://www.sqldbadiaries.com/2010/11/07/the-check_policy-and-check_expiration-options-cannot-be-turned-off-when-must_change-is-on/
    http://sqldbpool.com/2012/10/08/the-check_policy-and-check_expiration-options-cannot-be-turned-off-when-must_change-is-on-microsoft-sql-server-error-15128/
    Hope it Helps!!

  • Looking for MS SQL Server(2012 or older versions) all commands

    HI,
    Looking for MS SQL Server(2012 or older versions) all commands, i.e., all available sql server commands.
    Is there any reference manual or MS KL to get all commands in SQL Server ?
    Thanks in advance

    Thanks for link
    Sean Gallardy and Jingyang Li.
    This is is not usefull for my requirement.
    What i am looking is..SQL Server commands  like examples... DDL/DML comands...create/alter
    Examples: Insert/update/delete,truncate.......etc
    Create database,create procedure.......etc
    Alter database,shutdown,drop,create....etc
    Looking for list of all commands in one place/doc in SQL Server.
    Thanks

  • All the SSIS Packages are not editable to the all users after deploying to the other server

    Hi All,
    I am using sqlserver2012.
    How to make all the SSIS Packages are not editable to all users after deploying to the other server. is there any way to achieve this,
    if yes, could share the information how to do....
    Thanks in advance....
    RamarajuC

    Typically, you change the protection level as listed in the following steps:
    During development, leave the protection level of packages set to the default value,
    EncryptSensitiveWithUserKey. This setting helps ensure that only the developer sees sensitive values in the package. Or, you can consider using
    EncryptAllWithUserKey, or DontSaveSensitive.
    When it is time to deploy the packages, you have to change the protection level to one that does not depend on the developer's user key. Therefore you typically have to select
    EncryptSensitiveWithPassword, or EncryptAllWithPassword. Encrypt the packages by assigning a temporary strong password that is also known to the operations team in the production environment.
    After the packages have been deployed to the production environment, the operations team can re-encrypt the deployed packages by assigning a strong password that is known only to them. Or, they can encrypt the deployed packages by selecting
    EncryptSensitiveWithUserKey or EncryptAllWithUserKey, and using the local credentials of the account that will run the packages.
    Please Mark This As Answer or vote for Helpful Post if this helps you to solve your question/problem. http://techequation.com

  • How do I automatically backup SQL Agent jobs and SSIS packages on the mirror daily?

    I have seen this question asked before but I could not find a satisfactory answer. What is the best solution to get your SQL Agent jobs/schedules/etc. and your SSIS packages on the mirror server? Here's the details:
    Server A is the principal with 2 DBs mirrored over to server B. Everything is fine and dandy, DBs are synched and all good. In Disaster Recovery testing, we need to bring up server B, which now will serve as the principal. Server A is inaccessible. Now,
    we need all our jobs that are setup in server A to be in server B, ready to go, but disabled. We also need all our SSIS packages properly stored. Yes, we store our packages in the MSDB in server A.
    Now, I can see a few answers coming my way.
    1- Backup the MSDB to server B. When you bring server B up as principal in DR, restore the MSDB. All your jobs,schedules,steps, SSIS packages will be there. Is this possible? Can this be done on server B without rendering it incapable of serving as the principal
    for the mirrored DBs? My fear with this option is that there may be information in the MSDB itself about the mirroring state of the DBs? Or is all that in the Master DB? Have you tried this?
    2- Right click each job, script them out, re-create them on server B... No, thank you very much. :) I am looking for an AUTOMATED, DAILY backup of the jobs and SSIS packages. Not a manual process. Yes, we do change jobs and packages quite often and doing
    the job twice on two servers is not an option.
    3- Use PowerShell.. Really? Are we going back to scripting at the command prompt like that, that fast?
    Since I fear option number 3 will be the only way to go, any hints on scripts you guys have used that you like that does what I need to do?
    Any other options?
    Any help GREATLY appreciated. :-) I can be sarcastic but I am a good guy..
    Raphael
    rferreira

    I would go with option number 3. Once  you have a script simple run it....
    param([string]$serverName,[string]$outputPath) 
    function script-SQLJobs([string]$server,[string]$outputfile) 
        [reflection.assembly]::LoadWithPartialName("Microsoft.SqlServer.Smo") | Out-Null 
        $srv = New-Object Microsoft.SqlServer.Management.Smo.Server("$server") 
        $db = New-Object Microsoft.SqlServer.Management.Smo.Database 
        $scrp = New-Object Microsoft.SqlServer.Management.Smo.Scripter($srv) 
        $scrp.Options.ScriptDrops = $FALSE 
        $scrp.Options.WithDependencies = $TRUE 
        $jobs = $srv.JobServer.get_Jobs() 
        $jobs=$jobs | Where-Object {$_.Name -notlike "sys*"}     
        foreach($job in $jobs) 
            $script=$job.Script() 
            $script >> $outputfile 
            "GO" >> $outputfile 
    ---script-SQLJobs "SQLSRV12" "C:\Jobs\test.txt" 
    Best Regards,Uri Dimant SQL Server MVP,
    http://sqlblog.com/blogs/uri_dimant/
    MS SQL optimization: MS SQL Development and Optimization
    MS SQL Blog:
    Large scale of database and data cleansing
    Remote DBA Services:
    Improves MS SQL Database Performance

  • How to delete entire data from MDS UI using SQL query or Using SSIS

    Hi All,
    Using SSIS i loaded data in MDS UI(Subcription view),I want when i execute my package next time and load data in MDS it should be blank or data should not be in UI.

    Hi
    please checked following url to insert data  in your MDM
    http://msdn.microsoft.com/en-us/library/jj819772.aspx

  • Query to display all records count of all tables in a schema

    Hi,
    I need to count all the records of all my tables in a certain schema and display the max amount of the record count. Do you have a script for this one??

    SQL> create function countrec(in_tab in varchar2) return number is
      2  retval number;
      3  begin
      4    execute immediate 'select count(*) from '||in_tab into retval;
      5    return retval;
      6  end;
      7  /
    Function created.
    SQL> select table_name, countrec(table_name)
      2  from tabs;
    TABLE_NAME                     COUNTREC(TABLE_NAME)
    QUERY_TOOL_DATA_COLLECTION                        5
    TEST01                                            0
    T1                                               14
    EMP                                              14
    SALGRADE                                          5
    FILES                                             0
    PROVA                                             0
    TEST                                              0
    T_MASTER                                          1
    T_CHILD                                           3
    TAB_ONE                                          30
    TAB_TWO                                          10
    A                                                 3
    B                                                 4
    C                                                 3
    D                                                 3
    BONUS                                             0
    DEPT                                              4
    18 rows selected.Max

  • Can I show a color bar instead of a color bullet in iCal Monthly view for all my events in all calendars?

    In the Monthly view of iCal the only events that show a color bar in the event is the Birthday Calendar. All other events in all my other calendars only show a color bullet next to the event (unless I click on that event which then shows as a color bar). I would like to know if it is possible for all the calendar events to have a color bar in the monthly view instead of just that tiny color bullet.

    Greetings Judith,
    Before making any attempts at deleting calendar data, backup what you have just in case:
    Click on each calendar on the left hand side of iCal one at a time highlighting it's name and then going to File Export > Export and saving the resulting calendar file to a logical location for safekeeping.
    iCal has an automated function located in iCal > Preferences > Advanced > Delete events "X" days after they have passed.  By typing in a value for days you can tell iCal to delete all events before that time frame.
    Example:
    Today is 4-16-2012.
    If I wanted to delete all events prior to 1 year ago (4-16-2011) I would type in "365" for the number of days.
    Once you type in the number of days you want kept in iCal, close the preferences and then quit iCal.
    Re-open iCal and check to see if the events are gone.  If not you may want to leave it open for several minutes and then quit again.
    Once the events are removed go back to  iCal > Preferences > Advanced > Delete events "X" days after they have passed and make sure the check mark is removed to prevent future deletion.
    Hope that helps.

  • I dowloaded ios5 for my iphone4, and when i finished the backup i lost all the music and all the videos i bought from the iTunes store. How can i get it back?

    I dowloaded ios5 for my iphone4, and when i finished the backup i lost all the music and all the videos i bought from the iTunes store. How can i get it back?

    Backup do not contaiuns music or apps, you will either need to resync them from your itunes computer, or you can go to purchase history in itunes or app store app and go to purchase history and redownload them.

  • Nano3rdgen Copying playlist from itunes to ipod and it says it can't copy certain songs to ipod cos of file type.These problem songs all from same album, all ACC file type and all copied fine when I copied the album itself few weeks ago-help!

    When copying playlist from itunes to nano3rd message comes saying 9 of songs can't be copied to ipod cos of file type. These 9 are all from same album, all ACC and all copied fine when I copied the album itselm some weeks ago.  Rest of playlist was ACC too.  P.S. album is a CD-R copied version-is this an issue? HELP I AM DESPAIRING!

    I never laughed in all my life. Here I was, inserting the cd Dance with Me, thinking it's not going to copy the whole thing. yeah, right.
    Sure, it prompted me, Do I want to replace the songs, so of course I said "no, don't replace the songs". and guess what it did. It copied the whole cd (at least I got my two songs).
    I went into the itunes library, and sure enough, all the songs were there twice. So what did my intelligent brain think of? I just unchecked the doubled songs and stuck my ipod into my computer and it synched up and I got what I wanted.
    Now what did I do wrong when it prompted me. I clicked on "don't replace" and it replaced them anyway.
    Am I dumb? or is itunes stupid?
    thanks, Melody

  • My songs have all been duplicated.  All my music is on external hard drive and I want it to stay there.  Am using a new computer and thought I'd done all the required things for my library to show on iTunes 11.

    I'm using a new computer.  All my music is on an external hard drive and I want it to stay there.  Before installing iTunes on the new computer, I followed turingtest2's instructions as to the layout of all the iTunes folders (Itunes folder, Itunes Media, Music, etc...).  I also followed his instructions as to get all my music in iTunes on the new computer (but keeping all the media on the external drive).  Now my songs are all duplicated in my library (I normally have about 15,000 of them). They all play when I click onto them.  Also, I've lost all my playlists, and all the songs in the library are now ticked. I've already spent hours trying to sort this out, but can't seem to find a solution.  Help please!

    Hi turingtest2,
    Thank you for replying.  I've been looking at your posts about setting up iTunes using an external drive.  A bit of background info.  My old computer broke down, so I got a new one from work.  I've always kept my music files on my external drive (I have around 170GB of music).  I didn't follow the suggestions in the "migrate iTunes library" post because my old computer died on me.  However, I've looked at your posts about lay out of the iTunes folders.  Before connecting to iTunes on the new computer, I had reorganized my iTunes folder on my E: external drive, and created an iTunes folder which contains an iTunes Media subfolder.  The music files are in a Music folder, which is in the iTunes Media folder.  I've also put the following files in my iTunes folder: iTunes Library.itl, iTunes Library Extras.itdb, iTunes Library Genius.itdb, iTunes Music Library.xml, as well as sentinel.  I then followed the advice you gave bmw99 on 2 Nov 2013 at 9.58 (sorry, can't copy the exact answer). 
    Just to clarify:  I'd like to keep all my music media on the E: external hard drive (the computer hard drive is too small), and I'm aware i might never have my playlists back, and that all the songs in my library might be ticked.  However, I don't want my library to be duplicated. 
    I'm at work at the moment, but will be back home in about 10 hours, so I can be more specific about folders / files names. 
    I'm glad you picked up my message and greatly appreciate your help.
    Patrice

  • I have just purchase Apple TV and uploaded my itunes from my pc to icloud, this has been shared to my ipad and i now have a little cloud with a red arrow next to all my tunes, so all would seem well, but on the apple tv nothing in music ? is it me?

    I have just purchased Apple TV and an ipad, i have uploaded my itunes to icloud from my pc and activated sharing, i now have all my tunes on the ipad and pc and all have a little cloud with a red arrow next to them so all seems well, but when i go on Apple TV, and i'm all signed in and all boxes checked, i click on music, my music, and all files are empty, should i not see my itunes and be able to play them from my apple TV, or have i misunderstood what the cloud is for.?
    Any answers please. Apple TV 3rd Gen,  Ipad Air.

    Welcome to the Apple Community.
    You can't play music from the computer if there is none there. If you are using iTunes Match you should now be able to download and play all your music on the Apple TV from the music feature on the main page.

  • HT204053 my 2 sons have iPods and I have them connected to my apple id but everytime I send a text or they download an app, we all get it on all 3 devices.  How do I separate their stuff from mine?

    I got my 2 sons ipod touches for christmas and used my apple Id thinking I could have more control over what they download, but what happens is that everytime I do something on my iphone or they do something on thier ipods, we all get it on all 3 devices.  That includes my text messages and the many apps that they download.  How do I separate thier activity and mine while still having the ability to control what they do on thier ipods???

    For Messages they will each need their own iCloud accounts. You should also enable Restrictions in Settings > General > Restrictions and disable auto-download on the other devices.

  • My form has a list of yes or no questions.  I want to use either cheboxes or radio buttons.  I can't make it work.  When I check yes for one question, it marks all yes answers for all other questions.  If I mark some radio buttons to answer, it unmarks la

    My form has a list of yes or no questions.  I want to use either cheboxes or radio buttons.  Unfortunatly I can't make it work.  When I check yes for one question, it marks all yes answers for all other questions.  If I mark some radio buttons to answer, it unmarks later.  Need immediate help!!!

    Is it that you are using a PDF-based form? Did you copy and paste the Yes/No fields all the way down your form? If so, then all of the yes/no boxes are copies of each other and have the same name and properties in the PDF.
    For example, if you had:
         Question 1 and Question 1 Yes/No checkboxes
         Question 2 and Question 1 Yes/No checkboxes
         Question 3 and Question 1 Yes/No checkboxes
         Question 4 and Question 1 Yes/No checkboxes
    Then whatever answer you selected in Question 1 would populate down through the rest of the form.
    Changing the Yes/No field properties in questions two through four would eliminate the problem.
    I hope that helps,
    Brian

Maybe you are looking for