SSIS write custom logging information to sql 2012 table

I have a Package that I want to put logging information in to a SQL table.
I can do this with a script task but each time the package runs it will open a SQL connection write the log information and then close this connection it will do this 10 times every time the package executes and i could 50 of the packages running at the
same time so i am looking for a better way to do this.
My thought was to create a in memory table object at the start of the package and then insert the log records in to it and at the end do a bulk insert at the end of the package.
My problem is I don't know if this can be done and if it can how would I create the in memory table insert the records and then bulk load it to the SQL server table.

While I do not see any justification for what you want, "My thought was to create a in memory table object at the start of the package and then insert the log records in to it and at the end do a bulk insert at the end of the package" is doable
by creating an ADO table, adding records to it and then writing to a destination:
https://support.microsoft.com/en-us/kb/195082?wa=wsignin1.0 has all the needed code it seems. Just wrap into the Script Task.
Arthur
MyBlog
Twitter

Similar Messages

  • Issue with SSIS Custom Components - 64 bit SQL 2012

    Hello All,
    Our SSIS packages built on SQL 2008 R2 make use of some custom components. These custom components are installed as part of an MSI. The dll's are copied over to the Windows/assemble [GAC Folder] and also to the Program Files(x86)/SQL Server/110/DTS/* folder.
    The installation does not copy the dll's to the 64 bit program files folder.
    X:\Program Files\Microsoft SQL Server\110\DTS
    These packages are executed via SQL Agent jobs on a 64 bit SQL server and there does not seem be any issue.
    Now we are upgrading our servers to SQL 2012 and we have a new installer for the custom components as well. The new custom components use .NET Framework 4.0 and when installed the dll files get copied over to the Ms.NET 32 bit runtime GAC folder and also
    to the SQL Server DTS Folder in x86. The upgraded packages work only when we set the runtime mode to 32 bit. The packages successfully executes within the 32 bit dtexec utility, but when we try to run the same package using a 64 bit dtexec utility the
    process errors out with a component failed to load message. The package moves data between two SQL Server instances.
    The custom components have always been built for 32 bit runtime. I can run a older package through the dtexec utility (from the 64 bit folder in program files) and it does work without any issues. After the upgrade the package will only execute on
    a 32 bit utility. can someone help me understand this issue?
    Regards, Dinesh

    Thank you Arthur.
    i think we got the answer as well, as the .NET framework 3.0 installer copied the files over to the C:\windows assembly the dtexec utility [32 bit/64 bit] was able to load the components.
    Now with the new installer the files are copied to specific runtime gac folders as Arthur has mentioned. The 64  bit  utility does not find the dlls in the GAC whereas the 32 bit version will find them.
    Regards, Dinesh

  • How to configure log files in SQL 2012?

    I'm installing a SharePoint Solution and using Microsoft SQL 2012 and I have limited knowledge in installing SQL.  I simply run the wizard and create the DB for SharePoint. 
    Is there any links/materials available demonstrating how to correctly configure the log files step by step in a specific partition during the SQL installation for SharePoint 2013?
    thanks, 

    Hello,
    SQL Server database log files benefit from RAID 1 and RAID 10 configurations. RAID 10 is recommended for both data files and log files.
    To control de grow of transaction log files, please backup them regularly. The following article may explain you in detail why:
    http://technet.microsoft.com/en-us/library/ms175495.aspx
    Monitor the growth of log files using Performance Monitor and the SQLServer:Databases --> Log Growths counter. Adjust the size of log files until Log Growths
    is constantly zero.
    Hope this helps.
    Regards,
    Alberto Morillo
    SQLCoffee.com

  • Can I write custom databse Trigger over a apps table.

    Hi All,
    Kindly explain if I can write custom database triggers over any apps defined table (for example per_jobs). Also let me know if any recommendations or best coding standards.
    Thanks & Regards
    Purushotham.M

    Hi Purushotham,
    Oracle says "We strongly advise against the use of database triggers." - Oracle Applications Developer’s Guide 11i.
    Also, setting :new values of columns (apart from maybe Descriptive Flexfield Columns) is going to be "behind Oracle Applications back" meaning you'll potentially be breaking the business rules built into Apps.
    However, that doesn't mean you can't use them - just be it on your head!
    I'm not aware of any table you can't use them on, but have seen cautions about putting them on a couple of FND tables, unfortunately I can't find a reference to that, but one to watch out for was fnd_concurrent_requests.
    If you are able to, use Personalization or Extension to achieve your requirements. If that isn't possible, then the usual coding standards apply with respect to naming standards, etc (see the developers guides).
    If your logic is not critical to the business process (or concurrency can be handled by mechanisms other than row locking), e.g. you just want to kick off some other process based on a trigger point, consider using an autonomous transaction in your trigger logic.
    Gareth

  • SSIS Service and SSISDB Catalog in SQL 2012

    I have currently production Server cluster (DB Engine / SQL Agent) with 2 nodes (active/active). It has SSIS installed on it.
    Also another server built in a VM environment dedicated to SSISDB with SSIS installed to move away all the ETL processing/load from the production server.
    We are facing some slowness in executing SSIS and looking at moving just the SSISDB from the VM machine to the production cluster. 
    Here are my questions and concerns
    1. What is the interaction between SSISDB and SSIS Service in the case of  package executions through the SSISDB Catalog ? Can I have just SSISDB and not have the SSIS service running to execute these packages?
    2. Can I scale out the SSIS with SSIS Service running on one host and the SSISDB in another host?
    Any input would be appreciated.
    Thanks

    Hi GiriATL,
    Question 2: No. We cannot create SSISDB catalog database without installing SSIS service. For more information, you can refer to the following blog:
    http://blogs.msdn.com/b/pfebi/archive/2013/07/25/can-i-use-ssdb-without-installing-ssis.aspx 
    Question 1: As said above, the SSISDB catalog database cannot be created or work without installing SSIS. To execute a package, both DTExec utility and SSIS runtime are required, and both of them are installed by SSIS service. However, during
    the execution of a package, the SSIS service is not required to be running. The SSIS service is a Windows service that monitors the SSIS packages that are running, and the SSIS service also manages the storage of SSIS packages. For more information, please
    see:
    http://support.microsoft.com/kb/942176 
    Saying you have to install the service is not 100% correct. You have to install Integration Services, which installs DTEXEC, the runtime and the SSIS service. The service itself is a byproduct necessary to support the package deployment model. You can disable
    the service without any issue.
    MCSE SQL Server 2012 - Please mark posts as answered where appropriate.

  • How to write log information into SM37 batch job log

    Hi,
    I have a report running in batch mode, and I would like to log the start time and end time for some part of the code (different Function modules). I need to write this log information into the batch job log. Therefore I can check the time frame of my FMs.
    After search the SDN, I can only get some information on how to write log into the application log displayed in SLG1, but that's not I want. I want to write batch log information, and check it in SM37.
    If you have some solution or code to share, please. Thanks a lot.
    Best Regards,
    Ben

    Hi Nitin
    Thanks for the reply. Could you explain it with some code ?
    I tried to use the write statement , but it did not wrok. I could not see the result in SM37.
    write : "start of the FM1 processing".
    FM1 code
    write : "end of the FM1 processing".
    but those two statement did not show in SM37..
    1) how to use  a information message  ?
    2) how to use NEW PAGE PRINT ON and PRINT OFF command. ?
    I would appreciate if you can write some code ,that I can use directly.
    Thanks a lot.
    Best Regards,
    Ben

  • How to write custom messages to Data Manager package log from within SSIS?

    Just wonder in BPC 7.0/7.5 MS how do you write custom messages to Data Manager package log from within SSIS?  I mean I want to log output of specific custom tasks (non-BPC) in SSIS control flow to the resultant BPC package log created when running SSIS package in Data Manager.  Can this be done in a Script Task or some other way?  Not much documentation out there on this.
    Thanks,
    Brian

    Hi Brian
    In order to achieve what you want, it can get tricky in your package, ultimately It would depend on the task, for example if you had an execute SQL task, you could use a RAISEERROR command in your SQL statement and BPC would return the error message that you specified.
    If you had other tasks, then it would be a bit more tricky, you would have to have custom messages based on event handlers.
    Please see below links for examples :
    [Custom messages for logging|http://msdn.microsoft.com/en-us/library/ms345174.aspx]
    [Custom Logging Using Event Handlers |http://consultingblogs.emc.com/jamiethomson/archive/2005/06/11/SSIS_3A00_-Custom-Logging-Using-Event-Handlers.aspx]
    Hope this helps
    Kind Regards
    Daniel

  • Failed to install ms sql 2012, it shows 1406 Could not write value to key \SOFTWARE. Verify that you have sufficient access to that key, or contact your support personnel. error

    Hi friends,
    i have tried to install ms sql 2012 on windows 8.1 pro 64 bit  but it got failed due to the following error 1406 Could not write value  to key \SOFTWARE.   Verify that you have sufficient access to that key, or contact your support personnel.. 
     I had run the set up file as administrator. I have ony one user account on my laptop. I have seen in the regestry about permission of user. There is full control to the user. I have also tried the solution related to this issue that are available in
    this forum and other.
    I have also attache the detail.txt log file here
    So i am heartily requesting you guys if you have the solution then please post your solution as soon as possible. 
    Thanks in  advance!!!

    Looks like this is a permission issue with account you logged in in your operating system or account if any you chooses to run you SQL Server installation . Or for executing setup file try using run as administrator.
    http://www.sperrysoftware.com/kb/questions/7/How+to+Troubleshoot+a+%22Error+1406%3A+Could+not+write+value...%22+or+%22Error+1406%3A+Could+not+write+InstanceIndex+to+key...%22+error+message
    http://support.microsoft.com/kb/913559
    Raju Rasagounder Sr MSSQL DBA

  • Trying to install SQL 2012 Std on Win 8.1Pro - Fails right away with no log

    When trying to install MSDN SQL 2012 Std 64 or x86 on Windows Pro 8.1, i get an error immediately with no information posted to the summary log. Screen shot of error. Just Modal dialog with "SQL Server Setup" and a Big red X in it.
    Any help would be appreciated.
    -jeff

    Hello,
    Please enable .NET Framework 3.5 using Control Panel -> Programs and Features. If you have it already enabled, please repair it using
    the following tool:
    http://www.microsoft.com/en-us/download/details.aspx?id=30135
    Open the Command Prompt as Administrator and run the following command:
    winmgmt /verifyrepository
    The result of the command should be “WMI Repository is consistent”.
    SQL Server cannot be installed on a compressed or encrypted drive.
    You need to run SQL Server setup with a Windows account with administrative privileges.
    Hope this helps.
    Regards,
    Alberto Morillo
    SQLCoffee.com

  • SCOM2012 - SQL 2012 DB Log File Discovery isssue

    Dear Experts,
    I have some SQL 2012 servers, that has few log files (.LDF) stored in a specific drive. SCOM discovers these files but, has a wrong value in 'Display Name'.
    For example. The log file name on the server is PROD and the file path c:\SQL\PROD.LDF, but in the console it shows name as UAT and file path as c:\SQL\PROD.LDF (file path is correct as expected). It always stays in critical state stating that the log file
    is out of space while it is not the case.
    We even have tried wiping the agent off the server and reinstalling it. But it did not fix the issue. When I remove the agent 'Operations Manager' under event log disappears, but when I reinstall the agent after few days, I see the log created,
    but with older events too dated well prior to the uninstallation.
    And the other thing is, we had this issue while using 2007 R2 and even after switching over to 2012 it continues.
    SCOM 2012 was a fresh setup and was not an upgrade.
    Hope someone could help me out with this.
    Regards,
    Saravanan

    Hi Niki,
    Sorry for the delay in reply.
    I hope the image can explain better. I have that log file on a SQL server, which is being discovered with a wrong File name but with the correct path. The actual file name what I see on the server is exactly the same as it shows in the File path in console.
    But 'File Name' in the console is completely irrelevant. Also, this log file is in critical state for log file full, which is statistically false. There are few other log files on this server, for which we do not have this issue.
    Please let me know if any other information would be required
    Regards,
    Saravanan

  • Is it possible to Deploy a SSIS package to catalog with a Sql Server Identity in SQL Server 2012 programmly ?

    Hi,
    how can I deploy a SSIS package with a SQL SERVER identity to the catalog in SQL Server 2012 programmly?
    I tried to use the [SSISDB].[catalog].[deploy_project], but the error said that it could only be used with a WINDOWS identity.
    This question has worried me for days.
    Is there any way to make the catalog.deploy_project available for a SQL SERVER identity ?
    Or just an new solution instead of using the catalog.deploy_project?
    best wishes from Qingyuan Lee.

    Sorry, my description may be confusable.
    I use EXECUTE AS , not EXECUTE WITH, to solve the problem.
    In fact,  I don't care about the authority.
    I want to execute the stored procedure using a SQL SERVER Authentication Account , since my codes run on a linux OS and it is hard to connect to SSIS using a Windows Authentication Account.
    so, using EXECUTE AS to change the execution context is accaptable.
    The requirement for a Windows Authentication Account makes sense.
    I consider that the SSIS executes outside the context of SQL Server and as part of an OS process, which means that a Windows account is necessary in order to deploy a package.
    Thanks very much for your help.

  • SQL 2012 SSIS package runs from the command line (dtexec.exe) and completes right away ...

    Hi
    I’m upgrading our SSIS packages from SQL 2005 to SQL 2012 .
    Everything is working fine in Visual Studio, but when I’m submitting dtexec.exe it’s finishing right away in the command line (the actual execution takes long time). 
    It looks to me that as the return code doesn’t pass properly.
    As I have depending tasks how I can make sure all jobs will be executed in the proper order.
    (We never had this issue in SQL 2005)
    C:\Program Files (x86)\Microsoft SQL Server\110\DTS\Binn>dtexec.exe /ISSERVER "\"\SSISDB\Direct_Prod\Direct_SSIS_Package
    \DD_Load_Customer.dtsx\"" /SERVER TORSQLSIS01 /ENVREFERENCE 2
    Microsoft (R) SQL Server Execute Package Utility
    Version 11.0.2100.60 for 32-bit
    Copyright (C) Microsoft Corporation. All rights reserved.
    Started:  10:21:55 AM
    Execution ID: 21138.
    To view the details for the execution, right-click on the Integration Services Catalog, and open the [All Executions] report
    Started:  10:21:55 AM
    Finished: 10:21:56 AM
    Elapsed:  0.766 seconds

    As per MSDN /ENVREFERENCE argument is used only by SQL Server Agent
    see
    https://msdn.microsoft.com/en-us/library/hh231187.aspx
    below part is what it says
    /Env[Reference] environment reference ID
    (Optional). Specifies the environment reference (ID) that is used by the package execution, for a package that is deployed to the Integration Services server. The parameters configured to bind
    to variables will use the values of the variables that are contained in the environment.
    You use /Env[Reference] option together with the /ISServer and the /Server options.
    This parameter is used by SQL Server Agent.
    Please Mark This As Answer if it solved your issue
    Please Vote This As Helpful if it helps to solve your issue
    Visakh
    My Wiki User Page
    My MSDN Page
    My Personal Blog
    My Facebook Page

  • SSIS failed to execute in SSDT-BI for Visual studio 2012 after SQL 2012 SP2 upgrade

    Hello,
    I have applied sql server 2012 sp2 on my dev workstation and I am currently having this error message when trying to execute a SSIS package from SSDT-BI for visual studio 2012 : "Method
    not found: 'Boolean Microsoft.SqlServer.Dts.Design.VisualStudio2012Utils.IsVisualStudio2012ProInstalled()'. (Microsoft.DataTransformationServices.VsIntegration)"
    I found this link on SSIS forum :
    IsVisualStudio2012ProInstalled() method not found error when running an SSIS package from VS2012
    I don't have neither Gacutil.exe nor Visual studio professional installed so it can't be fixed this way... and anyway, this means we will use the old assembly version of ssis...
    sql server 2012 sp2 setup doesn't apply to SSDT-BI 2012 ?
    what can we do ?
    thanks,
    Stéphane

    Hi Stephane Guilleminot,
    According to your description, you do not execute a SSIS package from SSDT-BI for visual studio 2012 after upgrading to SQL Server 2012 SP2. The workaround is still necessary to apply SP2 and CU1 for SP2 (11.0.5532), then you still have to replace Microsoft.SqlServer.Dts.Design.dll
    (11.0.5532.0) with the last working version: 11.0.3412.0 from SP1 CU9.
    There is similar issue about executing SSIS package after upgrading SQL Server 2012, you can review the following similar post.
    http://social.msdn.microsoft.com/Forums/sqlserver/en-US/53606f1b-38b0-48d3-a526-12cac5258d0b/vs2012-reports-failed-to-start-project-when-executing-ssis-package-after-applying-cu10-for-sql?forum=sqltools
    Regards,
    Sofiya Li
    If you have any feedback on our support, please click
    here.
    Sofiya Li
    TechNet Community Support

  • Executing a SQL 2012 SSIS project with DTExec using Protection Level "EncryptAllWithPassword"

    I'm trying to execute a very basic SQL 2012 Project that only has 1 single package that contains a simply email task.
    The project will run fine from command line using basic syntax.
    dtexec /project  "F:\SSIS\Projects\project1\bin\Project 1 Configuration\project1.ispac" /package "Main.dtsx"
    Once I add the protection level of EncryptAllWithPassword to the project and the package things suddenly stop working. New command line syntax is.
    dtexec /project  "F:\SSIS\Projects\project1\bin\Project 1 Configuration\project1.ispac" /package "Main.dtsx" /P "Password"
    I get the following output.
    Microsoft (R) SQL Server Execute Package Utility
    Version 11.0.2100.60 for 64-bit
    Copyright (C) Microsoft Corporation. All rights reserved.
    Started:  5:14:49 PM
    Could not load project "F:\SSIS\Projects\project1\bin\Project 1 Configuration\project1.ispac" because of error 0x80131500.
    Description: The encrypted data in project manifest failed to load. The project
    manifest is corrupted or if the project is password protected, the specified password is incorrect.
    Source: Microsoft.SqlServer.ManagedDTS
    Started:  5:14:49 PM
    Finished: 5:14:49 PM
    Elapsed:  0.078 seconds
    I'm confident my passwords are entered correctly in the two spots. Is there something I'm missing in my command line to use protection levels correctly. I have also built the project and deployed to SSISDB.
    Thank you

    Add the PassWord attribute as  :
    dtexec /project  "F:\SSIS\Projects\project1\bin\Project
    1 Configuration\project1.ispac" /package "Main.dtsx" /P "YourPassWord"
    For more info :
    http://technet.microsoft.com/en-us/library/ms138023(v=sql.105).aspx

  • SQL 2012 Transactional Replication Publisher is also AG Primary, AG fine, but Replication status continuously 'waiting for next Log backup'

    Hi,
    I have a SQL 2012 instance that hosts 3 databases that are publishers in (push) transactional replication to subscribers in other domains and are also primary in an Availability Group to other instances in the same domain.  The AG is healty (synchronized),
    and replication is ok for 2 databases, but the third continuously shows the Publisher to Distributor history status as 'Replicated transactions are waiting for next Log backup or for mirroring to catch up.'
    I have weekly full, nightly differential, and hourly log.  Those are running and I've tested restoring the publisher db from them into a test instance.  The option for sync_with_backup is 0.  I generated a new snapshot, no change (haven't
    reinitialized, that will be last possible option as this is production).  When the log backups run, status will briefly change to 'Approximately (x) log records have been scanned in pass # ..., 0 of which were marked for replication' and some data
    does replicate, but then status immediately changes back to the message about waiting for log backup.   We are only replicating tables (no views, sp's, etc).  All subscribers and AG secondaries are SQL 2012.  My db is less than 200
    GB data. 
    The jobs are continuously running and data updates are occurring on the primary.  The distribution database is on the same instance as the publishers.  Anyone have any ideas what could be happening? 
    Thanks greatly!

    Update - I resolved this. 
    We had added a new asynchronous instance to the AG, but hadn't added the databases on the new node to the AG yet.  Conceptually it seems that adding the instance must have marked the logs and interferred with transactional replication on the
    primary.  Once I added the databases to the AG on the secondary, then the replication status cleared on the primary and has remained cleared.  It is odd though that it only impacted one of the databases in the AG. 
    cheers. 

Maybe you are looking for

  • How do you determine early from late 2006 MacBooks?

    We need to replace the battery in a 2006 Macbook but don't know whether it's an early 2006 or late 2006. How do we determine this (for this purpose as well as any other purpose?)

  • Faux syncing and "other" media

    Weird problem that I've had several times. When I connect my ipod to my computer & sync it, everything appears fine on itunes. I can play all recently synced songs from the ipod on my computer even if I disconnect my external hard drive which holds m

  • Hot spots

    Being an established Gmail user, I have set my Btinternet.com Email up as client based, If and when I get a laptop/tablet,  will I be able to log into BT Hot Spots or do I need to setup a BT web mail.?

  • What permissions require to create CASES (Service Module). Creating new role.

    Hi All, Can you please suggest me what permission require to create new case from service module. Steps is below i followed to create case and assigned to user. 1- Created new blank role. 2- Went in service module. 3- Given the Create, Read, Write, A

  • Does FCP X support Sony .MTS files for import?

    I have a Sony HDR-XR160 Handycam which produces *.MTS files. Can I import these files into FCP X? Thanks,   Randy