Simultanious package execution

Hi All,
I am just starting with ODI, but have experience in data warehouse design, so spare me.
I wonder if that is going to work:
Creating package with interfaces, that will be executed using context substitution.
Will it work if SIMULTANIOUSLY I set agent to run the same package using different contexts ?
Package would be using different sources then, but target would be one common table for all executing processes.
We have fairly complex project and I am trying to help myself with further supervision and changes to ETL when necessary
Any remarks are welcome :-)

Turtle wrote:
1) reason : I have a full control on "my staging dimension table", while ODI "internal staging" is ODI dependent.
2) reason: if I use ODI as it is, I load the same data twice: to my and ODI internal staging, so perfomance argument
but perhaps my thinking is skewed by working with OWB.Haha I made the jump before 11GR1 and I've not looked back, in all honesty you could set up ODI such that the staging tables were in a seperate 'staging' schema (use the 'work' schema on your dataserver to do this) and you could go as far as dropping the C$ prefix leaving you with an identical object name as the target schema. (If it wont let you have Null in the Physical schema, Work tables prefix you could get around it with some Java find/replace tags in the Knowledge module API, But I do know what you mean, in OWB you hand cranked everything, ODI pushes a certain methodology onto you thats difficult to work with at first, my reason for skipping the C$ table is we where using ODI to get data from an AS/400 and load a 'landing' table to an Oracle DWH , using...... OWB for the ETL stuff ! since it was truncate / reload we didnt need the C$ stuff either as its a double load if there is no transformation going on.
For really complex mappings I'd take OWB all day long but with ODI 11G at least you can do temporary interfaces to get you that 'derived select' functionality that OWB does for every single joiner / splitter etc.

Similar Messages

Package execution problem with SQLSERVERAGENT

Hello,
I'm having a problem with the execution of SSIS packages: I created locally a package with SQL Server Data Tools, then I deployed it on my server in the SSISDB database. Then I created a job to execute periodically this package with SQL Server Agent. The
package works fine and does everything I want, but I always get a failed notification. In the global view of the execution I can see:
Result: "failed"
Package name: "Package.dtsx"
Task name: "Package"
Execution access path: "\Package"
Every other task returns a success and their execution access path is a subfolder of "\Package". I tried everything I found on the web, check the "Runtime 32bits" box, create a "Job Account" user to execute the package via a
"Job Account Proxy", modify the ProtectionLevel of the package to "DontSaveSensitive", give the "Job Account" user full rights on everything, nothing works.
Do you have an idea of what my problem is and how to solve it?
I must precise that all my packages do the same thing: I have a loop on all the csv files in a folder that for each file loads the contents of the file in my database and the delete the file. I observed that the error message do not appear when there is
no file in the folder when the package executes.
Thanks in advance.

Not sure what error message, is the one "est introuvable.
Cette erreur est retournée par la collection Connections lorsque l'élément de connexion spécifique est introuvable"?
So, here is in short: the ForEach Loop once in its beginning of operation "senses" the files currently present in the working folder. If a new file arrives after this stage, it will not see it.
In your case it seems that the ForEach "saw" a file at the beginning, but it was gone at the time the processing wanted it for execution.
Like many above indicated, the common practice, call
it elegant or not is to indeed move the files to a different directory (working folder). This is done to avoid collisions with some other activity in that folder.
Furthermore, you can have all the files deleted using a ForEach against the working folder as say in
http://beyondrelational.com/modules/2/blogs/88/posts/10178/ssis-delete-files-from-specified-folder-using-file-system-task-in-sql-server.aspx And there is no need to delete the working folder
itself.
The job is run in SQL Server's Agent
need to be set to run using a domain proxy account as it needs write access to the file system.
Arthur My Blog

SQL Server Agent job fails with error : The package execution returned DTSER_FAILURE (1).

Copyright (C) Microsoft Corp 1984-2005. All rights reserved. Started: 10:00:01 AM Error: 2014-08-15 10:00:07.20
Code: 0xC0047062 Source: Data Flow Task LEAN [6761]
Description: Teradata.Client.Provider.TdException: [Teradata Database] [3006] Logons are disabled.
at Teradata.Client.Provider.WpMessageManager.CheckForError()
at Teradata.Client.Provider.WpMessageManager.ProcessResponse()
at Teradata.Client.Provider.WpLogonManager.Action()
at Teradata.Client.Provider.WpSession.ManagerAction(WpMessageManager manager, UtlStopwatchWrapper watch, Int32 connectionTimeout)
at Teradata.Client.Provider.WpSession.Open(Int32 connectionTimeout, String password)
at Teradata.Client.Provider.WpSession.Open(Int32 connectionTimeout)
at Teradata.Client.Provider.Connection.Open(UtlConnectionString connectionString, UInt32 timeout)
at Teradata.Client.Provider.ConnectionPool.CreateConnection(UInt32 timeout)
at Teradata.Client.Provider.ConnectionPool.GetConnectionFromPool(Object owningObject)
at Teradata.Client.Provider.ConnectionFactory.GetConnection(Object owningObject, UtlConnectionString connStr)
at Teradata.Client.Provider.TdConnection.Open()
at Microsoft.SqlServer.Dts.Runtime.ManagedHelper.GetManagedConnection(String assemblyQualifiedName, String connStr, Object transaction)
at Microsoft.SqlServer.Dts.Runtime.Wrapper.IDTSConnectionManager100.AcquireConnection(Object pTransaction)
at Microsoft.SqlServer.Dts.Pipeline.DataReaderSourceAdapter.AcquireConnections(Object transaction)
at Microsoft.SqlServer.Dts.Pipeline.ManagedComponentHost.HostAcquireConnections(IDTSManagedComponentWrapper100 wrapper, Object transaction)
End Error Error: 2014-08-15 10:00:07.35 Code: 0xC0047017 Source: Data Flow Task SSIS.Pipeline
Description: component "LEAN" (6761) failed validation and returned error code 0x80004005. End Error Error: 2014-08-15 10:00:07.48
Code: 0xC004700C Source: Data Flow Task SSIS.Pipeline Description: One or more component failed validation.
End Error Error: 2014-08-15 10:00:07.60 Code: 0xC0024107 Source: Data Flow Task
Description: There were errors during task validation. End Error
DTExec: The package execution returned DTSER_FAILURE (1). Started: 10:00:01 AM Finished: 10:00:07 AM
Elapsed: 6.692 seconds. The package execution failed. The step failed.
SQL Server agent job fails with above error, Please let me know process tohandle it.
Thanks,
Vishal.

The error message suggests its the issue with Teradata source database to which SSIS tries to connect within the data flow task. Make sure Teradata database is available and LOGON is enabled.
Please Mark This As Answer if it helps to solve the issue Visakh ---------------------------- http://visakhm.blogspot.com/ https://www.facebook.com/VmBlogs

SSIS Package execution completes without giving any success or failure message

Hi,
We are using SSIS to load some 100k records from flat file to Oracle Destination. We are using Oracle 10g client.
But during the execution after some 5hrs or 6hr with 900k records upload we are getting the message Package execution completed. In the Execution results there is no message related to success or failure and the tasks in the Data Flow where yellow in color. What might be the problem? Any information regarding this case will be helpful for us.
Regards,
Roopa.

Usually you will have a red object somewhere in your package if there was a failure. Make sure you are looking in both the Control Flow and all the Data Flows in the package for a red object. In the execution results in the far right tab, usually you will see several warning messages somewhere in the list of steps, followed by a message that says you have exceded the number of errors for the package and that it is failing. This is not at the bottom but where the errors actually occur in the flow of messages.
I don't know if that helps, but that is about all I can add. Good luck.

How to keep track of all errors occured during package execution

Hi all,
I want to keep track of all exceptions raised in my package execution .I am following this approach .
Please correct me ,if i am wrong .
DECLARE
   v_empno_is_null      EXCEPTION;
   PRAGMA EXCEPTION_INIT (v_empno_is_null, -01400);
   v_deptno_not_found   EXCEPTION;
   PRAGMA EXCEPTION_INIT (v_deptno_not_found, -02291);
   v_sal_is_zero        EXCEPTION;
   PRAGMA EXCEPTION_INIT (v_sal_is_zero, -02290);
BEGIN
   INSERT INTO emp
               (empno, deptno, sal
        VALUES (1245, 60, 0
EXCEPTION
   WHEN v_empno_is_null
   THEN
      DBMS_OUTPUT.put_line ('Empno should not be null ');
   ---Insert into Error table to keep track of errors
   WHEN v_deptno_not_found
   THEN
      DBMS_OUTPUT.put_line ('This dept does not exist ');
   ---Insert into Error table to keep track of errors
   WHEN v_sal_is_zero
   THEN
      DBMS_OUTPUT.put_line ('Sal should not be less than zero');
   ---Insert into Error table to keep track of errors
   WHEN OTHERS
   THEN
      DBMS_OUTPUT.put_line (SQLCODE || SQLERRM);
   ---Insert into Error table to keep track of errors with oracle error message
END;Thanks,
P Prakash
Edited by: prakash on Jul 31, 2011 9:56 PM
Edited by: prakash on Jul 31, 2011 10:14 PM

Ugly. Uses all uppercase.
Do you not know the de facto naming standards in use today?
Suggest that you familiarise yourself with Guidelines for Names (MSDN) for .Net and Code Conventions for the Java Programming Language for Java.

.rdl file for ssis package execution reports.

Can I get a .rdl file for the SSIS Packages execution report which shows execution timing status and etc.. .where it stores (location) and possibility of the Stored procedure for the .rdl file

Not sure where the SQL Server is storing the .rdl file for the above mentioned reports. But if you just want to get summary of ETL execution you can use something similar to below query and create your own report in SSRS.
SELECT e.executable_id AS [ID] , CASE es.execution_result when 0 then 'Success' when 1 then 'Failure' when 2 then 'Completion' when 3 then 'Cancelled' END AS [Status] , ex.folder_name AS [FolderName] , ex.project_name AS [ProjectName] , e.package_name AS [PackageName] , CONVERT(datetime, es.start_time) AS [StartTime] , CONVERT(datetime, es.end_time) AS [EndTime] , CAST(es.execution_duration AS float)/1000 AS [Duration]FROM catalog.executables e INNER JOIN catalog.executable_statistics es ON e.executable_id = es.executable_id AND e.execution_id = es.execution_id INNER JOIN catalog.executions ex ON e.execution_id = ex.execution_id AND es.execution_id = ex.execution_idWHERE package_path = '\Package'ORDER BY StartTime DESC
More info can be found
here.

Support Packages Execution consuming too long time to get complete

Hi,
I installed the SAP Solution Manager 4.0 SR1, I have downloaded the support package stack till june 2007, and applied it. while importing I faced one error XPRA_EXECUTION, Resolved it (by refering sap note:979594) and again click on import. Now it is post 2 and the half day, the execution is still under process. Displaying status as "tp buffer prepared OK". Can anyone tell me how much time it will take to complete.

I have found these errors in tp log can any tell me the solution
WARNING: /usr/sap/trans/tmp/N110531.PRO : cant open
WARNING: : cant open
WARNING: /usr/sap/trans/tmp/N110531.PRO : cant open
WARNING: : cant open
STOP tp_getprots PRO Y 20110531061009
WARNING: /usr/sap/trans/tmp/N110531.PRO : cant open
WARNING: /usr/sap/trans/tmp/N110530.PRO : cant open
WARNING: : cant open
WARNING: : cant open
unable to open the open the files...???

Problem with the package execution

Hi Guys,
I am getting the problem while executin the package for 1 of the interface named PRE_SYNC_EX_ARMY_VEH_DEL.
I am looking in the operator as at the first session task is in the process(running mode) with "DROP FLOW TABLE" as the message while all other sessions tasks are in the waiting mode related to that interface named PRE_SYNC_EX_ARMY_VEH_DEL.
But when i see to description of last session task (ie Drop flow table) of the previous interface ie PRE_SYNC_AXLE_DELNR ,it is like *"Flow table is still locked in open transaction=>Drop flow table skipped"*
The source tables are EX_ARMY_VEH_TMP and CONTROL_DEL & target table is CONTROL_DEL
All the three tables are in the same schema named RTO_STAGE.

Not sure what error message, is the one "est introuvable.
Cette erreur est retournée par la collection Connections lorsque l'élément de connexion spécifique est introuvable"?
So, here is in short: the ForEach Loop once in its beginning of operation "senses" the files currently present in the working folder. If a new file arrives after this stage, it will not see it.
In your case it seems that the ForEach "saw" a file at the beginning, but it was gone at the time the processing wanted it for execution.
Like many above indicated, the common practice, call
it elegant or not is to indeed move the files to a different directory (working folder). This is done to avoid collisions with some other activity in that folder.
Furthermore, you can have all the files deleted using a ForEach against the working folder as say in
http://beyondrelational.com/modules/2/blogs/88/posts/10178/ssis-delete-files-from-specified-folder-using-file-system-task-in-sql-server.aspx And there is no need to delete the working folder
itself.
The job is run in SQL Server's Agent
need to be set to run using a domain proxy account as it needs write access to the file system.
Arthur My Blog

SSIS Package execution error from powershell

Hi I have written a powershell script to exec SSIS packages. I am passing variable values at runtime by reading it from a text file.
But it throws an error.
Here is my script.
#Read Data from Parameter File
$variable=Get-Content D:\Batch\Bin\ETLParameters.txt
$variable.GetType()
$dest_conn_str=$variable[1]
$mail_cc=$variable[2]
$mail_from=$variable[3]
$mail_str=$variable[4]
$mail_to=$variable[5]
$pkg_path=$variable[6]
$ps_exe_path=$variable[7]
$ps_script_cmd=$variable[8]
$ps_script_path=$variable[9]
$src_conn_str=$variable[10]
$src_sys_id=$variable[11]
#Executing SSIS packages from command prompt
$cmd = 'dtexec /f "' + $pkg_path + 'pkg_ntas_all.dtsx' + '"' + ' /SET \Package.Variables[User::src_conn_str].Properties[Value];"' + $src_conn_str + '"' + ' /SET \Package.Variables[User::dest_conn_str].Properties[Value];"'
+ $dest_conn_str + '"' + ' /SET \Package.Variables[User::src_sys_id].Properties[Value];"' + $src_sys_id + '"' + ' /SET \Package.Variables[User::pkg_file_path].Properties[Value];"' + $pkg_path + '"' +
' /SET \Package.Variables[User::mail_str].Properties[Value];"' + $mail_str + '"' + ' /SET \Package.Variables[User::mail_from].Properties[Value];"' + $mail_from + '"' + ' /SET \Package.Variables[User::mail_to].Properties[Value];"'
+ $mail_to + '"'+ ' /SET \Package.Variables[User::mail_cc].Properties[Value];"' + $mail_cc + '"' + ' /SET \Package.Variables[User::ps_exe_path].Properties[Value];"' + $ps_exe_path + '"' + ' /SET
\Package.Variables[User::ps_script_path].Properties[Value];"' + $ps_script_path + '"' + ' /SET \Package.Variables[User::ps_script_cmd].Properties[Value];"' + $ps_script_cmd + '"'
$cmd
DECLARE @returncode int
EXEC @returncode = xp_cmdshell @cmd
Types of error that I get -
If I run the $cmd directly. Which is the concatenated string of dtexec command with parameters. It doesn't do anything, not even an error and the pwoershell execution just completes.
If I use exec EXEC @returncode = xp_cmdshell @cmd    ----- Then it throws me the error that the exec is not recognized name of cmdlet.
Please help as I a noob in any kind of scripting.

I did but it just printed the entire string on the command line and assigned all the values.
PS C:\Users\dev_setup> . D:\Batch\Bin\runETL.ps1
IsPublic IsSerial Name                                     BaseType
True     True     Object[]
System.Array
dtexec /f "D:\ETL\test\test\test_ETL\pkg_all.dtsx" /SET "\Package.
Variables[User::src_conn_str].Properties[Value]";"Data Source=SOURCE1;Initial Catalog=tagtest;Provider=SAOLEDB.11;Integrated Security=SSPI;
" /SET "\Package.Variables[User::dest_conn_str].Properties[Value]";"destinationDb;User ID=FP_DEV;Password=**********;Initial Catalog=xyz;Provider=SQLOLEDB.1;Persist Security Info=True;Auto Translate=False;Packet Size=10240;" /SET " \Package.Variables[User::src_sys_id].Properties[Value]
";"100200" /SET "\Package.Variables[User::pkg_file_path].Properties[Value]";"D:\ETL\test\test\test_ETL\" /SET "\Package.Variables[User::mail_str].Properties[Value]";"SmtpServer=mailhost.test.com;UseWindowsAuthentication=False;EnableSsl=False;" /SET "\Package.Variables[User::mail_from].Properties[Value]
";"[email protected]" /SET "\Package.Variables[User::mail_to].Properties[Value]";"[email protected]" /SET "\Package.Variables[User::mail_cc].Properties[Value]";"[email protected]" /SET "\Package.Variables[User::ps_exe_path].Properties[Value]";"C:\Windows\System32\WindowsPower
Shell\v1.0\PowerShell.exe" /SET "\Package.Variables[User::ps_script_path].Properties[Value]";"D:\ETL\Powershell_Script\" /SET "\Package.Variables[User::ps_script_cmd].Properties[Value]";"create_historical_reject_file.ps1 -input_variable"
PS C:\Users\svc_sp_fp_dev_setup>
But when I just run it without passing the parameters it runs. What might be the issue.

Process Chain error during info package execution

I'm trying to load master data into 0CURTYPE using process chain.
I have created a start process and included the info package and corresponding DTP in the chain to load master data into the info object.
On scheduling the process chain I get an error during the execution of the info package
"Could not find code page for receiving system
For the logical destination Q4900CL010, you want to determine the code page in which the data is sent with RFC. However, this is not currently possibe, and the IDoc cannot yet be dispatched."
Is it relatred to some setting whihc I have missed out?
Any help on this please?
Thanks,

Hi, this is a issue related to idocs in the source....
U can contact ur BASIS team to solve this...
From their end they have to chk the unprocessed/errored idocs in BD87...
this issue belongs to source...
They have to run a report:RBDAGAIN to process the errore idoc's in the source which are in STATUS:'02'.
This is a solution for temporarilrly basis...
if this occurs regularly they have to change the settings in SM59..
refer exact note for this: 784381
rgds,

Error in package execution

Hi
iam new to odi.here i am posting my doubt @ i have 10 interfaces in a package.while executing 5th interface is failed but i want execute from failed interface
how can we do please suggest me

I just tried it again and it works perfectly on my side.
Here are the steps I followed :
- Create a package with a dummy procedure and an interface with error.
- Run the package. Dummy procedure ends successfully and the execution fails at the second tasks on the interface.
- Go to the operatar, expand Date -> Today (or all executions, or ...)
- Right click on the execution of my package
- Select restart. The package starts at the second tasks (integration) of the second step (interface) as expected.
Which version of ODI do you use ?
Are your steps executed synchronously ?

How to stop package execution from SSIS or logic

Hi all,
I need to stop the execution of a package (datamanager) when the user chooses a particular (wrong) category from the package prompt. It is a security check.
The package uses a custom dtsx chain.
I need to stop it using a SSIS task (I don't know how) or if it is simplier I need to stop just a script logic (.lgf) that is inside the package.
Thank you
(bpc ms version 10)

Only the user who ran the package can cancel it.
If you want to cancel another user you should log into Integration services server, and then stop the running package. (System admins usually do this)

Does SSIS package execution require xp_regread and xp_instance_regread execution privileges

Hi,
I'm still trying to learn what privileges are required to execute packages in the SSISDB catalogue. Our DBA requested that we REVOKE EXECUTE ON xp_instance_regread and xp_regread FROM public because it is a high vulnerability.
However, I noticed that once they were revoked I was getting the following error when I tried to execute any packages...
The EXECUTE permission was denied on the object 'xp_regread', database 'mssqlsystemresource', schema 'sys'...
Which leads to my question .. Are xp_regread and xp_instance_regread execution privileges required to be able to execute packages?
Everywhere I look seems to point that these priviliges are required which maybe a problem for me.
Thanks,
Rog

+ 1 for Katherine
Xiong answer, this seems more authentic and well documented.
Right now, I have access to production environments only , so I can't test the scenario. But I have few suggestions
IMO, even if your revoke rights sysadming or db_owner can easily access it. So, based on that I thought that you can grant rights to any specific person (may be service account) to execute this SP
USE master
GO
GRANT EXECUTE ON OBJECT::master.dbo.xp_instance_regread TOServiceAccount(or Sysadmin)
GO
You can then, probably, able to execute it if you are running SSIS/SQL environment from that login.
Disclaimer: This answer is just based on my assumption and should not be applied to production environment before testing. I would still recommend you to consider Katherine
Xiong answer.

Ssis packages execution fails while executing it by c#

Hi,
I have created one ssis package on server. Now i want to execute this package using c# code.
I have implemented following code to perform this task:
string pkgLocation = @"Package.dtsx";
Package pkg;
Application app;
DTSExecResult pkgResults;
Variables vars;
app = new Application();
pkg = app.LoadPackage(Server.MapPath(pkgLocation), null);
vars = pkg.Variables;
// vars["MonthInt"].Value = "02";
// vars["YearInt"].Value = "2014";
pkgResults = pkg.Execute();
foreach (DtsError pkgerror in pkg.Errors)
string err = "Error Source: " + pkg.Name + ": " + " Description: " + pkg.Description;
//lblErrorMsg.Visible = true;
// lblErrorMsg.Text = err;
if (pkgResults == DTSExecResult.Success)
// lblErrorMsg.Visible = true;
// lblErrorMsg.Text = "Package ran successfully in " + count;
else
But all the time it gets fail.
Its giving me following error in DTSError:
'To run a SSIS package outside of Business Intelligence Development Studio you must install Standard Edition of Integration Services or higher.'
Can you please guide me how can i resolve this issue.
Is it necessary to install SSIS standard edition on all client machine.
PLease help.
Thanks in advance!
Regards
Rajni

Hi ArthurZ,
Thanks for the quick response.
Actually i have created this package on server where everything is installed and i am executing this package from my local machine after coping this package in local machine.
SO is it mandatory to installed everything on local machine to execute the package?
can i execute package remotely?
Please help to solve the issue.
Thanks & Regards
Rajni

Use of variable in a package execution

Hi all,
I've a procedure to call in this way
exec schema.procedure('VAR').
There's a really simple sql that asks the user to input the value and have to convert is using UPPER and pass to procedure.
I've done as:
exec :myvar := upper(&INPUT)
exec schema.procedure(:myvar) ;
I get the error that bind variable is not declared and my not sure that package is executed correcly...
I've also tried to
declare
myvar varchar2(255) := upper(&INPUT)
begin
schema.procedure(:myvar) ;
end;
But is not working, I get PLS-00201 and PLS-00320 ...
Any hint?
Thanks
Ste

I've done as:
exec :myvar := upper(&INPUT)
exec schema.procedure(:myvar) ;If you take this approach then you must declare myvar. You can do it like this
VAR myvar VARCHAR2(255)
exec :myvar := upper(&INPUT)
exec schema.procedure(:myvar)
I've also tried to
declare
myvar varchar2(255) := upper(&INPUT)
begin
schema.procedure(:myvar) ;
end;
/If you want this then you need to pass the variable without the : symbol. In PL/SQL are variable are bind variable by itself. So you dont have to specify the :symbol.
declare
myvar varchar2(255) := upper(&INPUT)
begin
schema.procedure(myvar) ;
end;
/

Simultanious package execution

Similar Messages

Maybe you are looking for