Pipeline Delimeter issue in SSIS flatfile.

I have flat file with pipeline delimeter with 9 columns, each columns having diffrent data lenth. i am having issue with one on of the column Address.
User enter the value with PIPELINE symbol in his address, so that my developed SSIS packages got messed up bcoz my package is consider user enterted additional pipeline is one column. so my package is got failed. please help me, how to sort this issue or
i need to consider this record as bad data.

I would say you would have to consider this as bad data, and then process rest of the records as pipe is the only way one can identify the change in the columns, the way to identify bad data is,
a) Use a script task before your DFT, check each row for number of pipes expected ex: if you expect 10 columns then pipe should be 9 .
b) Compare this condition and write a new file with only valid records and process this file.
Abhinav http://bishtabhinav.wordpress.com/

Similar Messages

Connection Manager Issue in SSIS 2008 version

Hello Team,
I'm facing weird issue in SSIS 2008 version. In one SSIS package, whenever I open the connection manager and point the DEV server abd test the connection it is working. When I open again, it is throwing the below error eventhough I have access to the database.
"Test connection failed because of an error in initializing provider. Login failed for user"
It's with Windows Authentication.
Additional Details:
Package configuration is disabled.
Connection manager has expression which is gettiing the details from variables that has DEV server details.

Are the variables having static values? or is the values getting passed through a parameter?
Please Mark This As Answer if it solved your issue
Please Vote This As Helpful if it helps to solve your issue
Visakh
My Wiki User Page
My MSDN Page
My Personal Blog
My Facebook Page

Issue with SSIS Custom Components - 64 bit SQL 2012

Hello All,
Our SSIS packages built on SQL 2008 R2 make use of some custom components. These custom components are installed as part of an MSI. The dll's are copied over to the Windows/assemble [GAC Folder] and also to the Program Files(x86)/SQL Server/110/DTS/* folder.
The installation does not copy the dll's to the 64 bit program files folder.
X:\Program Files\Microsoft SQL Server\110\DTS
These packages are executed via SQL Agent jobs on a 64 bit SQL server and there does not seem be any issue.
Now we are upgrading our servers to SQL 2012 and we have a new installer for the custom components as well. The new custom components use .NET Framework 4.0 and when installed the dll files get copied over to the Ms.NET 32 bit runtime GAC folder and also
to the SQL Server DTS Folder in x86. The upgraded packages work only when we set the runtime mode to 32 bit. The packages successfully executes within the 32 bit dtexec utility, but when we try to run the same package using a 64 bit dtexec utility the
process errors out with a component failed to load message. The package moves data between two SQL Server instances.
The custom components have always been built for 32 bit runtime. I can run a older package through the dtexec utility (from the 64 bit folder in program files) and it does work without any issues. After the upgrade the package will only execute on
a 32 bit utility. can someone help me understand this issue?
Regards, Dinesh

Thank you Arthur.
i think we got the answer as well, as the .NET framework 3.0 installer copied the files over to the C:\windows assembly the dtexec utility [32 bit/64 bit] was able to load the components.
Now with the new installer the files are copied to specific runtime gac folders as Arthur has mentioned. The 64 bit utility does not find the dlls in the GAC whereas the 32 bit version will find them.
Regards, Dinesh

Unexpected Termination SSIS package - Issue with SSIS packages after upgrading to SQL Server 2014

Hello everyone.
We performed an upgrade of SQL Server 2012 installation to SQL Server 2014 (with last CU5 update). We found that all SSIS packages (even the simplest with just Execute SQL tasks) fail with Unexpected termination message. The issue reproduces with both new
packages (that are created from scratch with SSDT 2014) and old packages (migrated from SQL 2012).
Any ideas, please? We googled and found that it is a known issue that should have fixed with CU4. But we use CU5 and the issue is still here.
MCP

Hi. Yes...
- <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
- <System>
<Provider Name="SQLISServer" />
<EventID Qualifiers="0">1</EventID>
<Level>2</Level>
<Task>0</Task>
<Keywords>0x80000000000000</Keywords>
<TimeCreated SystemTime="2015-02-12T07:41:56.000000000Z" />
<EventRecordID>469950</EventRecordID>
<Channel>Application</Channel>
<Computer>abtru1a11.oneabbott.com</Computer>
<Security />
</System>
- <EventData>
<Data>The SSIS Execution Process could not write to the IS catalog: ABTRU1A11\DEV:SSISDB Error details: Cannot find the symmetric key 'MS_Enckey_Proj_Param_5', because it does not exist or you do not have permission.; at System.Data.SqlClient.SqlConnection.OnError(SqlException
exception, Boolean breakConnection, Action`1 wrapCloseInAction) at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose) at System.Data.SqlClient.TdsParser.TryRun(RunBehavior
runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady) at System.Data.SqlClient.SqlDataReader.TryConsumeMetaData() at System.Data.SqlClient.SqlDataReader.get_MetaData()
at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString) at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream,
Boolean async, Int32 timeout, Task& task, Boolean asyncWrite, SqlDataReader ds) at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method, TaskCompletionSource`1 completion,
Int32 timeout, Task& task, Boolean asyncWrite) at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method) at System.Data.SqlClient.SqlCommand.ExecuteReader(CommandBehavior
behavior, String method) at System.Data.SqlClient.SqlCommand.ExecuteReader() at Microsoft.SqlServer.IntegrationServices.Server.Shared.ExecutionParameterLoader.GetExecutionParameter() at Microsoft.SqlServer.IntegrationServices.Server.ISServerExec.ProjectOperator.StartPackage()
at Microsoft.SqlServer.IntegrationServices.Server.ISServerExec.ProjectOperator.PerformOperation()</Data>
</EventData>
</Event>
MCP

CDC Issue in SSIS Package

Hi All,
We have created SSIS package using CDC(2012). But performance is very slow. How can we improve performance in the below senario's
Scenario A:
Source DB & CDC in SSIS in the same server:
When we run the CDC package in this scenario. We are able to extract 500K records in 50seconds.
Scenario B:
Source DB is in physical server 1 and CDC with SSIS in physical server 2:
When we run the CDC package in the scenario, 10,000 record extraction takes 15min. We have Primary Key in the table from where we extracting the records.
Please reply us how to solve the delay in data extraction using CDC in the Scenario B. We are using 2012.
Kindly provide us multiple options.
Thanks in Advance...

Hi All,
We have created SSIS package using CDC(2012). But performance is very slow. How can we improve performance in the below senario's. The issue in reading transaction log
When we run the CDC package in this scenario, 10,000 record extraction takes 15min. We have Primary Key in the table from where we extracting the records.
Issue : CDC source component is taking time to read data from transaction log file . Is there any way to track this ?
Our requirement is reading changed data e.g. 100,000 records in some seconds .
Kindly provide us multiple options.
Thanks in Advance...

For Each Loop Issue In SSIS

Hi All,
I have a SSIS Package which unzips the folder and then imports the files and then moves *.zip folders into processed location. If this task is executed as a part of total package its not moving the *.zip folders into processed location, but when we execute
for each loop alone this *.zip folders are moved into processed location. I am unable to figure out the issue here.
ZippedFile variable holds fully qualified file name..
Rohan
Rohan

Can you put a OnPreExec breakpoint on the file system task (Right click on the file system task > Edit Breakpoint... > Select the first (OnPreExec breakpoint) and when the pkg pauses @ the breakpoint, chk the value of the ZippedFle and FilePathDestination
variable values in your Local window (should be available at the bottom of the SSIS window as one of the tabs and you will need to expand the variable tree and find these two variables and their values). Are you seeing the appropriate values here?

Really, Really Weird issue in SSIS and DQS...

My SSIS package is spitting out all 35 of my rows into the "invalid" bucket. In fact, only 4 of the rows should be invalid. When I used dataviewer to the output of the Data Cleansing task, I find that input columns are being wrongly selected
as error by using a totally irrelevant domain. I have checked my task and ensured that the columns use the appropriate domains. Any thoughts? TIA
Jagannathan Santhanam

Hi Jagannathan,
A number of things come to mind and I may be shooting in the dark here.
1) Was the DQS install a clean install? Further have you had any issues with your SSIS project. I have know packages to become corrupted.
2) By an irrelevant 'domain', I would presume that your incoming data into the 'data cleansing task' is coming from Master Data Services. I would suggest that you have a look to ensure that the subscription view that you are using (to pull the data) is
valid and if NOT, recreate the view AND MOST IMPORTANTLY SEE THAT THE CURRENT FLAG is set. Should you need any help with this, PLEASE let me know. I have an excellent example which I recently presented at the PASS Amsterdam Rally in November 2013.
3) Next, within the Data Cleansing task ensure that the mappings are correct.
4) I would also presume that your output from the data cleansing task is going into a conditional split. This one was a 'Gotha' to me. As I am sure that you are ware, you must think this one out carefully as the order in which you handle the output
is CRITICAL to the logic. As I mentioned, this one caught me, giving faulty 'invalids' as well.
In summary:
This said, AND if I understand correctly what you have said above, please check that your subscription view is correct and what you think that it should be. A corrupt subscription view can cause this to happen.
Sincerest regards
Steve Simon SQL Server MVP

Log In Issues with SSIS

I installed SSIS two moths after installing SQL Server 2014. I granted permissions to users per MSDN in Dcomcnfg.exe and rebooted SSMS and SSIS.
I am able to log on via my client without any issues; however, the other user cannot even see the instance via her client. The only way she can log on is to remote into server, right click administrator and it works.
She has permissions for everything other than sysadmin, which I don't want to grant to her.
Any ideas why she wouldn't even see the instance from her client when given permissions?
Thank you!

To grant access to the Integration Services service
1. Run Dcomcnfg.exe. Dcomcnfg.exe provides a user interface for modifying certain settings in the registry.
2. In the Component Services dialog, expand the Component Services > Computers > My Computer > DCOM Config node.
3. Right-click Microsoft SQL Server Integration Services 11.0, and then click Properties.
4. On the Security tab, click Edit in the Launch and Activation Permissions area.
5. Add users and assign appropriate permissions, and then click Ok.
6. Repeat steps 4 - 5 for Access Permissions.
7. Restart SQL Server Management Studio.
8. Restart the Integration Services Service.
Thanks

Issues with SSIS in named instance SQL 2014

I have a SSIS job that runs without any issues with native mode. But for named instance, I get these errors:
Any suggestions?
Failed to decrypt protected XML node
"DTS:Password" with error 0x8009000B "Key not valid for use in specified state.
You may not be authorized to access this
information. This error occurs when there is a cryptographic error. Verify that the correct key is available. End Error Error: 2015-04-22 15:20:01.67     Code: 0xC0016016
Source: xxx Description: Failed to decrypt protected XML node "OraPassword" with error 0x8009000B
"Key not valid for use in specified state.". You may not be authorized to access this information. This error occurs when there is a
cryptographic error. Verify that the correct key is available. End Error Error: 2015-04-22 15:20:02.22     Code:
0x000002BD     Source: xxx Connection manager "yyy"     Description:Oracle Home not found. End Error Error: 2015-04-22
15:20:02.22     Code: 0x0000020F    Source: mmm[96]
Description: The AcquireConnection method call to the connection manager yyy failed with error code 0x80004005. There may be error messages posted before this with more information on why the AcquireConnection method call failed. End Error
Thanks

Hi LaxUdas,
If I understand correctly, the package is developed with Protection Level set as "EncryptSensitiveWithUserKey". When you execute the package via a SQL Agent Job in a Named Instance, the account that job step using is not the user that create the
package. So it cannot be able to connect to the Oracle DB with invalid connection string.
This behavior is by design. In an Integration Services package, the following information is defined as sensitive:
The password part of a connection string.
The task-generated XML nodes that are tagged as sensitive.
Any variable that is marked as sensitive.
EncryptSensitiveWithUserKey uses a key that is based on the current user profile to encrypt only the values of sensitive properties in the package. Only the same user who uses the same profile can load the package. If a different user opens the package,
the sensitive information is replaced with blanks and the current user must provide new values for the sensitive data.
To work around this issue, please refer to the following suggestions:
Change the SSIS Package ProtectionLevel property to EncryptSensitiveWithPassword. This setting uses a password for encryption. To open the package in SSIS Designer (SSDT), the user must provide the package password.
Change the Named Instance SQL Agent Service account to the user that create the package, like your Default Instance.
Use SSIS Package configuration files to store sensitive information, and then store these configuration files in a secured folder. You can then change the ProtectionLevel property to DontSaveSensitive so that the package is not encrypted and does not try
to save secrets to the package. When you run the SSIS package, the required information is loaded from the configuration file. Make sure that the configuration files are adequately protected if they contain sensitive information.
The following two documents are for your references:
https://msdn.microsoft.com/en-us/library/ms141747.aspx
https://support.microsoft.com/en-us/kb/918760
Thanks,
Katherine Xiong
Katherine Xiong
TechNet Community Support

Issue with SSIS SystemAvailabletask

Hi BPC & SSIS Experts,
We are working on SAP BPC 5.1 MS version. We have the below requirment.
" We have to take the BPC Appset (Application Set) offline for maintenance activities and at the same time a notification has to be sent to all the users". The said functionality should be achieved using SSIS package.
The steps we followed.
1) We logged into SSIS and crated a new project
2) from the tool box we dragged the systemavailabletask to the control flow.
3) we tried editing the properties of this task
4) when we edit and provide the information as, Description, Appset, Server details as(hostname/userid) and check thesystem setting option as Unavaialable to the user and click ok then we get the below error despite of the fact that all the entries provided are correct.
"Invalid UserID or the server is not respondin properly LateGetObject Variable or With Block variable not set."
Request you to please share your experties and provide a relevent resolution.
Any help in this regard will be much appreciated.
Regards,
Mohd Mujahd Zaffar
9966235863

Hi,
The user id should be mentioned as hostname\userid and not hostname/userid. This id should have the admin access to the appset. Try this and let me know if you see issues.
Hope this helps.
Badrish

Pipelined function issue

I have a function which populates a collection, pipe the elements in the collection and throws the piped table object. Its working fine.
But when I pass the collection to this function instead of populating them in the function. Its throwing a error message stating that unable to access rows from nested table.
How to overcome this.
Thanks in advance

That's strange. When I tried your code on my system I didn't get the error. Here's what I did. Note that some object definitions were not provided so I had to replace some of your code with my own. Also, there was a typo in the definition of tabFun you provided, two "END;" lines were listed where only one was needed.
SQL> CREATE OR REPLACE TYPE myObj AS OBJECT
2 ( F1 INT,
3    F2 DATE,
4    F3 VARCHAR2(25)
5 );
6 /
Type created.
SQL> show errors
No errors.
SQL>
SQL>
SQL> CREATE OR REPLACE TYPE myTab AS TABLE OF myObj;
2 /
Type created.
SQL> show errors
No errors.
SQL>
SQL>
SQL> CREATE OR REPLACE FUNCTION tabFun(tab1 IN myTab)
2    RETURN myTab PIPELINED
3 IS
4 BEGIN
5    FOR i in 1 .. tab1.count
6    LOOP
7      PIPE ROW (myObj(tab1(i).F1,tab1(i).F2,tab1(i).F3));
8    END LOOP;
9    RETURN;
10 END;
11 /
Function created.
SQL> show errors
No errors.
SQL>
SQL>
SQL>
SQL> CREATE OR REPLACE PROCEDURE myProc
2 --(ref_cur out weakcur.cur_defn)
3    ( ref_cur out sys_refcursor )
4 AS
5    tab myTab;
6 BEGIN
7
8 select myobj( 1, sysdate, 'test' )
9 bulk collect into tab
10 from dual
11 ;
12
13 /*
14 SELECT myObj(case_id, creation_date, bo_id_cust)
15 bulk collect into tab
16 from ps_rc_case
17 where rownum < 10;
18 */
19
20 dbms_output.put_line(tab.count); --- While debugging its shows the value as 10
21
22 open ref_cur for select * from table(tabFun(tab));
23 --- Here I am getting the error "Unable to read rows from non nested table"
24
25 END;
26 /
Procedure created.
SQL> show errors
No errors.
SQL>
SQL> set serveroutput on
SQL>
SQL> variable v_ref_cursor refcursor
SQL>
SQL> execute myproc( :v_ref_cursor );
1
PL/SQL procedure successfully completed.
SQL>
SQL> print v_ref_cursor
        F1 F2        F3
         1 29-APR-06 testTry running the test case from a clean user. Perhaps some objects aren't really defined the way you expect them to be in the user you're currently testing under.

CSV Export delimeter issues

Hi,
I have column in report which contains comma,tab and various special characters. The report displays fine on dashboard but when it is exported via CSV or tab delimited option the cells values shift sideways where it encounters a comma ( in case of csv) which gives incorrect result for the rest of the columns when report is downloaded. Is there a workaround to resolve the issue at OBIEE level?
Version OBIEE 11g
Thanks.

Hi all,
Thanks for the inputs.
The issue is now resolved.
I had amended the " before and after the field in the mapping by writing a simple udf.
str = '"' + str + '"';
return str;
The csv file is now generating properly.
Since the file has to open as an excel file other options like fixed field lengths and having a different delimiter that , were not working properly.
This approach works perfectly.

Time datatype issue from SSIS

Hi All,
I have a SSIS pkg, I have
1. Sart_time (datetime datatype)
2. End_time (datetime datatype)
3. Run_time ( Time(0) datatype)
I am using the Run_time to get the difference of star & end time to calculate the time taken to complete the execution of pkg.
When I run the below query in Management Studio I am getting the desired result as below, however when ran through SSIS expression using Execute SQL Task to update this table, it is updating it to NULL, as I am having the same code in SSIS I was expecting
it give the the result.
select CAST(END_TIME - START_TIME AS TIME(0)) from Log_table
00:00:56
Regards
Neil

Hello,
Just as others post above, please try to use the DATEDIFF() function rather then minus operation. Please check the following statements:
SELECT CONVERT(time(0), DATEADD(ss, DATEDIFF(ss,Sart_time, End_time), 0))
Regards,
Fanny Liu
If you have any feedback on our support, please click here.
Fanny Liu
TechNet Community Support

ExcelDestination Format issue using SSIS

Hi Everyone,
I have one scenario, My Process is exporting some data from sql server table to excel destination with date related columns like below.
Query:
SELECT
TOP 10 [DiscountAmount]
,[ProductStandardCost]
,[TotalProductCost]
,[SalesAmount]
,[TaxAmt]
,[Freight]
,[CarrierTrackingNumber]
,[CustomerPONumber]
,[OrderDate]
,[DueDate]
,[ShipDate]
FROM [AdventureWorksDW2012].[dbo].[FactInternetSales]
Create excel Tab script:
CREATE
TABLE `Excel Destination`
`Copy of SalesAmount` MONEY,
`Copy of TaxAmt` MONEY,
`Copy of TotalProductCost`
MONEY,
`Copy of Freight` MONEY,
`Copy of ProductStandardCost`
MONEY,
`Copy of OrderDate` DATE,
`Copy of DueDate` DATE,
`Copy of ShipDate` DATE,
`Copy of DiscountAmount`
DOUBLE PRECISION,
`Copy of CustomerPONumber`
NVARCHAR(25),
`Copy of CarrierTrackingNumber`
NVARCHAR(25)
Please share your suggestions on this.
Regards,
Vaishu

Thanks Vaibhav for your help.
once we export data into excel, we expect date filed with date format only, not in string format.
Regards,
Vaishu
Hi Vaishu,
You can try below two options.
1. You can have one blank excel file as a template. In advance, use the required date filter.
Select above template excel file in Excel Destination folder. Once SSIS package loads the data in excel, user can use that filtering.
2. After the Excel Destination/Data flow task, you can have one Script Task. Write .NET code which will change that filtering. (I'm not sure if that is possible through .NET)
-Vaibhav Chaudhari

Pipeline classpath issues

I have a webflow that is working as expected, and i've just added a very simple
pipeline (1 PC class, just a regular java class).
I attached the pipeline to the webflow, compiled the class, jar'd the class and
stuck in the the WEB-INF directory for my app. Immediately, i got a ClassNotFound
exception on the PC class. So, i then put the jar in the portalApp directory with
all of the other portal jars and even added it explicitly to the server classpath.
Now, it finds my PC class but can't find PipelineComponentSupport!!!! What is
going on ?? Should i have to put this in the path manually too?

Joe,
You are correct, I blanked out, sorry. IPs are executed within the web-tier
while PCs are executed in the EJB tier. You need to either put your PC in
pipeline.jar or reference your own JAR from the pipeline.jar's classpath
entry in its manifest.
Sincerely,
Daniel Selman
"Joe B" <[email protected]> wrote in message news:[email protected]...
>
I created a jsp to test this (just imported and created an instance of theclass),
dropped it in the webapp and it found the pipeline class no problem. So,again,
does this have to do with the pipeline "executor" running in an ejbcontext? or
why would my jsp see the file but not the portal framework?
"Joe" <[email protected]> wrote:
here's a zip with the log, pln file and the jar
"Joe B" <[email protected]> wrote:
Daniel,
I've tried it exploded
i.e., WEB-INF\classes\<mypackage>\myClassPC.class
I've also tried it jar'd by itself
~\beaApps\portalApp\myjar.jar
and
WEB-INF\lib\myjar.jar
is there anything related to EJB? i've done about 8 input processors
with no problem.
i was getting rmi RemoteExceptions before i defined a
java.rmi.RemoteException
node in the webflow calling this pipeline
"Daniel Selman" <[email protected]> wrote:
Joe,
When you say you put it in WEB-INF, did you mean WEB-INF\classes or
WEB-INF\lib? Putting the class (under the correct package directory
structure under WEB-INF\classes, or the JAR (under WEB-INF\lib) should
both
work.
Make sure your JAR does not contain more than one dot (".") in its
name...
You should not put this class in system classpath as this will break
hot-deploy of your web application, and won't work (as you saw) because
other classes used by your PC are not in system classpath.
Sincerely,
Daniel Selman
"Joe Bugh" <[email protected]> wrote in message
news:[email protected]...
I have a webflow that is working as expected, and i've just added
a
very
simple
pipeline (1 PC class, just a regular java class).
I attached the pipeline to the webflow, compiled the class, jar'd
the
class and
stuck in the the WEB-INF directory for my app. Immediately, i got
a
ClassNotFound
exception on the PC class. So, i then put the jar in the portalAppdirectory with
all of the other portal jars and even added it explicitly to the
server
classpath.
Now, it finds my PC class but can't find PipelineComponentSupport!!!!What
is
going on ?? Should i have to put this in the path manually too?

Pipeline Delimeter issue in SSIS flatfile.

Similar Messages

Maybe you are looking for