Output a derived column to a flatfile destination

I'm a newbie. My source is a sql server table. Then I dropped a derived column icon. And finally a flatfile destination.
When I open the flatfile destination, it does show the new column as 'Derived Column 1' in the Available Input Columns, but it's not shown in the Available Destination Columns.
How do I add it to the Available Destination Columns?

I had to open the flat file connection manager window and manually add a new column.

Similar Messages

Flatfile destination

Hi All
I have 6 csv files in a folder with "different file names" and similar metadata and my aim is to bring all those to the same folder or a different folder in .txt format after applying a simple dervied column transformation.
I took a for each loop container for iteration of 6 csv files(created a variable "filename" for looping the files )
within the for each loop container i dragged a dataflow task and within DF task i have a flatfile source where i randomly chose one of the 6 csv files
connected the above FF Source to a dervied column transformation and applied some transformation which is fine
adn the lastthing is i connected the dervied column to a flatfile destination and i want all the 6 csv files to come into the folder or a diff folder(either is fine) with .txt format dynamically.
Please help with this.

I know how to loop files through for each loop container, i have done this several times.
BUT, i am not asking the same thing...my original query is different from the above answers.
can anyone tell me how to send multiple .csv files from source to destination ( not oldedb, but flatfile) using dataflow task in between to do some trasnformation(ex: dervied column ) and
"send each transformed file to a folder destination( not oledb)".
"NOTE : i dont want to send them to the destination folder without the transformation being applied"
(I ALREADY KNOW HOW TO USE & CONFIRGURE FOREACH LOOP CONTAINER, IM NOT NEW THAT, BUT PLEASE TRY AND UNDERSTAND MY ORIGINAL QUERY)

SSIS: Using CASE Statement Within A Derived Column Transformation Expression

The following is my Data Flow:
Ole DB Source > Copy Column > Derived Column >Ole DB Command
My OLE DB Source has the following SQL command task:
SELECT *
FROM Repair R
LEFT OUTER JOIN Vehicle V
ON R.SN = V.SN
AND R.Reg = V.Reg
LEFT OUTER JOIN Product P
ON R.PID = P.PID
This yields a column of concern for me named PartNumber, which are represented by the following 2 formats:
The following are my Copied Columns:
Input Column = PartNumber
Output Alias = Copy of PartNumber
The following are my Derived Column expressions:
Derived Column Name Derived Column Expression
Name Replace 'PartNumber' LEFT(PartNumber,FINDSTRING(PartNumber,"-",1) - 1)
Copy of Name Replace 'Copy of PartNumber' RIGHT([Copy of PartNumber],LEN([Copy of PartNumber]) - FINDSTRING([Copy of PartNumber],"-",1))
So My PartNumber Column is Replaced with ######### of type
string and the Copy of PartNumber column is replaced with
#### of type int
As I stated earlier PartNumber also is in the format of %-%-% which raises the following question:
How can I replace PartNumber Column with NULL if in format of
%-%-% and take the PartNumber and put it in new column named
SubPart while keeping all the logic within SSIS objects?
So in essence I want to do something like the following:
PartNumber = CASE
WHEN 'PartNumber' LIKE '%-%-%'
THEN ABC = PartNumber AND PartNumber IS NULL
END
I have tried the case statement in a Derived Column expression and is not working. Would I add the
CASE statement to the SQL command task in my OLE DB Source?
I hope this question is concise for you.

If it must be in SSIS, I would put a derived column stage earlier in the process to do my check for me. Then in the second derived column stage do the check against each output column (part, subpart, copy of part) to decide which way it goes
PS Not sure if it's a 2012 function but SSIS has a function called TOKEN that will allow you to pull your subparts for you rather than the left/right you are doing

Adding Rank for a Column in SSIS Derived Column

H
I want to add rank to a column in ssis in a derived column actually.
Can u please tell how to add it and the syntax for adding it in a derived column.

Thanks,
Can you please give the script for the same.
I am unable to write the script for my logic to implement in SSIS.
I will be very thankful if some one could help me with it.
Thanks in Advance.
Sam thats too generic there would be several ways to solve one problem ...
can u tell us what is ur source [i mean table view THE DATA which records tuples you suspect is having duplicates ]
on what basis u decide duplicate ? and what row qualifies to go into destination ... for eg say ur source is
select id,name, date from source_table
duplicacy could mean id and name being repeated multiple times and may be business rule is like pick up rec with latest date vale if thats the case go for source as :
select id, name,max(date) from sourcetable
group by id,name
if all the columns are repeating in your source table may be use DISTINCT ...
would be great if u give ur destination, source and business requirement ..
if you have got primary key violations and any record qualifies then may be simply at destination component set Ignore failure property and ur job is done ...
may be removing duplicate at source itself ... might solve ur problem
check out more on duplicates : http://blog.sqlauthority.com/2009/06/23/sql-server-2005-2008-delete-duplicate-rows/
i hope u got the idea ...
Hope that helps ... Kunal

Data Conversion and Derived Column issues

I have a strange issue occurring with one of my SSIS packages.
What needs to happen:
I have to read data from a table that stores a field in NCHAR(40)
Send it through a Data Cleansing SSIS Component that forcefully outputs the data at a cleansed state as DT_STR(50)
Update the same source table with cleansed data - Using an UPSERT third party tool
Of course, I can't update the tables that stores data in NCHAR(40) with data from DT_STR(50), so I'm trying to use the Data Conversion Component, the Derived Column Component or a combination of, to set the data to DT_WSTR as well as to set the correct length,
from 50 to 40.
The Data Conversion Component fails when I try to set the incoming data (DT_STR(50)) to DT_WSTR(40):
[Data Conversion [186]] Error: Data conversion failed while converting column "MD_Address1" (97) to column "_MD_Address1" (190). The conversion returned status value 2 and status text
"The value could not be converted because of a potential loss of data.".
Now I tried the same thing with the Derived Column Component, converting the data from the DQS component from (DT_STR(50)) to DT_WSTR(40) and there's the error message:
[Derived Column [196]] Error: The "Derived Column" failed because truncation occurred, and the truncation row disposition on "Derived Column.Inputs[Derived Column Input].Columns[_MD_Address1]"
specifies failure on truncation. A truncation error occurred on the specified object of the specified component.
I also tried a combination of Data Conversion (From DT_STR(50) to DT_WSTR(50)) and a Derived Column just casting to the correct size, but that didn't work either.
Every time I try to change the size, one of the components fail.
It appears that my issue has nothing to do with the data types, but the actual data length.
Why can't SSIS CAST my input from 50 to 40?
What do I need to do to cast/convert data from DT_STR(50) to DT_WSTR(40)?

Hi IgorSantos_FL,
It is the expected behavior when we try to convert DT_STR(50) (means maximum 50 characters in the value) to DT_WSTR(40) (means maximum 40 characters in the value). However, the truncation issue should not occur if you convert DT_STR(50) to
DT_WSTR(50). Could you post the error message that you received when converting DT_STR(50) to DT_WSTR(50)? It may be a different issue.
Regards,
Mike Yin
TechNet Community Support

Best way to derive a "week ending" date using the Derived Column Transformations

Hi, I have an interesting challenge. I am working on creating a BI DB that contains timesheet data. The data contains a column representing the date "worked" ([Date]. Nearly all output reporting is based on a timesheeting week that end on
a Wednesday. My thinking has been to create a derived column "WE" (week ending) that represents the entries of the preceding 6 days.
(Originally I entertained deriving this value view SQL view - however we are talking about a DB that is is a substantial DB (excess of 100M timesheet bookings) and an index on the WE field is warranted) so decided a derived WE column was best approach.
The Date field is represented from a SAP format (German long dated format) - however I cannot use the convert option ;( in the TE.
The Date field is derived via: (DT_DATE)(SUBSTRING([Date ],7,4) + "-" + SUBSTRING([Date ],4,2) + "-" + SUBSTRING([Date ],1,2))
I would welcome some recommendation on how to best derive a WE column. (The DT_DATE format is not susceptible to a /7, mod7 operation).
Thanks in advance,
/L

Try this solution :
http://stackoverflow.com/questions/1803987/how-do-i-exclude-weekend-days-in-a-sql-server-query

Problem with derived column

Hi friends,
[Reports 6i - 6.0.8.20.1 & DB 9.2.0.8 ]
I have problems in one report to rerieve the value of a column in my Query1 in a derived column.
Let's say that we have a query (Q1) that obtain the departments of a company.
And a second query that obtain certain data for each department of Q1 and (this is important) we can't (due to others requirements) use a "link" to join the querys.
So my idea was to create a formula column (FC1) in Q1 where the departement is stored... and, out of the query, create a derived column that would have the value of FC1.
Then ... in design model... I'll have 2 repeating frames... one based on Q1 and other based on Q2..
Q1 Frame is ok and shows all departments... but... Q2 repeating... only shows values for 1 department... So my derived column seems not to change when the value of FC1 changes...
Any ideas?
Thanks,
Jose

If your query1 is something like
SELECT ...
d.deptno,
FROM
...dept d
WHERE ...
and you want to do query2
SELECT ...
FROM ...
for all departments but then split the output on display by department.
I'd go with the suggestion from the other poster above:
Do your query1 as
SELECT ...Query1Fields,
Query2Fields
FROM Query1Tables, (put query2 here) q2
where q2.deptno = dept.deptno -- or whatever the conditions are
Or use
WITH q2 AS (put query2 here)
SELECT ...Query1Fields,
Query2Fields
FROM Query1Tables, q2
where q2.deptno = dept.deptno -- or whatever the conditions are
That second format should ensure that query2 is done as one whole unit.
Then in the data model drag and drop the query2fields outside the query1 group to form a new group below it. Then in the layout you have a repeating group based on the query1 group and within it a repeating group based on the new "query2" group.

Finding correlation using derived columns?

I want to use the aggregate CORR_S function which take two numerical columns and output the correlation between them. Assuming I have the table:
Employee(name, gender, salary) where I want to create two derived columns, how would I do it?
Like,
SELECT CORR_S(
SELECT salary
FROM employees
WHERE salary < 10000,
SELECT salary
FROM employees
WHERE salary > 10000
) as correlation
FROM employees
It will find the correlation between salary's less than $10,000 in one column and salary's more than $10,000 in another column. Of course, I get an error when running the query. It's no problem if I use CORR_S(salary, salary) but I want to use derived columns, is it possible? Or, will I need to create a new table view with two new columns CORR_S(lowSalary, highSalary)?
Thanks.

It may be correct sql query, but statistically ( i mean here the OP question, or at least how i interpret it) it is nonsense.
Clearly , correlation show how good distribution of one set of items corresponds to another set, so the order of item in the itemset does matter. Consider
SQL> WITH t AS (
2          SELECT 1 id1, 10 id2, 30 id3 FROM dual UNION ALL
3          SELECT 2 , 20 , 10 FROM dual UNION ALL
4          SELECT 3 , 30 , 20 FROM dual )
5 SELECT corr_s(id1,id2), corr_s(id1,id3)
6 FROM t
7 /
CORR_S(ID1,ID2) CORR_S(ID1,ID3)
              1             -.5So, it would make sense to calculate correlation between salary and hiredate, salary and department number, salary and employee name's length, whatever, but how can be correlated salary less 10k to salary more than 10k - i can't understand it. They are not in the same set, but in two different sets ( while one can have either more 10k or less 10k, but not both) and relation between these two sets is not given by OP.
Best regards
Maxim

Derived column dot showing in CSV

Hi, I have a derived column.
When I unabled the report so it downloads the data into a spreadsheet,
I see that it's showing all the column headings but not tha value.
Is there a way to fix it so it also shows the values of the derived columns in CSV?
Thanks,

Hi,
I had a similar problem. As I understand it, the csv export function re-runs the SQL statement and produces the output based on that rather than the contents of your report (which is why, for instance, you get all of the data even though your report may be paginated).
The only solution, which I had to use, was to change the SQL statement to include the derived values rather than create them on the report definition.
Regards,
Andy

"Derived Column" failed because truncation occurred !!!!

HI Friends,
I have got this Error so My package execution keep Failing so Please find my error Below.
Error Message :
Executed as user: BSSLOCAL\DB-CLUS-SQL-Server. Microsoft (R) SQL Server Execute Package Utility Version 11.0.2100.60 for 32-bit Copyright (C) Microsoft Corporation. All rights reserved. Started: 08:41:00 Error: 2014-02-03
08:41:13.92 Code: 0xC020902A Source: Upload to Sharepoint Derived Column [14] Description:The "Derived Column" failed because truncation occurred, and the truncation row disposition
on "Derived Column.Outputs[Derived Column Output].Columns[SDate]" specifies failure on truncation. A truncation error occurred on the specified object of the specified component. End Error Error: 2014-02-03 08:41:13.95 Code:
0xC0047022 Source: Upload to Sharepoint SSIS.Pipeline Description: SSIS Error Code DTS_E_PROCESSINPUTFAILED. The ProcessInput method on component "Derived Column" (14) failed with error code 0xC020902A while processing
input "Derived Column Input" (15). The identified component returned an error from the ProcessInput method. The error is specific to the component, but the error is fatal and will cause the Data Flow task to stop running. There
may be error messages posted before this with more information about the failure. End Error DTExec: The package execution returned DTSER_FAILURE (1). Started: 08:41:00 Finished: 08:41:17 Elapsed: 17.5 seconds. The
package execution failed. The step failed.
Please give me how Can I rectify This Error Thanks For Advance.

Hi Cherukuri19,
If I understand correctly, you checked the data type of the input column from the Advanced Editor for the Derived Column component, right? Based on the error message, it seems that the data length of the target column outputted by the component above the
Derived Column (may be the Source Component) is more than 10 characters. However, you just modify the Input Column data type and length from the Derived Column component, hence, the data truncation error occurs. In this condition, you can avoid this issue
by using the following steps:
Keep the original nature of the SDate Input Column, e.g. don’t set the data type of this column to [DT_STR] or set its length.
Use the following expression instead for the Derived Column:
(DT_STR,10,1252)((DT_DBDATE)SDate)
Regards,
Mike Yin
TechNet Community Support

Truncation in derived column

Hi guys, I have got a table with a column (wrongfile nvarchar (300)) and a package with a Derived Column in this way:
I I had to ignore the truncation because it always retrieved "truncation occurred..." but now if I see the table the column wrongfile is empty and I am wondering why by the way that the column in the csv file it less than 100 characters..

Change your expression to cast the variable to Unicode:
(DT_WSTR, 300) @[User::NBFile]
It appears that if you create a column using derived column it gets the length from the variable. So if it is empty when you create the component, SQL Server will create a 0 length output.
Russel Loski, MCT, MCSE Data Platform/Business Intelligence. Twitter: @sqlmovers; blog: www.sqlmovers.com

Drill down in Table or Pivot for derived column

Hi All,
I have a requirement like Top 5 products should show the product names and remaining should go to *'Others'*. I achieved it by using Rank function and case when.
ex: case when rank(product) <=5 then product else 'Others' end
Now the problem is, I am not getting the Drill option for this column. When I use product column without case when or case when on same column like
case when product = 'xyz' then 'abc' else product end. In this case drill is working.
Can anybody suggest me to get the drill on the derived columns.
Thanks in advance

A similar thread was recently talked about here:
In OBIEE charts, how do you make drill downs work on custom formulas?
With a calculated column, you need to make sure the source query and destination query have the EXACT same formula and the same name for the field. If it is not 100% identical, it may not work correctly. (I typically use the NAVIGATE function to get to the second query rather than the Drill function.)
Now in some cases, if you try to create a second drill down query with more details in it, the original formula doesn't work the way you want it to. The calculated formulas have to make sense in both source query and destination query in order to work properly. If you have a ranking of the Top 5 Products in the source query, the destination query also has to have the same ranking of Top 5 products in order to work. If you add more fields into the destination query (e.g., adding on product attributes, or # of records associated with that product name etc.), then sometimes the ranking / aggregation in the destination doesn't add up the same way. This causes the system to think that the formulas are different and prevents the drill down.
I managed to get around this by having a 3rd query in between the original summary view and the detailed data view. So the sequence would be (for example):
- Dashboard based summary query (including the calculated ranked product names formula)
- drill down into virtually identical query (add on one slight bit of difference here, for example adding on a second Product Name field ... not a calculated version, but the actual name). Create a drill down this time by clicking on the regular Product Name field (don't drill down on the custom formula)
- final detailed drill-down query
You can format this intermediary drill down query to explain to users that they need to click a second time.
It's not ideal, but it should work. Hope this helps.
Ted

Avoiding derived columns for mapping purposes only

Hi. In 2012 has there been any advancement that would alleviate us from adding a derived column simply for the purpose of mapping an existing var or param to an oledb output column?

Hi DB042189,
According to your description, what you would like to do is directly mapping a variable or parameter to an Output column of a Data Flow Task control without adding a new column through the Derived Column transform. If so, this is not supported in SSIS 2012
currently. I would suggest you submitting a wish at
https://connect.microsoft.com/sql.
Connect site is a connection point between you and Microsoft, and ultimately the larger community. Your feedback enables Microsoft to make software and services the best that they can be, and you can learn about and contribute to exciting projects.
Regards,
Mike Yin
TechNet Community Support

How to cast a column of a derived column in a view to NULL

I have a derived column in my view and this column by default is picked up as not null.
I want to cast it to null how can i do it ?
Actual column
column1 varchar(3) not null
Expected column
column1 varchar(3) null
Mudassar

Shridar I havent defined it but the derived column has been picked up as not null which is strange
CREATE VIEW test
AS
SELECT
CASE
WHEN COMPANY_SIZE IS NULL then 'a'
ELSE 'b'
END AS
column1 ----- added derived column identifier
,[Company_Size] AS [CompanySize]
FROM xyz
Mudassar

In BI Publisher Report of excel output for one column is filled with spaces

Hello,
We have issue in BI Publisher report of excel output.
In Excel output for one column few records are displaying as left alignment
and few record are displaying as right alignment.
If you are having any xml tag or any syntax for removing spaces for left side
Please help on this issue.
Thanks in Advance.
Regards,
Swaraj

Please post the details of the application release, database version and OS along with BI Publisher.
Is the issue with all reports or specific one(s) only?
Do you have the patches in the following docs applied?
Overview of Available Update Patches for Oracle BI Publisher Enterprise 10g [ID 797057.1]
Overview of Available Update Patches for Oracle BI Publisher Enterprise 11g [ID 1276869.1]
Overview of Available Patches for Oracle XML Publisher embedded in the Oracle E-Business Suite [ID 1138602.1]
BI Publisher 10g RTF Template Changes Alignment Settings for Numbered Lists [ID 1418504.1]
Thanks,
Hussein

Output a derived column to a flatfile destination

Similar Messages

Maybe you are looking for