SSIS function inside derived column !!
HI Friends My Source csv files has one column has below rows:
counts
1
12
13
12/10
13/13
my destination col datatype is FLOAT, which rows has numerator and denaminator. i want only numerator. I don't want '/10'.
how can i write ssis function inside the derrived column ! thanks in advance.
LEFT((DT_STR,10,1252) COUNT + (DT_STR,2,1252) "//",FINDSTRING(COUNT,"/",1)-1)
Please Mark This As Answer if it solved your issue
Please Mark This As Helpful if it helps to solve your issue
Visakh
My MSDN Page
My Personal Blog
My Facebook Page
Similar Messages
-
Can't change the connection string of SSIS package with derived columns?
We upgraded SQL server 2008 to 2012, copied and converted all SSIS packages from Visual Studio 2008 to 2010. When I opened a package in VS 2010 and tried to change the connection string, in the local connection managers, if the data source is another
SSIS package B(.dtsx file) with derived columns, I can't change the connection string of package B. When I opened the file connection manager editor for package B and tried to locate a dtsx file in another location, saved the change, reopened the project.
Package B still pointed to previous file. Other packages without derived columns work fine. Any thoughts?We are using the package deployment model and refer to other packages in the same project. If
we changed the path of package B (with derived columns) to "D:\Visual Studio 2010\xxxx", and refer it in package A, in the A's connection manager, the connection string of package B is still its previous location "D:\Visual
Studio 2008\xxxx". When we ran the package A in the SQL server agent, the data source is still
"D:\Visual
Studio 2008\xxxx", so how can I change it to "D:\Visual
Studio 2010\xxxx"? Why has the package C (without derived columns) no such problem? thanks. -
Adding Rank for a Column in SSIS Derived Column
H
I want to add rank to a column in ssis in a derived column actually.
Can u please tell how to add it and the syntax for adding it in a derived column.Thanks,
Can you please give the script for the same.
I am unable to write the script for my logic to implement in SSIS.
I will be very thankful if some one could help me with it.
Thanks in Advance.
Sam thats too generic there would be several ways to solve one problem ...
can u tell us what is ur source [i mean table view THE DATA which records tuples you suspect is having duplicates ]
on what basis u decide duplicate ? and what row qualifies to go into destination ... for eg say ur source is
select id,name, date from source_table
duplicacy could mean id and name being repeated multiple times and may be business rule is like pick up rec with latest date vale if thats the case go for source as :
select id, name,max(date) from sourcetable
group by id,name
if all the columns are repeating in your source table may be use DISTINCT ...
would be great if u give ur destination, source and business requirement ..
if you have got primary key violations and any record qualifies then may be simply at destination component set Ignore failure property and ur job is done ...
may be removing duplicate at source itself ... might solve ur problem
check out more on duplicates : http://blog.sqlauthority.com/2009/06/23/sql-server-2005-2008-delete-duplicate-rows/
i hope u got the idea ...
Hope that helps ... Kunal -
Hi Friends,
I am looking import CSV file data into SQl Table using SSIS Package. In that file one of the column has No_of cunts column.
some of the rows has integer value, some of them are : 12/13 like below
No_Of_Counts
12
11
10/12
09/0
10
My requirment is I want onlu numerator value where rows has '/' , how can i wright function in derived column can you please help me.Above Expression from Vaibhav works only SSIS 2012 or above,,
if you trying in 2008 you have to replace left with substring expression
Thanks
Thats true
LEFT was a new addition in SSIS 2012
the equivalent SUBSTRING expression would be
SUBSTRING(Count,1,(FINDSTRING(Count,"/",1) == 0 ? LEN(Count) : FINDSTRING(Count,"/",1) - 1))
Please Mark This As Answer if it solved your issue
Please Mark This As Helpful if it helps to solve your issue
Visakh
My MSDN Page
My Personal Blog
My Facebook Page -
SSIS: Using CASE Statement Within A Derived Column Transformation Expression
The following is my Data Flow:
Ole DB Source > Copy Column > Derived Column >Ole DB Command
My OLE DB Source has the following SQL command task:
SELECT *
FROM Repair R
LEFT OUTER JOIN Vehicle V
ON R.SN = V.SN
AND R.Reg = V.Reg
LEFT OUTER JOIN Product P
ON R.PID = P.PID
This yields a column of concern for me named PartNumber, which are represented by the following 2 formats:
The following are my Copied Columns:
Input Column = PartNumber
Output Alias = Copy of PartNumber
The following are my Derived Column expressions:
Derived Column Name Derived Column Expression
Name Replace 'PartNumber' LEFT(PartNumber,FINDSTRING(PartNumber,"-",1) - 1)
Copy of Name Replace 'Copy of PartNumber' RIGHT([Copy of PartNumber],LEN([Copy of PartNumber]) - FINDSTRING([Copy of PartNumber],"-",1))
So My PartNumber Column is Replaced with ######### of type
string and the Copy of PartNumber column is replaced with
#### of type int
As I stated earlier PartNumber also is in the format of %-%-% which raises the following question:
How can I replace PartNumber Column with NULL if in format of
%-%-% and take the PartNumber and put it in new column named
SubPart while keeping all the logic within SSIS objects?
So in essence I want to do something like the following:
PartNumber = CASE
WHEN 'PartNumber' LIKE '%-%-%'
THEN ABC = PartNumber AND PartNumber IS NULL
END
I have tried the case statement in a Derived Column expression and is not working. Would I add the
CASE statement to the SQL command task in my OLE DB Source?
I hope this question is concise for you.If it must be in SSIS, I would put a derived column stage earlier in the process to do my check for me. Then in the second derived column stage do the check against each output column (part, subpart, copy of part) to decide which way it goes
PS Not sure if it's a 2012 function but SSIS has a function called TOKEN that will allow you to pull your subparts for you rather than the left/right you are doing -
How can I implement an user function in a derived column of a report ?
Hello,
I've a report and added a derived column.
In this column should be displayed the result of a function.
GETANZGJMONATE ( to_date(#START_AFA#,'DD.MM.YYYY'), #ND#, :P302_GJ );
How can I implement this?
Thanks in advance
Regards UlrikeUlrike - I would do this in the SQL statement (there may be other ways).
Presumably START_AFA and ND are table columns?
Presumably you've also created the GETANZGJMONATE function?
So, something like this should work (this also assumes that START_AFA is of a DATE type - you'll need the TO_DATE call if not):
SELECT COL1
, COL2
, START_AFA
, ND
, GETANZGJMONATE (START_AFA, ND, :P302_GJ)
from TABLE
where ...
Can't remember if you have to grant any particular execute permissions on the function ('grant execute on GETANZGJMONATE to public', for example) when you call it from SQL on a page, but you could try that if the function call fails.
Depending on what's in :P302_GJ and what the function parameter data type is, you might need to use the '&P302_GJ.' syntax or TO_NUMBER etc.
Hope this helps.
Regards,
John. -
How to handle line break embeded inside CSV column
Hi there,
I am under the pressure to make it work. I already put this question on APEX forum, but on second thought, I think it relates more to PL/SQL rather than APEX since APEX 4.1 already have utility to handle CSV Upload.
If you read it already in APEX forum, please ignore.
I am sorry for that. Thanks for reading.
I need to develop an app that allows user to upload CSV file to a interface table.
The APEX version at my workplace is 4.0.2.
I used the code from
http://dbswh.webhop.net/htmldb/f?p=BLOG:READ:0::::ARTICLE:11000346061523
It all works well till recently I find out
If a column in a CSV file cotain a line break (or new line) e.g. (The tester copy and paste this text which has line break into a column in a spreadsheet)
This is the first sentence.
This is the second sentence.
It will break the “This is the second sentence”. To a new column.
The contents of the CSV viewed in Notepad look as below
Assessment Date,Scheduled Date,Assessment Provider,Assessor Name,Court,First Name,Middle Name,Last Name,PRN Person Record Number,NHI Number,Defendant Attended Y/N,Is Dependent Y/N,Notes,Primary Ethnicity,"Ethnicity Other, please specify",Gender,Currently in Treatment Y/N,Substance of Concern 5,Other Substance Specified
22/09/2012,,Provider Co Name,Warren Edgley,Wellington,,,Salty,2545554,dgsdf,ergerg,,"This is the first sentence.
This is the second sentence.",Japanese,,Female,b,,
Here is the code from the CSV UTIL, please help me to find out how can I replace the line break to a space so that the uploading process is correct.
CREATE OR REPLACE PACKAGE BODY "CSV_UTIL"
AS
--strip the beginning and the end quotes, then replace double quotation with single
FUNCTION de_quote (p_str IN VARCHAR2, p_enc_by IN VARCHAR2)
RETURN VARCHAR2
IS
v_str VARCHAR2(32767) := p_str;
BEGIN
IF (p_enc_by IS NULL)
THEN
RETURN p_str;
ELSE
IF SUBSTR(p_str,-1) = p_enc_by THEN
v_str := SUBSTR(p_str,1,LENGTH(p_str)-1);
END IF;
IF SUBSTR(p_str,1,1) = p_enc_by THEN
v_str := SUBSTR(v_str,2);
END IF;
RETURN REPLACE (v_str,
p_enc_by || p_enc_by,
p_enc_by
END IF;
END de_quote;
PROCEDURE parse (p_str IN VARCHAR2, p_enc_by IN VARCHAR2, p_sep IN VARCHAR2)
IS
l_n NUMBER DEFAULT 1;
l_in_quote BOOLEAN DEFAULT FALSE;
l_ch NCHAR (1);
l_len NUMBER DEFAULT NVL (LENGTH (p_str), 0);
BEGIN
IF (l_len = 0)
THEN
RETURN;
END IF;
g_words := g_empty;
g_words (1) := NULL;
FOR i IN 1 .. l_len
LOOP
l_ch := SUBSTR (p_str, i, 1);
IF (l_ch = p_enc_by)
THEN
l_in_quote := NOT l_in_quote;
END IF;
IF (l_ch = p_sep AND NOT l_in_quote)
THEN
l_n := l_n + 1;
g_words (l_n) := NULL;
ELSE
g_words (l_n) := g_words (l_n) || l_ch;
END IF;
END LOOP;
g_words (l_n) := de_quote (g_words (l_n), CHR(10));
g_words (l_n) := de_quote (g_words (l_n), CHR(13));
FOR i IN 1 .. l_n
LOOP
g_words (i) := de_quote (g_words (i), p_enc_by);
END LOOP;
END parse;
Author: Oleg Lihvoinen
Company: DbSWH
Changes:
10.02.2011, There was a miscalculation of the file line last position in case it is the end of file
PROCEDURE upload (p_file_name VARCHAR2, p_collection_name VARCHAR2, p_enc_by IN VARCHAR2, p_sep_by IN VARCHAR2, p_rows NUMBER)
IS
v_blob_data BLOB;
v_clob_data CLOB;
v_clob_len NUMBER;
v_position NUMBER;
v_char NCHAR (1);
c_chunk_len NUMBER := 1;
v_line VARCHAR2 (32767) := NULL;
v_data_array vcarray;
v_rows NUMBER := 0;
n_seq NUMBER := 1;
dest_offset NUMBER := 1;
src_offset NUMBER := 1;
amount INTEGER := DBMS_LOB.lobmaxsize;
blob_csid NUMBER := DBMS_LOB.default_csid;
lang_ctx INTEGER := DBMS_LOB.default_lang_ctx;
warning INTEGER;
l_sep VARCHAR2(100) := CASE WHEN p_sep_by = '\t' THEN chr(9) ELSE p_sep_by END;
BEGIN
htmldb_collection.create_or_truncate_collection
(p_collection_name => p_collection_name);
-- Read blob from wwv_flow_files
SELECT blob_content
INTO v_blob_data
FROM wwv_flow_files
WHERE NAME = p_file_name;
v_position := 1;
DBMS_LOB.createtemporary (lob_loc => v_clob_data,
CACHE => TRUE,
dur => DBMS_LOB.SESSION
DBMS_LOB.converttoclob (v_clob_data,
v_blob_data,
amount,
dest_offset,
src_offset,
blob_csid,
lang_ctx,
warning
v_clob_len := DBMS_LOB.getlength (v_clob_data);
IF v_clob_len = 0 THEN
RETURN;
END IF;
WHILE (v_position <= v_clob_len + 1)
LOOP
v_char := DBMS_LOB.SUBSTR (v_clob_data, c_chunk_len, v_position);
v_line := v_line || v_char;
v_position := v_position + c_chunk_len;
-- When the whole line is retrieved and not end of file or end of file
IF v_char = CHR (10) AND v_position < v_clob_len OR v_position = v_clob_len + 1
THEN
parse (p_str => v_line, p_enc_by => p_enc_by, p_sep => l_sep);
v_data_array := g_words;
FOR i IN 1..g_words.count LOOP
IF i <= 50 THEN
v_data_array(i) := g_words(i);
ELSE
exit;
END IF;
END LOOP;
FOR i IN g_words.count + 1..50 LOOP
v_data_array(i) := null;
END LOOP;
v_rows := v_rows + 1;
-- exit if uploaded specified number of rows
IF p_rows IS NOT NULL AND v_rows > p_rows THEN
EXIT;
END IF;
-- Store data to collection
n_seq :=
htmldb_collection.add_member
(p_collection_name => p_collection_name,
p_c001 => v_data_array
(1),
p_c002 => v_data_array
(2),
p_c003 => v_data_array
(3),
p_c004 => v_data_array
(4),
p_c005 => v_data_array
(5),
p_c006 => v_data_array
(6),
p_c007 => v_data_array
(7),
p_c008 => v_data_array
(8),
p_c009 => v_data_array
(9),
p_c010 => v_data_array
(10),
p_c011 => v_data_array
(11),
p_c012 => v_data_array
(12),
p_c013 => v_data_array
(13),
p_c014 => v_data_array
(14),
p_c015 => v_data_array
(15),
p_c016 => v_data_array
(16),
p_c017 => v_data_array
(17),
p_c018 => v_data_array
(18),
p_c019 => v_data_array
(19),
p_c020 => v_data_array
(20),
p_c021 => v_data_array
(21),
p_c022 => v_data_array
(22),
p_c023 => v_data_array
(23),
p_c024 => v_data_array
(24),
p_c025 => v_data_array
(25),
p_c026 => v_data_array
(26),
p_c027 => v_data_array
(27),
p_c028 => v_data_array
(28),
p_c029 => v_data_array
(29),
p_c030 => v_data_array
(30),
p_c031 => v_data_array
(31),
p_c032 => v_data_array
(32),
p_c033 => v_data_array
(33),
p_c034 => v_data_array
(34),
p_c035 => v_data_array
(35),
p_c036 => v_data_array
(36),
p_c037 => v_data_array
(37),
p_c038 => v_data_array
(38),
p_c039 => v_data_array
(39),
p_c040 => v_data_array
(40),
p_c041 => v_data_array
(41),
p_c042 => v_data_array
(42),
p_c043 => v_data_array
(43),
p_c044 => v_data_array
(44),
p_c045 => v_data_array
(45),
p_c046 => v_data_array
(46),
p_c047 => v_data_array
(47),
p_c048 => v_data_array
(48),
p_c049 => v_data_array
(49),
p_c050 => v_data_array
(50)
-- Clear the line
v_line := NULL;
END IF;
END LOOP;
END;
END;In my apps, I save these straight into a table rather than an APEX collection because the number of columns can be longer than 50.
I want to find out how can replace these line break inside a column to a space.
If any one has any ideas, please let me know.
Thanks a lot in advance.
AnnAnn586341 wrote:
I think the code split the whole thing by this line
-- When the whole line is retrieved and not end of file or end of file
IF v_char = CHR (10) AND v_position < v_clob_len OR v_position = v_clob_len + 1
THEN
Yes, exactly. That piece of code believes all CHR(10) occurences are record delimiters.
It is not smart enough to recognize that a CHR(10) within quotation marks are part of the data.
Optimally a solution should keep the CHR(10) rather than replacing with spaces, but that will be a bigger rewrite of the UTL_CSV code ;-)
If you are happy with replacing with spaces, a "simple" solution could be something like:
Declare a boolean variable in upload procedure:
v_within_text_column boolean := false;And use it like this:
WHILE (v_position <= v_clob_len + 1)
LOOP
v_char := DBMS_LOB.SUBSTR (v_clob_data, c_chunk_len, v_position);
IF v_char = '"' THEN
v_within_text_column := NOT v_within_text_column;
ELSIF v_char = CHR(10) AND v_within_text_column THEN
v_char := ' ';
END IF;
v_line := v_line || v_char;
v_position := v_position + c_chunk_len;
-- When the whole line is retrieved and not end of file or end of file
IF v_char = CHR (10) AND v_position < v_clob_len OR v_position = v_clob_len + 1
THEN
v_within_text_column := false; -- To be safe always set this on "true" linebreaks
{code}
+(This is untested code just written here in the text editor.)+
It should work by toggling a flag whether you are "within" the quotes or not and then replacing CHR(10) with a space if you are within a text column.
This way we avoid having to go through the clob more than once (it is enough that this code walks the clob one character at a time...)
It will not handle if the clob contains situations like:
{code}
abc,123,"This is a text with a quote from a man who said \"To Be,
or Not To Be\" some hundred years ago",123,xyz
{code}
Escaped quotes would need separate attention ;-) -
Can i create any procedure or function inside a oracle reserve package?
Hi!
Can i create any procedure or function inside a oracle reserve package. Suppose, I want to create a function called x in the dbms_output package. Can i do that? Or can i extend the features of this package and create/derived a function from it like we extend any class in JAVA. I'm not sure - whether this is at all possible. I'll be waiting for your reply.
Thanks in advance.
Satyaki De.No, but you can write a wrapper package and use that instead of using the Built-In package directly. So, instead of calling DBMS_OUTPUT, you call your own Package.
Steven Feuerstein wrote a wrapper for DBMS_OUTPUT, called P:
Re: DBMS_OUTPUT.PUT_LINE -
Custom aggregate function inside a package.
Hi there,
I'm trying to write a custom aggregate function and group that function inside a package together with some other functions that I have. As an example (to simulate the problem I have) suppose my custom aggregation to do a summation of numbers looks like:
CREATE OR REPLACE TYPE SUM_AGGREGATOR_TYPE AS OBJECT (
summation NUMBER,
STATIC FUNCTION ODCIAggregateInitialize(agg_context IN OUT
SUM_AGGREGATOR_TYPE) RETURN NUMBER,
MEMBER FUNCTION ODCIAggregateIterate(self IN OUT SUM_AGGREGATOR_TYPE,
next_number IN NUMBER) RETURN NUMBER,
MEMBER FUNCTION ODCIAggregateMerge(self IN OUT SUM_AGGREGATOR_TYPE,
para_context IN SUM_AGGREGATOR_TYPE) RETURN NUMBER,
MEMBER FUNCTION ODCIAggregateTerminate(self IN SUM_AGGREGATOR_TYPE,
return_value OUT NUMBER, flags IN NUMBER) RETURN NUMBER
CREATE OR REPLACE TYPE BODY SUM_AGGREGATOR_TYPE IS
STATIC FUNCTION ODCIAggregateInitialize(agg_context IN OUT
SUM_AGGREGATOR_TYPE)
RETURN NUMBER IS
BEGIN
agg_context := SUM_AGGREGATOR_TYPE(NULL);
RETURN ODCIConst.Success;
END;
MEMBER FUNCTION ODCIAggregateIterate(self IN OUT SUM_AGGREGATOR_TYPE,
next_number IN NUMBER)
RETURN NUMBER IS
BEGIN
IF self.summation IS NULL THEN
self.summation := next_number;
ELSIF summation IS NOT NULL THEN
self.summation := self.summation + next_number;
END IF;
RETURN ODCIConst.Success;
END;
MEMBER FUNCTION ODCIAggregateMerge(self IN OUT SUM_AGGREGATOR_TYPE,
para_context IN SUM_AGGREGATOR_TYPE)
RETURN NUMBER IS
BEGIN
self.summation := self.summation + para_context.summation;
RETURN ODCIConst.Success;
END;
MEMBER FUNCTION ODCIAggregateTerminate(self IN SUM_AGGREGATOR_TYPE,
return_value OUT NUMBER, flags IN NUMBER)
RETURN NUMBER IS
BEGIN
return_value := self.summation;
return ODCIConst.Success;
END;
END;
If I write the following function definition:
CREATE OR REPLACE FUNCTION MY_SUM(input NUMBER)
RETURN NUMBER PARALLEL_ENABLE AGGREGATE USING SUM_AGGREGATOR_TYPE;
and corresponding type declaration to test:
CREATE OR REPLACE TYPE VECTOR
IS
TABLE OF NUMBER;
this statement:
select my_sum(column_value) from table(vector(1, 2, 1, 45, 22, -1));
gives the correct result of 70. However, creating a package with the function definition:
CREATE OR REPLACE PACKAGE MY_FUNCTIONS AS
FUNCTION MY_SUM(input NUMBER)
RETURN NUMBER PARALLEL_ENABLE AGGREGATE USING SUM_AGGREGATOR_TYPE;
END;
and calling it via:
select MY_FUNCTIONS.my_sum(column_value) from table(vector(1, 2, 1, 45, 22, -1));
explodes with:
ORA-00600: internal error code, arguments: [17090], [], [], [], [], [], [], [], [], [], [], []
Is it possible to have custom aggregate functions nested inside package declarations?
I'm using Oracle 11g, Release 2 (11.2.0.1.0).HiddenName wrote:
Is it possible to have custom aggregate functions nested inside package declarations?Yes, it is possible, you have succesfuly created your function. Your problem is that the database throws ORA-600 on execute. And with ORA-600 you can do 2 things: 1) google ORA-600 17090 or 2) contact your Oracle Support.
You could also try to declare the function without PARALLEL_ENABLE - just to try to see if it changes anything. You can also try to call your function against a regular table with rows and columns - not against an collection type with table() operator.
Anyway - these 2 tests should be usefull for Oracle Support.
I never tried to put a custom aggregate function into a package. First - the cases when you need a custom aggregate function to be written for your system are very rare. Second - even if I needed 1 then I never needed 2 or more custom aggregate functions on my system. And as I do not like to make my life more complex than necessary, I have created it as a stand-alone function. And it is works (slowly).I tried using a standard table as you suggested:
CREATE TABLE TEST_DATA
test_value NUMBER
INSERT INTO TEST_DATA
(SELECT column_value test_value from TABLE(vector(1, 2, 1, 45, 22, -1)));
COMMIT;
select my_sum(test_value) from test_data;
select my_functions.my_sum(test_value) from test_data;
I also tried removing the PARALLEL_ENABLE clause to create the package as follows:
CREATE OR REPLACE PACKAGE MY_FUNCTIONS AS
FUNCTION MY_SUM(input NUMBER)
RETURN NUMBER AGGREGATE USING SUM_AGGREGATOR_TYPE;
END;
And unfortunately it still breaks with the following error: SQL Error: ORA-00600: internal error code, arguments: [17090]. This looks like an Oracle bug to me as the PL/SQL parsing engine should have disallowed me to even create this if it is not supported in Oracle. Instead, it allows me to create the package, and breaks when I call the function with this weird error (additionally cutting my connection from the database) instead of disallowing me to do this altogether and printing a nice error message telling me that Oracle doesn't support this. How would I go about logging a ticket for this?
Edited by: wcmatthysen on Dec 1, 2010 12:51 PM -
Calling a Function inside a procedure
Can you call a function inside a procedure?...if so....how?
Not all built-in functions can be used directly in an assignment.
SQL> CREATE PROCEDURE p (p_val IN VARCHAR2) AS
2 l_v VARCHAR2(10);
3 BEGIN
4 l_v := DECODE(p_val,'YES','TRUE','FALSE');
5 END;
6 /
Warning: Procedure created with compilation errors.
SQL> show error
Errors for PROCEDURE P:
LINE/COL ERROR
4/4 PL/SQL: Statement ignored
4/11 PLS-00204: function or pseudo-column 'DECODE' may be used inside
a SQL statement onlyTTFN
John -
Calling a function inside an iFrame
I have a app I'm building that plays videos through an iFrame (src is our server). I want to have buttons in my client that when clicked can call functions inside the iFrame (e.g. play, pause). Obviously the problem is with doing this is browser security will not allow it. I understand AIR provides extra parameters for an iFrame to achieve this. I have been reading up on this and seen to get the impression that calling functions in the iFrame can only be done before the page loads?
Has anyone any basic examples of how I could achieve this?Not all built-in functions can be used directly in an assignment.
SQL> CREATE PROCEDURE p (p_val IN VARCHAR2) AS
2 l_v VARCHAR2(10);
3 BEGIN
4 l_v := DECODE(p_val,'YES','TRUE','FALSE');
5 END;
6 /
Warning: Procedure created with compilation errors.
SQL> show error
Errors for PROCEDURE P:
LINE/COL ERROR
4/4 PL/SQL: Statement ignored
4/11 PLS-00204: function or pseudo-column 'DECODE' may be used inside
a SQL statement onlyTTFN
John -
Unable to Load CSV file with comma inside the column(Sql Server 2008)
Hi,
I am not able load an CSV file with Comma inside a column.
Here is sample File:(First row contain the column names)
_id,qp,c
"1","[ ""0"", ""0"", ""0"" ]","helloworld"
"1","[ ""0"", ""0"", ""0"" ]","helloworld"
When i specify the Text Qualifier as "(Double quotes) it work in SQL Server 2012, where as fail in the SQL Server 2008, complaining with error:
TITLE: Microsoft Visual Studio
The preview sample contains embedded text qualifiers ("). The flat file parser does not support embedding text qualifiers in data. Parsing columns that contain data with text qualifiers will fail at run time.
BUTTONS:
OK
I need to do this in sql server 2008 R2 with Service pack 2, my build version is 10.50.1600.1.
Can someone let me know it is possible in do the same way it is handle in SQL Server 2012?
Or
It got resolved in any successive Cumulative update after 10.50.1600.1?
Regards HarshHello,
If you have CSV with double quotes inside double quotes and with SSIS 2008, I suggest:
in your data flow you use a script transformation component as Source, then define the ouput columns id signed int, Gp unicode string and C unicode string. e.g. Ouput 0 output colmuns
Id - four-byte signed
gp - unicode string
cc - unicode string
Do not use a flat file connection, but use a user variable in which you store the name of the flat file (this could be inside a for each file loop).
The user variable is supplied as a a readonly variable argument to the script component in your dataflow.
In the script component inMain.CreateNewOutputRows use a System.IO.Streamreader with the user variable name to read the file and use the outputbuffer addrow method to add each line to the output of the script component.
Between the ReadLine instraction and the addrow instruction you have to add your code to extract the 3 column values.
public override void CreateNewOutputRows()
Add rows by calling the AddRow method on the member variable named "<Output Name>Buffer".
For example, call MyOutputBuffer.AddRow() if your output was named "MyOutput".
string line;
System.IO.StreamReader file = new System.IO.StreamReader( Variables.CsvFilename);
while ((line = file.ReadLine()) != null)
System.Windows.Forms.MessageBox.Show(line);
if (line.StartsWith("_id,qp,c") != true) //skip header line
Output0Buffer.AddRow();
string[] mydata = Yourlineconversionher(line);
Output0Buffer.Id = Convert.ToInt32(mydata[0]);
Output0Buffer.gp = mydata[1];
Output0Buffer.cc = mydata[2];
file.Close();
Jan D'Hondt - SQL server BI development -
Data Conversion and Derived Column issues
I have a strange issue occurring with one of my SSIS packages.
What needs to happen:
I have to read data from a table that stores a field in NCHAR(40)
Send it through a Data Cleansing SSIS Component that forcefully outputs the data at a cleansed state as DT_STR(50)
Update the same source table with cleansed data - Using an UPSERT third party tool
Of course, I can't update the tables that stores data in NCHAR(40) with data from DT_STR(50), so I'm trying to use the Data Conversion Component, the Derived Column Component or a combination of, to set the data to DT_WSTR as well as to set the correct length,
from 50 to 40.
The Data Conversion Component fails when I try to set the incoming data (DT_STR(50)) to DT_WSTR(40):
[Data Conversion [186]] Error: Data conversion failed while converting column "MD_Address1" (97) to column "_MD_Address1" (190). The conversion returned status value 2 and status text
"The value could not be converted because of a potential loss of data.".
Now I tried the same thing with the Derived Column Component, converting the data from the DQS component from (DT_STR(50)) to DT_WSTR(40) and there's the error message:
[Derived Column [196]] Error: The "Derived Column" failed because truncation occurred, and the truncation row disposition on "Derived Column.Inputs[Derived Column Input].Columns[_MD_Address1]"
specifies failure on truncation. A truncation error occurred on the specified object of the specified component.
I also tried a combination of Data Conversion (From DT_STR(50) to DT_WSTR(50)) and a Derived Column just casting to the correct size, but that didn't work either.
Every time I try to change the size, one of the components fail.
It appears that my issue has nothing to do with the data types, but the actual data length.
Why can't SSIS CAST my input from 50 to 40?
What do I need to do to cast/convert data from DT_STR(50) to DT_WSTR(40)?Hi IgorSantos_FL,
It is the expected behavior when we try to convert DT_STR(50) (means maximum 50 characters in the value) to DT_WSTR(40) (means maximum 40 characters in the value). However, the truncation issue should not occur if you convert DT_STR(50) to
DT_WSTR(50). Could you post the error message that you received when converting DT_STR(50) to DT_WSTR(50)? It may be a different issue.
Regards,
Mike Yin
TechNet Community Support -
Finding correlation using derived columns?
I want to use the aggregate CORR_S function which take two numerical columns and output the correlation between them. Assuming I have the table:
Employee(name, gender, salary) where I want to create two derived columns, how would I do it?
Like,
SELECT CORR_S(
SELECT salary
FROM employees
WHERE salary < 10000,
SELECT salary
FROM employees
WHERE salary > 10000
) as correlation
FROM employees
It will find the correlation between salary's less than $10,000 in one column and salary's more than $10,000 in another column. Of course, I get an error when running the query. It's no problem if I use CORR_S(salary, salary) but I want to use derived columns, is it possible? Or, will I need to create a new table view with two new columns CORR_S(lowSalary, highSalary)?
Thanks.It may be correct sql query, but statistically ( i mean here the OP question, or at least how i interpret it) it is nonsense.
Clearly , correlation show how good distribution of one set of items corresponds to another set, so the order of item in the itemset does matter. Consider
SQL> WITH t AS (
2 SELECT 1 id1, 10 id2, 30 id3 FROM dual UNION ALL
3 SELECT 2 , 20 , 10 FROM dual UNION ALL
4 SELECT 3 , 30 , 20 FROM dual )
5 SELECT corr_s(id1,id2), corr_s(id1,id3)
6 FROM t
7 /
CORR_S(ID1,ID2) CORR_S(ID1,ID3)
1 -.5So, it would make sense to calculate correlation between salary and hiredate, salary and department number, salary and employee name's length, whatever, but how can be correlated salary less 10k to salary more than 10k - i can't understand it. They are not in the same set, but in two different sets ( while one can have either more 10k or less 10k, but not both) and relation between these two sets is not given by OP.
Best regards
Maxim -
Derived column dot showing in CSV
Hi, I have a derived column.
When I unabled the report so it downloads the data into a spreadsheet,
I see that it's showing all the column headings but not tha value.
Is there a way to fix it so it also shows the values of the derived columns in CSV?
Thanks,Hi,
I had a similar problem. As I understand it, the csv export function re-runs the SQL statement and produces the output based on that rather than the contents of your report (which is why, for instance, you get all of the data even though your report may be paginated).
The only solution, which I had to use, was to change the SQL statement to include the derived values rather than create them on the report definition.
Regards,
Andy
Maybe you are looking for
-
Airport extreme card disappeared!
I've struggled with using my Macbook Pro's airport for a while, and I finally got it to a maintain a steady signal for about two weeks. Then, today when I came home and turned on my MBP, there was no icon for the Airport card at the top. I looked in
-
I cannot display my desktop on my tv
I am trying to view my laptop screen on my tv. I am using an HDMI cable and a belkin adapter that I bought from apple. The desktop screen appears but my icons do not. I am trying to watch a video from my laptop onto my tv.
-
Example of Extension installer ?
Hi Anybody has an example of a JNLP application that uses an installer (installer-desc) ?? I am tyring to make one, but somehow, I cannot get the application to start after the installer finishes... cheers Yann.
-
How to use session state protection
I use Apex 3.2.1 I access my site by a url passing a parameter like this : f?p=101:1:0::::ITEM1:1234567. There is no login and password to access the site. The value of the parameter ITEM1 is the authorization of the first page, with a database funct
-
How to send recursive deadline monitoring mails?
Hi all, I have to send a deadline monitoring mails to the approver for every 3 days till the approver takes some action. I am able to send the deadline monitoring mail once by using 'LATEST END' tab. even tried using loop, but i am getting mails cont