Loading Pipe delmiter flat file with ',' (comma)

Hi Guys,
I have a field which contains ',' as its value. The value is 0315A Mgr, Cust Service for a infoobject called ZJOB_TIT.
Also, I have wrote the following code in field routine since previously this field was throwing hexadecimal errors and the load is through flat file.
DATA: v_gpurqtrnr TYPE /BIC/OIZJOB_TIT.
MOVE SOURCE_FIELDS-/BIC/ZJOB_TIT TO v_gpurqtrnr.
TRANSLATE v_gpurqtrnr TO UPPER CASE.
IF v_gpurqtrnr CN
',<>?/{}[]\|;:"''ABCDEFGHIJKLMNOPQRSTUVWXYZ!@#$%^&*()_-+=123456789~` .'
RESULT = space.
ELSE.
RESULT = v_gpurqtrnr.
ENDIF.
After loading the data, in preview of PSA - everything looks fine. Field ZJOB_TIT shows to have the value 0315A Mgr, Cust Service. But when I load to the cube - I get blank for the same field.
I am not sure what more i can do to eradicate this problem.

Hi
PSA is the data before under going your tranformation or field routine. so you are able to see the data in that field . Now as per the code which you have written above it will surely be deleting all the characteristics which you have given under '.......' . Modify your code to eliminate only the comma and not the alpha numberic values.
Rajesh

Similar Messages

Multiple flat files with Comma delimiter and Pipe Delimiter in the sub folders.

Hi,
I have a directory C:\doc\Outcomes\Health --(This is the main path).
In the path above i have multiple subfolders like
A
B
C
D
Folder A & B have 20 flat files each which are comma separated and pipe delimiter files.
Folder C&D have 20 excel files each.
1) So, In SSIS while looping through the subfolders how do i limit to loop only excel files one time and flat files one time.
2) In folder A&B, how do i loop only Pipe delimiter files (neglecting comma saperated files). I want to loop only pipe delimiter files while using for each loop container.
Thanks

Both are txt files, but the data inside the files is saperated by ',' and '|'. ( comma and pipe)
Thats ok
If delimiters are not consistent you can use this method
http://visakhm.blogspot.in/2014/07/ssis-tips-handling-inconsistent-text.html
Please Mark This As Answer if it solved your issue
Please Mark This As Helpful if it helps to solve your issue
Visakh
My MSDN Page
My Personal Blog
My Facebook Page

Down loading data to flat file with header descritpions

Hi,
am using Function Module "GUI_DOWNLOAD"
am passing internal table which contains all header descriptions to Fieldnames in TABLES.
CALL FUNCTION 'GUI_DOWNLOAD'
 EXPORTING
 filename = g_file
 filetype = 'ASC'
 write_field_separator = 'X'
 header = '00'
 TABLES
 data_tab = g_t_extract_fin
 fieldnames = l_t_header.
My question is the above code is works in all SAP versions or it's works in 4.6C only
Regards
Suresh

Hai,
The Type of Header data may not be same as Data In it.
Therfore use Two GUI_UPLOAD one for header and another for data.
But Second time use the option APPEND to previous dat in tha FM GUI_UPLOAD.
Reward Points if it helps you.
Regds,
Rama.Pammi

Infospoke flat file with tab seperator

Hello,
I have a infospoke which creates a flat file with comma field separator. I would like to have tab character as a field seperator. How would I change it.

Varun,
as Ram told you can give ',' as Separator.
all the best.
Reagrds,
Nagesh Ganisetti.

Unable to Load CSV file with comma inside the column(Sql Server 2008)

Hi,
I am not able load an CSV file with Comma inside a column.
Here is sample File:(First row contain the column names)
_id,qp,c
"1","[ ""0"", ""0"", ""0"" ]","helloworld"
"1","[ ""0"", ""0"", ""0"" ]","helloworld"
When i specify the Text Qualifier as "(Double quotes) it work in SQL Server 2012, where as fail in the SQL Server 2008, complaining with error:
TITLE: Microsoft Visual Studio
The preview sample contains embedded text qualifiers ("). The flat file parser does not support embedding text qualifiers in data. Parsing columns that contain data with text qualifiers will fail at run time.
BUTTONS:
OK
I need to do this in sql server 2008 R2 with Service pack 2, my build version is 10.50.1600.1.
Can someone let me know it is possible in do the same way it is handle in SQL Server 2012?
Or
It got resolved in any successive Cumulative update after 10.50.1600.1?
Regards Harsh

Hello,
If you have CSV with double quotes inside double quotes and with SSIS 2008, I suggest:
in your data flow you use a script transformation component as Source, then define the ouput columns id signed int, Gp unicode string and C unicode string. e.g. Ouput 0 output colmuns
Id - four-byte signed
gp - unicode string
cc - unicode string
Do not use a flat file connection, but use a user variable in which you store the name of the flat file (this could be inside a for each file loop).
The user variable is supplied as a a readonly variable argument to the script component in your dataflow.
In the script component inMain.CreateNewOutputRows use a System.IO.Streamreader with the user variable name to read the file and use the outputbuffer addrow method to add each line to the output of the script component.
Between the ReadLine instraction and the addrow instruction you have to add your code to extract the 3 column values.
public override void CreateNewOutputRows()
Add rows by calling the AddRow method on the member variable named "<Output Name>Buffer".
For example, call MyOutputBuffer.AddRow() if your output was named "MyOutput".
string line;
System.IO.StreamReader file = new System.IO.StreamReader( Variables.CsvFilename);
while ((line = file.ReadLine()) != null)
System.Windows.Forms.MessageBox.Show(line);
if (line.StartsWith("_id,qp,c") != true) //skip header line
Output0Buffer.AddRow();
string[] mydata = Yourlineconversionher(line);
Output0Buffer.Id = Convert.ToInt32(mydata[0]);
Output0Buffer.gp = mydata[1];
Output0Buffer.cc = mydata[2];
file.Close();
Jan D'Hondt - SQL server BI development

How can we load a flat file with very, very long lines into a table?

Hello:
We have to load a flat file with OWB. The problem is that each of the lines in the file might be up to 30.000 characters long (up to 1.000 units of information in each line, 30 characters long each)
Of course, our mapping should insert these units of information as independent rows in a table (1.000 rows, in our example).
We do not know how to go about it. We usually load flat files using table functions, but we am not sure that they will be able to cope with these huge lines. And how should we pivot those lines? Will the Pivot operator do the trick? Or maybe we should pivot those lines outside the database before loading them?
We are a bit lost. Any suggestion would be appreciated.
Regards
Edited by: [email protected] on Oct 29, 2008 8:43 AM
Edited by: [email protected] on Oct 29, 2008 8:44 AM

Yes, well, we could define a 1.000 column external table, and then map those 1.000 columns to the Pivot operator… perhaps it would work. But we have been investigating a little bit, and we think that we have found a better solution: there is a unix utility called “fold”. This utility can split our 30.000 character lines in 1.000 lines, 30 characters long each: just what we needed. Then we can load the resulting file using an external table.
We think this is a much better solution that handling 1.000 columns in the external table and in the Pivot operator.
Thanks for your help.
Regards
Edited by: [email protected] on Oct 29, 2008 10:35 AM

How to load a flat file with lot of records

Hi,
I am trying to load a flat file with hundreds of records into an apps table. when i create the process and deploy it onto the console it asks for an input in an html form. why does it ask for an input when i have specified the input file directory in my process? is there any way around tis where in it just reads all the records from the flat file directly??is custom queues anyway related to what I am about to do?any documents on this process will be greatly appreciated.If any one can help me on this it will be great. thank you guys....

After deploying it, do you see if it is active and the status is on from the BPEL console BPEL Process tab? It should not come up to ask for input unless you are clicking it from the Dashboard tab. Do not click it from the Dashboard. Instead you should put some files into the input driectory. Wait few seconds you should see the instances of the BPEL process is created and start to process the files asynchrously.

How to load a flat file with utf8 format in odi as source file?

Hi All,
Anybody knows how we can load a flat file with utf8 format in odi as source file.Please everybody knows to guide me.
Regards,
Sahar

Could you explain which problem are you facing?
Francesco

Data loading mechanism for flat file loads for hierrarchy

Hi all,
We have a custom hierarchy which is getting data from a flat file that is stored in the central server and that gets data from MDM through XI. Now if we delete few records in MDM, the data picked in BI will not consist of the records which are deleted. Does it mean that the hierarchy itself deletes the data it consists of already and does a full load or does it mean every time we load the data to the BI, do weu delete the records from the tables in BI and reload?
Also we have some Web Service(gets loaded from XI) text data sources.
Is the logic about updating the hierrarchy records different as compared to the existing web service interfaces?
Can anyone please tell me the mechanism behind these data loads and differentiate the same for above mentioned data loads?

create the ODS with the correct keys. And load full loads from the flat files. You can have a cube pulling data from the ODS.
Load data in ODS
Create the cube.
Generate export datasource ( rsa1 > rt clk the ods > generate export Datasource )
Replicate the export ds ( rsa1 > source system > ds overview > search the ds starting with 8 + the ODS name
press the '+' button activate the transfer rules and comm str
create the update rules for the cube with the above infource ( same as '8ODSNAME' Datasource )
create infopackage with intial load (in the update tab)
Now load data to cube
Now load new full loads to ODS
create a new infopackage for delta (in the update tab)
run in infopackage. (any changes / new records will be loaded to cube)
Regards,
BWer
Assing points if helpful.

Importing From Flat File with Dynamic Columns

HI
I am using ssis 2008,i have folder in which I have Four(4) “.txt” files each file will have 2 columns(ID, NAME). I loaded 4
files in one destination, but today I receive one more “.txt” file here we have 3 columns (ID, NAME, JOB) how can I get a message new column will receive in source. And how can I create in extra column in my destination table dynamically …please help me

Hi Sasidhar,
You need a Script Task to read the names and number of columns in the first row of the flat file each time and store it in a variable, then create a staging table dynamically based on this variable and modify the destination table definition if one ore more
new columns need to be added, and then use the staging table to load the destination table. I am afraid there is no available working script for your scenario, and you need some .NET coding experience to achieve your goal. Here is an example you can refer
to:
http://www.citagus.com/citagus/blog/importing-from-flat-file-with-dynamic-columns/
Regards,
Mike Yin
TechNet Community Support

Reading flat files with variable names in SSIS

I have a ssis package that reads a flat file from a network drive (using a flat file connection manager) and loads the data into sql server tables. I have this working on a fixed file name, however in reality the file name will not be the
same from run to run.      Essentially, I need to check a folder each day and if there is a file there with a certain prefix (for example, 'datafile'), I need to process this file(s). In other words, if there is a file in the folder
called datafilexyz, my process needs to read it in and process it.    If there are files named datafileabc, datafiledef, and testfile123, I need to read in and process the datafileabc and datafiledef flat files.
I'm not sure how to make this work.     I haven't had any SSIS training, just what I can find on the internet and looking at existing SSIS packages (I haven't found any that do what I'm trying to do here) so I'm kind of lost.
Any help is appreciated.

this is working well.    How can I, after loading each flat file, move the flat file to an archive folder?    I'm trying to use a file system task, but doing something wrong.    I created and reference
an archive folder connection manager in the destination connection, and reference the original folder connection manager for the source connection, but get 'sourcepath is not valid on operation movefile'.    I think a file system task is what
is needed (within the foreach loop after the data load for each flat file), but I'm not doing something correctly.
Sounds fine except for one thing. Did you assign the filename variable to connection string property of the source file connection manager used by file system task?
Also it should be existing file option you should choose for
source connection and existing folder for destination.
Please Mark This As Answer if it solved your issue
Please Mark This As Helpful if it helps to solve your issue
Visakh
My MSDN Page
My Personal Blog
My Facebook Page

SSIS project - read multiple flat files with different formats

hi all,
i need to import multiple flat files with different formats into different tables of the sql server database and not able to figure out the best way out in ssis to do so...
please advise the possible methods in ssis to do so and if possible the process which can be dynamic as file names or columns might change in future.

Hi AK1987,
To import flat files with dynamic columns, we can use Script Task inside a Foreach Loop Container to parse the first row of the flat file to get the columns names and save them into a .NET variable, then, we can create “Create Table” script based on this
variable, and then store the script into a SSIS package variable. After that, we create a staging table based on the package variable, load the flat file data to the staging table. Eventually, we load data from the staging table to the destination table. For
the detail steps, please walk through the following blog:
http://www.citagus.com/citagus/blog/importing-from-flat-file-with-dynamic-columns/
Regards,
Mike Yin
TechNet Community Support

SSIS - Import Multiple flat files with different metadata

Hi ,
I have set of flat files with different metadata structure, I would like to load them into staging tables.
1. ) Can we load the flatfiles into the staging tables with out having multiple data flow task.
2.) If possible , can we programmatically select the staging table based on the metadata of the flatfile and load them.
Please advise.
Thanks
Thiya

Nope in SSIS a data flow task needs to have a fixed metadata. So if your file metadata varies then best option would be use OPENROWSET syntax to pull the data and populate into your staging table. You may also use
SELECT .. INTO StagingTable ... FROM OPENROWSET (...)
syntax to create staging table at runtime based on the file metadata
http://sqlmate.wordpress.com/2012/08/09/use-your-text-csv-files-in-your-queries-via-openrowset/
If you want to do this in SSIS you need to create data flow dynamically using script task and build the metadata
see
http://www.selectsifiso.net/?p=288
Please Mark This As Answer if it helps to solve the issue Visakh ---------------------------- http://visakhm.blogspot.com/ https://www.facebook.com/VmBlogs

How to load a long flat file field into BW

Hi all,
I have a field from flat file with 256 characters to be loaded into BW. As we all know the max length an info object can take is only 60. How can I split the flat file field so that I can load that data into BW and how can I then concatenate them in the BEx.
Can anyone suggest a way?
Thanks in advance.
Vinay

Hi Vinay,
we already had this discussion in this forum. In short:
1. create a number range object
2. create a table with a number field a key and a text field
3. while loading the data check your text against the table. If the text already exist, get the number out of the table. If the text doesn't exist, get a new number out of your number range object and post this number + the text into your table.
4. in your loaded data always work with the number you assigned to the text
5. create a infoobject for the number-field
6. create a variable that takes it's value out of the table.
regards
Siggi

Flat File with multiple record types (OWB 10.2.0.2)

Hi!
I`m using OWB 10.2.0.2 and I`m trying to load a flat file with multiple record types, using SQL LOADER.
In the flat file editor in the Record tab, I`ve set the type values and the corresponding record names like this:
Type Value Record Name
======== ===========
T TRAILER
0 DETAILS
1 DETAILS
2 DETAILS
When using this flat file in a mapping to load the data in a staging table, the generated code looks like this:
INTO TABLE TRAILER
TRUNCATE
REENABLE DISABLED_CONSTRAINTS
WHEN (1:1) = 'T'
INTO TABLE DETAILS
APPEND
REENABLE DISABLED_CONSTRAINTS
WHEN (1:1) = '0,1,2'
The above clause (WHEN (1:1) = '0,1,2') is wrong as I expect one "INTO TABLE..." clause for each record type.
Could this be a bug or am I doing something wrong?
Thanks a lot for your help,
Yorgos

We`re using two target tables, one for the trailer record and the other for the details records.
We are facing this problem from the moment we upgraded from OWB 10.1 to OWB 10.2.0.2, so we think it must be something with the way the sql loader code is generated in the new version.
As our data sources are mainly flat files coming from mainframes, this is a huge problem for us. We even asked an expert in DW from Oracle to help us on this, but still haven`t found a solution.
Is there any workaround for this or should we forget sql loader and go with an external tables + custom PL/SQL code solution?
Your help is greatly appreciated Jean-Pierre.
Regards,
Yorgos

Loading Pipe delmiter flat file with ',' (comma)

Similar Messages

Maybe you are looking for