Reading a semi column delimited multi line Flat file on KM repository

Hi,
I have a requirement in our project to read a multi line, semi column delimited flat from kept on the Knowledge Management repository and display the contents in the Portal.
I had tried couple of options and was unable to read the file. I am not sure which are the correct APIs I should be using.
Can any one of the experts could guide me with the sample code, that will be great!
Your early response is highly appreciated.
Regards
Venkat

here you go.
//******* Read file from KM
String repository_km;
String FileURL;
repository_km="/documents/data/data.txt";
try
//Getting the user......
IWDClientUser wdClientUser = WDClientUser.getCurrentUser();
IUser sapUser = wdClientUser.getSAPUser();
com.sapportals.portal.security.usermanagement.IUser ep5User =
WPUMFactory.getUserFactory().getEP5User(sapUser);
//Getting the Resource.........
IResourceContext resourseContext = new ResourceContext(ep5User);
IResourceFactory resourseFactory = ResourceFactory.getInstance();
//path to the KM Folder ("/documents/data/data.txt")
RID rid= RID.getRID(repository_km);
com.sapportals.wcm.repository.IResource resource =
resourseFactory.getResource(rid, resourseContext);
if (resource != null)
     String text = "";
     BufferedReader in = new BufferedReader(new InputStreamReader(resource.getContent().getInputStream()));
     int count=0;
     while ((text = in.readLine()) != null)
     strText[count] =text;
     count++;
catch (RuntimeException e2) {
wdComponentAPI.getMessageManager().reportException("Error in reading file from KM : "+e2.getMessage(),true);
//                  TODO Auto-generated catch block

Similar Messages

Column types on a flat file

I am using a flat file as a target and when I define column types as anything other then string I get the following error on execution
000 : null : com.sunopsis.jdbc.driver.file.a.i
com.sunopsis.jdbc.driver.file.a.i
     at com.sunopsis.jdbc.driver.file.a.d.setDate(d.java)
     at com.sunopsis.sql.SnpsQuery.updateExecStatement(SnpsQuery.java)
     at com.sunopsis.sql.SnpsQuery.addBatch(SnpsQuery.java)
     at com.sunopsis.dwg.dbobj.SnpSessTaskSql.execCollOrders(SnpSessTaskSql.java)
     at com.sunopsis.dwg.dbobj.SnpSessTaskSql.treatTaskTrt(SnpSessTaskSql.java)
     at com.sunopsis.dwg.dbobj.SnpSessTaskSqlI.treatTaskTrt(SnpSessTaskSqlI.java)
     at com.sunopsis.dwg.dbobj.SnpSessTaskSql.treatTask(SnpSessTaskSql.java)
     at com.sunopsis.dwg.dbobj.SnpSessStep.treatSessStep(SnpSessStep.java)

hi lpantuso ,
suppose u want to transfer file data like
xxxx 1111 12-01-2005
yyyy 2222 12-03-2007
zzzz 3333 01-05-2007
to another blank file so you have do folloowing steps
craete file data server for individual data server suppose you r developing application on ur desk to so create single file dat server in topology manager with proper jdbc url and driver name as
Jdbc driver as : com.sunopsis.jdbc.driver.file.FileDriver choose from drop down menu
and add url as: jdbc:snps:dbfile
now for each different folder create physical schema with the schema and work schema name as the location of the folder .
click on context tab add the proper logical schema name
come to designer window
Create individual model for each physical schema (for each folder I can say)
Use the proper logical schema name (while you'll select technology as file all the logical schema related to the technology wil be displayed just select ur logical schema)
than create individual datastore for each file (you can create two data store at max for each file).
assuming you r using delimited file.
while reversing existing source file you will have all the columns of the file
change data type and size of the fields.
For the target File datastore file should be there wid blank structure so add column wid proper data type, for second field use numeric and for date use string as datatype .Size must be appropriate for the target file otherwise you'll have warning at designing time.
create an interface
use Global context with "Sunopsis memory engine" as staging area other than target.
then map the source and target file
at Flow tab
choose
LKM File to SQl for source area
NO LKM is there for Staging area
IKM SQL to file append for target (If these are not automatically updated)
Then try to execute,it's working
Note : If you want to transfer table data to a file then for date type data you have to check following things
1) at the time of mapping of date type from table to file interface use
to_char(<SourceTableNameAlias> . <columnName>) instead of
<SourceTableNameAlias> . <columnName>
and date type data field of the target file should be String type.............

How to split column wise into separate flat files in ssis

IN SSIS...
1.---->I have a sales table country wise regions like (india, usa, srilanka) ....
india usa
srilanka
a b
c
d e
f
so i want output like in
flat file1.txt has india flat file2.txt has usa flat file3.txt has srilanka
a b
c
d e
f
2.----->i dont know how many regions in my table....dynamically split into separate flat files ....
pls help me.....thank u

I think what you can do is this
1. Do a query based on UNPIVOT to get the data as rows instead of columns
For that you can use a query like this
IF OBJECT_ID('temp') IS NOT NULL DROP TABLE temp
CREATE TABLE temp
Country varchar(100),
Val decimal(25,5)
DECLARE @CountryList varchar(3000),@SQL varchar(max)
SELECT @CountryList = STUFF((SELECT ',[' + Column_Name + ']' FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME = '<SalesTableNameHere>' FOR XML PATH('')),1,1,'')
SET @SQL= 'SELECT * FROM <SalesTableNameHere> t UNPIVOT (Val FOR Country IN (' + @CountryList + '))p'
INSERT temp
EXEC (@SQL)
Once this is done you'll get data unpivoted to table
Then you can use a execute sql task with query like above
SELECT DISTINCT Country FROM Temp
Use resultset option as full resultset and store the result to an object variable
Then add a ForEach loop container with ADO enumerator and map to the object variable created above. Have variables inside loop to get inidvidual country values out.
Inside loop place a data flow task. Use a variable to store source query , make EvaluateAsExpression true for it and set Expression as below
"SELECT Val FROM Temp WHERE Country = " + @[User::LoopVariable]
Where LoopVariable is variable created inside loop for getting iterated values
Inside data flow task place a oledb source, choose option as SQL command from variable and map to the above query variable.
Link this to flat file destination create a flat file connection manager. Set a dynamic flat file connection using expression builder. Make it based on a variable and set variable to increment based on loop iteration
The core logic looks similar to this
http://visakhm.blogspot.ae/2013/09/exporting-sqlserver-data-to-multiple.html
dynamic file naming can be seen here
http://jahaines.blogspot.ae/2009/07/ssis-dynamically-naming-destination.html
Please Mark This As Answer if it solved your issue
Please Vote This As Helpful if it helps to solve your issue
Visakh
My Wiki User Page
My MSDN Page
My Personal Blog
My Facebook Page

Dynamic Column in multi line format URGENT

Hi All,
I am developing a packing slip report in RTF format and each line has more than one serial numbers like
item/QTY
Serial Number
ITM20401 / 10
SRL1001,
SRL1002,
SRL1003,
SRL1004,
SRL1005,
SRL1006,
SRL1007,
SRL1008,
SRL1009,
SRL1010
Some time report has more that 60 serial number for single line. So i want to print serial number in horizontal and vertical format like
item/QTY
Serial Number
ITM20401 / 10
SRL1001,SRL1002,SRL1003,
SRL1004,SRL1005,SRL1006,
SRL1007,SRL1008,SRL1009,
SRL1010
Can any body tell me solution for this.
Thanks
XMLP User

Hi Ravi--
Could you post what you put in the RTF fields in order to be able to do this?
Thank you very much!
Katie

Using a delimiter in a flat file

Hi People,
If I am writing a file to the PC and I have to use a delimiter '~' to seperate the values, what should I do? Can somebody give me some example or syntax?
Thanks,
AM

You can use:
CONCATENATE field1 field2 field3 INTO output SEPARATED BY '~'.

I need to combine a few thousand multi line txt files into individual lines in a master file.

I need to combine lots of text files into one file-but each file is 6 or 7 lines long, and I need to remove the line breaks so each file is an individual line in the output. I feel like I should be able to do this with CAT, but can't figure out how. Perhaps there's aother tool that would allow me to stip the line breaks before I cat them all together? (and perhaps this is a question that should be posted somewhere else?)

You can remove the line breaks with tr -d '\n', :
tr -d '\n' < file > newfile
If all the files are in the same directory, just use a loop:
for i in ~/Desktop/*.txt ; do tr -d '\n' < $i >> ~/Desktop/Untitled3.tmp; echo >> ~/Desktop/Untitled3.tmp; done
If in multiple directories, you can use find, but the files need similar names (i.e., Untitledxxx.xxx).
BTW, best place for this question would be the "Mac OS X Technologies" forum with a tag of UNIX
Tony

HP4284A Frequency Sweep with frequency points read from a tab delimited excel or lvm file

Hello,
I am using a HP4284A, and have downloaded the file here. However, I cannot figure out what format of file it takes as input: I understand this should be a .lvm tab delineated value file, but in what order is the data? If someone could provide an explanation, or an example data file, I'd greatly appreciate that.
Thanks.
Ps. I know the linked article says that 'The frequency points are as defined for the oscillator in the HP4284A user manual.', bit I have been through the manual from end to end, and I can't find anything of the sort. The closest I've been able to find is the codgin for remote access, but that isn't very relevant as Labview and the coding (C++, I believe) used are very different.
Solved!
Go to Solution.

Sure. I wound up adding some additional features to it, though, badly. It's perfectly functional but badly coded (veteran users please don't laugh); I could probably optimize it now, having learned more, but I'm too lazy to. There's a writeup, too.
Attachments:
HP4284A Frequency and voltage sweep.vi ‏288 KB
Program writeup.docx ‏12 KB

Open Source API for Flat Files?

I did some searches on SourceForge for a flat-file serializer and deserializer. I know flat files are simple, but I did want to find something that allowed schema-like validations to be declared and enforced. The projects I saw were either abandoned or too simple. Does anyone know of a good open source flat file API? Thanks!
- Saish

Ok. That one is nice, especially in terms of defining header and trailer records. I also found this one http://sourceforge.net/projects/flatworm/ which allows for multi-line flat-files, which is also nice.
The problem is both only deal with parsing. Granted, writing a serializer even via reflection would not be all that tough. But a few Dukes for you!
- Saish

FIDCC1 Validations / Substitutions and flat files?

I am investigating the potential for two different types of functionality from using the FIDCC1 IDOC for transferring financial postings between two separate SAP FICO systems. My understanding at this point in time is that this IDOC won't perform any of custom or standard substitutions or validations of the receiving system. Is this correct? can anyone confirm? Also, I understand that some IDOC will generate a flat file on the receiving system in addition to the posting of data there. Will FIDCC1 do this as well?
My apologies if this information is readily available elsewhere. I need to obtain these answer quickly and find that I can usually get rapid answers from the SAP Development Community.
Thanks,
David

Ok. That one is nice, especially in terms of defining header and trailer records. I also found this one http://sourceforge.net/projects/flatworm/ which allows for multi-line flat-files, which is also nice.
The problem is both only deal with parsing. Granted, writing a serializer even via reflection would not be all that tough. But a few Dukes for you!
- Saish

Flat File Import, Ignore Missing Columns?

The text files I'm importing always contain a fixed set of columns for example total number of full set of columns is 60, or a subset of those columns (some csv contain 40 columns, some contain 30 or 20 or any other number.) . I would like to import
these csv based on the column header inside the each csv, if it is a subset of full column set then the missing columns can be ignored with null value.
At the moment in SQL 2012, if I import a subset of columns in the csv file, the data doesn't import...I assume because the actual file doesn't include every column defined in the flat file source object?
Is it possible to accomplish this without dynamically selecting the columns, or using script component?
Thanks for help.
Sea Cloud

If the columns coming are dynamic, then you might have to first get it into staging table with a single column reading entire row contents onto that single column and parse out the column information using string parsing logic as below
http://visakhm.blogspot.in/2010/02/parsing-delimited-string.html
This will help you to understand what columns are present based on which you can do the insertion to your table.
Please Mark This As Answer if it helps to solve the issue Visakh ---------------------------- http://visakhm.blogspot.com/ https://www.facebook.com/VmBlogs

Import flat file to multiple tables based on identifier column

Hello,
I am trying to setup a package that will import one pipe-delimited flat file (a utility bill) to multiple data tables based on the value of the first column. I have been told it is similar in format to an EDI file, but there are some differences.
The number of columns is consistent where the first columns are the same. Meaning a record that has '00' in the first column will always have 10 columns; a record that has '01' in the first column will always have 9 columns; etc.
Each value in the first column represents a separate destination data table. Meaning a record that has '00' in the first column should be output to table '00'; a record that has '01' in the first column should be output to table '01'; etc. All
destination tables reside on the same SQL Server.
Identifier columns can repeat multiple times throughout the flat file. Meaning a record that starts with '01' may be repeated multiple times in the same.
Sample Data:
00|XXXXXXXX|XXX|XXXXXXXX|XXXXXX|XXXX|X|XXXXXXXXXX|XX|XXXXX
01|XXXXXXXXXXX|XXX|XXXXXXXX|XXXXX|XXXXXXXXXXXXXXXXXXXX|XXXXXXXXXX|XXXXXXX|XXXXXXXXXXXXXX
02|XXXXXXXXXXX|XXXXXXXX|XXXXXXX|XXXXX|XXXXX|XXXXX|XXXXX|XXXXX|XXXXX|XXXXX|XXXXX
04|XXXXXXXXXXX|XXXXXXXXXXXXX|XXX|XXXXXXXX
05|XXXXXXXXXXX|XXXXXXXXXXXXX|XXX|XXXXXXXX|XXXX
07|XXXXXXXXXXXXX|X|XXXXXXXXXXXXXXX|XXX|XXXXXXXX|XXXX|XXXXXXX|XXXXXXXXXXX
07|XXXXXXXXXXXXX|X|XXXXXXXXXXXXXXX|XXX|XXXXXXXX|XXXX|XXXXXXX|XXXXXXXXXXX
07|XXXXXXXXXXXXX|X|XXXXXXXXXXXXXXX|XXX|XXXXXXXX|XXXX|XXXXXXX|XXXXXXXXXXX
07|XXXXXXXXXXXXX|X|XXXXXXXXXXXXXXX|XXX|XXXXXXXX|XXXX|XXXXXXX|XXXXXXXXXXX
01|XXXXXXXXXXX|XXX|XXXXXXXX|XXXXX|XXXXXXXXXXXXXXXXXXXX|XXXXXXXXXX|XXXXXXX|XXXXXXXXXXXXXX
02|XXXXXXXXXXX|XXXXXXXX|XXXXXXX|XXXXX|XXXXX|XXXXX|XXXXX|XXXXX|XXXXX|XXXXX|XXXXX
04|XXXXXXXXXXX|XXXXXXXXXXXXX|XXX|XXXXXXXX
Any help would be appreciated.

Hi koldar.308,
If there are few distinct values in the first column, we can use Flat File Source connect to that flat file, then use Conditional Split Transformation to split the first column to multiples, and then load the data to multiple tables with OLE DB Destination
based on the outputs of Conditional Split.
After testing the issue in my environment, please refer to the following steps to achieve this requirement:
Drag a Flat File Source connect to that flat file with Flat File Connection Manager.
Drag a Conditional Split Transformation connects to the Flat File Source.
Double-click the Conditional Split Transformation, add several Output based on the first column values as below:
Drag same number OLE DB Destinations as the outputs of Conditional Split, connect to Conditional Split with one case output:
If there are any other questions, please feel free to ask.
Thanks,
Katherine Xiong
If you have any feedback on our support, please click
here.
Katherine Xiong
TechNet Community Support

Copying TEXT column from flat file into SQL results in empty fields....

I'm copying TEXT column from SQL to Flat file (ragged right fixed width) - DT_TEXT. It copies fine and I''ve checked the output file. Now, while trying to copy the flat file to sql server in another system. I find all the fields are empty.
When I preview the source from flatfile, I see all the entries there. But it copies other fields but the field with DT_TEXT is empty.
This is when I preview
SQL Table output
Any help will be helpful!!!

Hi, I'm not sure If I'm understanding what you're saying. The data got copied from SQL to Flat file. I've double checked the flat file and I see the DT_TEXT data there. The size of file also gives an indication that there's existence of TEXT Data
But, when I copy that data back to sql again (which I can also preview before the load), The DT_TEXT values goes missing. Same when I copy to Excel as CSV or as a Flat file. I don't see text data.
The TEXT data resides on the first output. But when I try to extract to other format from that output, it doesn't come out.

Split flat file column data into multiple columns using ssis

Hi All, I need one help in SSIS.
I have a source file with column1, I want to split the column1 data into
multiple columns when there is a semicolon(';') and there is no specific
length between each semicolon,let say..
Column1:
John;Sam;Greg;David
And at destination we have 4 columns let say D1,D2,D3,D4
I want to map
John -> D1
Sam->D2
Greg->D3
David->D4
Please I need it ASAP
Thanks in Advance,
RH
sql

Imports System
Imports System.Data
Imports System.Math
Imports Microsoft.SqlServer.Dts.Pipeline.Wrapper
Imports Microsoft.SqlServer.Dts.Runtime.Wrapper
Imports System.IO
Public Class ScriptMain
Inherits UserComponent
Private textReader As StreamReader
Private exportedAddressFile As String
Public Overrides Sub AcquireConnections(ByVal Transaction As Object)
Dim connMgr As IDTSConnectionManager90 = _
Me.Connections.Connection
exportedAddressFile = _
CType(connMgr.AcquireConnection(Nothing), String)
End Sub
Public Overrides Sub PreExecute()
MyBase.PreExecute()
textReader = New StreamReader(exportedAddressFile)
End Sub
Public Overrides Sub CreateNewOutputRows()
Dim nextLine As String
Dim columns As String()
Dim cols As String()
Dim delimiters As Char()
delimiters = ",".ToCharArray
nextLine = textReader.ReadLine
Do While nextLine IsNot Nothing
columns = nextLine.Split(delimiters)
With Output0Buffer
cols = columns(1).Split(";".ToCharArray)
.AddRow()
.ID = Convert.ToInt32(columns(0))
If cols.GetUpperBound(0) >= 0 Then
.Col1 = cols(0)
End If
If cols.GetUpperBound(0) >= 1 Then
.Col2 = cols(1)
End If
If cols.GetUpperBound(0) >= 2 Then
.Col3 = cols(2)
End If
If cols.GetUpperBound(0) >= 3 Then
.Col4 = cols(3)
End If
End With
nextLine = textReader.ReadLine
Loop
End Sub
Public Overrides Sub PostExecute()
MyBase.PostExecute()
textReader.Close()
End Sub
End Class
Put this code in ur script component. Before that add 5 columns to the script component output and name them as ID, col1, co2..,col4. ID is of data type int. Create a flat file destination and name it as connection and point it to the flat file as the source.
Im not sure whats the delimiter in ur flat file between the 2 columns. I have use a comma change it accordingly.
This is the output I get:
ID Col1
Col2 Col3
Col4
1 john
Greg David
Sam
2 tom
tony NULL
NULL
3 harry
NULL NULL
NULL

Error when exporting to flat file in ODI 11g

This works ok in ODI 10g. I'm using IKM SQL to File Append on Windows Server 2008 R2
Getting the following error when exporting to a flat file in ODI 11g: ODI-40406: Bytes are too big for array
I've seen a couple of threads like this on the forum, but I've never seen an answer to the problem.

Problem is with the difference in behaviour for the IKM SQL to File Append KM between 10g and 11g.
Our 10g target file datastore had a mixture of fixed string and numeric columns. Mapping from the source to target was simple one to one column mapping. It generated the desired fixed format flat file; numerics were right adjusted with embedded decimal point and leading spaces. Each numeric column in the generated flat file occupied the exact space allocated to it in the layout. This gave the desired results, even though documentation for the 10g IKM states that all columns in the target must be string.
When we converted to 11g and tried to run this interface, it generated an error on the "numeric" columns because it was wrapping them in quotation marks. The result column was being treated as string, and it was larger than the defined target once it acquired those quotation marks.
In order to get 11g to work, I had to change all the numeric columns in the target data store to fixedstring 30. I then had to change the mapping for these numeric columns to convert them to right adjusted character strings (i.e. RIGHT(SPACE(30) + RTRIM(MyNumericColumn),30).
Now it works.

Flat File Destination: Broken Rows

I am exporting data to a csv file. One of the columns has values with so mixed characters which end up confusing the system and breaking the rows before the real end of the row. What I mean is; on the flat file a resultant single row can be broken down into
two or more rows as a result of the characters in the offending column. I work for the healthcare industry. Please see the example of what I’m talking about below. This is a real value from one of the rows in my table which I have and this value(amongst many
more under tis column) is causing the affected row to be split into two rows. Where the letters
OE: is coming up as a new row on my flat file. I want all this to be in the same row. My column delimiter is double dagger and my row delimiter is {CR}{LF}
Example value:
Column name: Comments
Value:
BIBA: blah blah,
accidentally
kicked
brick
wall
to
lateral
aspect
to
R
foot.
reports
hearing
a
crack.
mobilised
after
injury.
pain
++
swelling
+
nil
obvious
deformity.
minimal
movement
of
the
toes.
LAS
obs:
entonox
to
good
affect
RR18 98%air
P66reg 114/75
5.3mmols
T36.8
pain 8-4/10
PMH:
Nil
Allergies-
nuts&prawns
OE:
swelling
and
pain
to
lateral
and
medial
malleolus
and 5th
metatarsals.
For
xray
Mpumelelo

I think I have managed to get a solution to my problem.
Instead of using csv file I have decided to use text file as my destination flat file format when exporting the data
I have used Tab{t} as my column delimiter
I have left the default system specified {CR}{LF} for row delimiter
On my flat file connection manager, I have gone to the properties and changed
AlwaysCheckForRowDelimiters from True to False.
After a good fight the above worked. No more broken or split rows :).
If anyone has a better approach or suggestion it will be gladly welcome.
Many thanks,
Mpumelelo

Reading a semi column delimited multi line Flat file on KM repository

Similar Messages

Maybe you are looking for