Processing flat file within ODI
Friends,
I need to read a flat file consisting of 'detail or main record' and 'trailier Record
1) Detail Record consist of each transaction details including amount etc.
2) Trailer Record consist of total value on detail records ( number of records and total amount)
How can I verify the total amount in trailer against sum of all detail records?
Thanks
KS
hehehehe
Was a "bad way" to say "execute the calculations".
An way to do it is:
1) create a ODI variable
2) at refresh tab put something like
select t.col_trailier_sum_amount - sum(d.col_amount)
from loaded_tabledetail d, loaded_table_trailier t
where d.record_type = 'detail' and t.record_type = 'trailer"
group by t.col_trailier_sum_amount
* I'm using 2 tables at this example but is the same with just one...
3) create a package and drag and drop:
a) the interface that load the temp table
b) the variable in Refresh mode
c) the variable in evaluate mode ( set the evaluate operator to "=" and value to "0")
d) choose the OK (true) and KO (false) path that you need to each case
Does it make sense to you?
Similar Messages
-
Unable to process flat-files without delimiters in ODI 11.1.1.7
Hi,
We have a requirement to process a flat file using ODI 11.1.1.7 (installed on Windows and mounted on Oracle 11g Database). The flat-file is a normal ASCII file coming from mainframes machine and has no delimiters (column as well as rows). The first 48 characters is ROW 1, from character 49 to 96 is ROW 2 and so on. I am unable to create a DataStore for this file in ODI. If I reverse engineer using the file RKM, it is creating a datastore with over 2000 columns which is not the case (actual file layout given below). The absence of a row delimiter is making ODI think that the entire file has only one single row.
I am also unable to create a DataStore manually. If I am not providing any value for the "row delimiter" it is throwing me an error. However, I am able to process this file using SQL*Loader (there is a FIX option available where we can specify the length of a row).
Is there a way to do it from within ODI? I tried to modify a KM to create a control file for SQL*Loader and execute it but the control file is not getting generated as expected.
File Layout:
Column 1 - Warrant Number (Position 1 - 9) - Number
Column 2 - Type (Position 10-10) - Number
Column 3 - Warrant Amount (Position 11-18) - Packed Decimal Signed
Column 4 - Issue Fund (Position 19-22) - String
Column 5 - Issue Sub Fund (Position 23-24) - String
Column 6 - Filler (Position 25-48) - String
Thanks
Srivatsan PHi LuizFilipe,
I tried also with your method but i am getting below error when i am trying to view data.
See com.borland.dx.dataset.DataSetException error code: BASE+62
com.borland.dx.dataset.DataSetException: Execution of query failed.
Chained exception:
java.sql.SQLException: ODI-40439: Could not read heading rows from file
at com.sunopsis.jdbc.driver.file.FileResultSet.<init>(FileResultSet.java:164)
at com.sunopsis.jdbc.driver.file.impl.commands.CommandSelect.execute(CommandSelect.java:57)
at com.sunopsis.jdbc.driver.file.CommandExecutor.executeCommand(CommandExecutor.java:33)
at com.sunopsis.jdbc.driver.file.FilePreparedStatement.executeQuery(FilePreparedStatement.java:135)
at com.borland.dx.sql.dataset.o.f(Unknown Source)
at com.borland.dx.sql.dataset.QueryProvider.e(Unknown Source)
at com.borland.dx.sql.dataset.JdbcProvider.provideData(Unknown Source)
at com.borland.dx.dataset.StorageDataSet.refresh(Unknown Source)
at com.borland.dx.sql.dataset.QueryDataSet.refresh(Unknown Source)
at com.sunopsis.graphical.frame.edit.AbstractEditFrameGridBorland.initialize(AbstractEditFrameGridBorland.java:628)
at com.sunopsis.graphical.frame.edit.AbstractEditFrameGridBorland.<init>(AbstractEditFrameGridBorland.java:869)
at com.sunopsis.graphical.frame.edit.EditFrameTableData.<init>(EditFrameTableData.java:50)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at oracle.odi.ui.editor.AbstractOdiEditor$1.run(AbstractOdiEditor.java:176)
at oracle.ide.dialogs.ProgressBar.run(ProgressBar.java:656)
at java.lang.Thread.run(Thread.java:662)
Well i created file datastore with selected below things
File format : fixed
Heading(number of lines) : 0
field separator : null
record separator : by default --> MS-DOS
text delimiter: (blank)
decimal separator : (blank)
then when i did reverse engineering, able to see the row of data and then i separate the data with c1,c2,c3,c4
please let me know did i miss something.
Thanks
Himanshu -
Error when exporting to flat file in ODI 11g
This works ok in ODI 10g. I'm using IKM SQL to File Append on Windows Server 2008 R2
Getting the following error when exporting to a flat file in ODI 11g: ODI-40406: Bytes are too big for array
I've seen a couple of threads like this on the forum, but I've never seen an answer to the problem.Problem is with the difference in behaviour for the IKM SQL to File Append KM between 10g and 11g.
Our 10g target file datastore had a mixture of fixed string and numeric columns. Mapping from the source to target was simple one to one column mapping. It generated the desired fixed format flat file; numerics were right adjusted with embedded decimal point and leading spaces. Each numeric column in the generated flat file occupied the exact space allocated to it in the layout. This gave the desired results, even though documentation for the 10g IKM states that all columns in the target must be string.
When we converted to 11g and tried to run this interface, it generated an error on the "numeric" columns because it was wrapping them in quotation marks. The result column was being treated as string, and it was larger than the defined target once it acquired those quotation marks.
In order to get 11g to work, I had to change all the numeric columns in the target data store to fixedstring 30. I then had to change the mapping for these numeric columns to convert them to right adjusted character strings (i.e. RIGHT(SPACE(30) + RTRIM(MyNumericColumn),30).
Now it works. -
How to extract data from Essbase to Flat File using ODI
Hi,
Anyone know how to extract data from Essbase to Flat File using ODI?
Thanks in advance.
Regards,
SumardiHi,
Have you read through :-
Oracle Data Integrator Adapter for Hyperion Essbase Getting Started - http://download.oracle.com/docs/cd/E10530_01/doc/epm.931/odiess_getting_started.pdf
Oracle Data Integrator Adapter for Hyperion Essbase User's Guide - http://download.oracle.com/docs/cd/E10530_01/doc/epm.931/odiess_users.pdf
If you have read them and are still have a problem at what stage are you having the issues?
Cheers
John
http://john-goodwin.blogspot.com/ -
Extracting data from Essbase & loading into flat file through ODI
Hi,
I want to extract data from Essbase and load it into a flat file through ODI(for extraction from essbase I'm using a report script) and I’m using these KM’s:- LKM Hyperion Essbase data to SQL,IKM SQL to FILE Append & for reversing I’m using RKM Hyperion Essbase.All the mappings have been done and the interface has been made. But when I’m executing the interface it is throwing the error below:-
ODI-1217: Session ESS_FILEI (114001) fails with return code 7000.
ODI-1226: Step ESS_FILEI fails after 1 attempt(s).
ODI-1240: Flow ESS_FILEI fails while performing a Loading operation. This flow loads target table ESS_FILE.
ODI-1228: Task SrcSet0 (Loading) fails on the target FILE connection FILE_PS_ODI.
Caused By: java.sql.SQLException: ODI-40417: An IOException was caught while creating the file saying The system cannot find the path specified
at com.sunopsis.jdbc.driver.file.impl.commands.CommandCreateTable.execute(CommandCreateTable.java:62)
at com.sunopsis.jdbc.driver.file.CommandExecutor.executeCommand(CommandExecutor.java:33)
at com.sunopsis.jdbc.driver.file.FilePreparedStatement.execute(FilePreparedStatement.java:178)
at oracle.odi.runtime.agent.execution.sql.SQLCommand.execute(SQLCommand.java:163)
at oracle.odi.runtime.agent.execution.sql.SQLExecutor.execute(SQLExecutor.java:102)
at oracle.odi.runtime.agent.execution.sql.SQLExecutor.execute(SQLExecutor.java:1)
at oracle.odi.runtime.agent.execution.TaskExecutionHandler.handleTask(TaskExecutionHandler.java:50)
at com.sunopsis.dwg.dbobj.SnpSessTaskSql.processTask(SnpSessTaskSql.java:2906)
at com.sunopsis.dwg.dbobj.SnpSessTaskSql.treatTask(SnpSessTaskSql.java:2609)
at com.sunopsis.dwg.dbobj.SnpSessStep.treatAttachedTasks(SnpSessStep.java:537)
at com.sunopsis.dwg.dbobj.SnpSessStep.treatSessStep(SnpSessStep.java:453)
at com.sunopsis.dwg.dbobj.SnpSession.treatSession(SnpSession.java:1740)
at oracle.odi.runtime.agent.processor.impl.StartSessRequestProcessor$2.doAction(StartSessRequestProcessor.java:338)
at oracle.odi.core.persistence.dwgobject.DwgObjectTemplate.execute(DwgObjectTemplate.java:214)
at oracle.odi.runtime.agent.processor.impl.StartSessRequestProcessor.doProcessStartSessTask(StartSessRequestProcessor.java:272)
at oracle.odi.runtime.agent.processor.impl.StartSessRequestProcessor.access$0(StartSessRequestProcessor.java:263)
at oracle.odi.runtime.agent.processor.impl.StartSessRequestProcessor$StartSessTask.doExecute(StartSessRequestProcessor.java:822)
at oracle.odi.runtime.agent.processor.task.AgentTask.execute(AgentTask.java:123)
at oracle.odi.runtime.agent.support.DefaultAgentTaskExecutor$2.run(DefaultAgentTaskExecutor.java:82)
at java.lang.Thread.run(Thread.java:619)
Please let me know what I'm missing and how I can resolve this error.
ThanksIt seems that you are trying to use the file as your staging areas. Hyperion LKM extracts essbase data into a DB staging area which can then be used by your file IKM to load it into file.
You need to use a RDBMS for your staging area. -
DB2 Version 8 to Flat Files using ODI
Hi,
We are trying to load data from DB2 V8 to flat files using ODI.
Can anybody help me in this to configure server in Topology.
Thanks
AshwiniHi,
How are you? Let me try contribute a little...
A)
1) The OS isn't a problem, once you link it at topology will be OK.
2) If you have a fix name just create a datastore to it, if not, there are techniques to got dynamic file names
3) When you say "*.dat, *.cvs" maybe you refer to the internal format of file (record separator, decimal point, etc) all of this is configurable on datastore that represents the file at ODI.
B) Yes, the only requisite is to have the write and read rights.
C) The most simple way is create a refresh variable that get the data in any format you wish..
Does it make any sense to you?
Cezar Santos -
Merge/Concat Multiple Columns in a Flat-File wthin ODI
Hi,
I was wondering if there is a quick way to merge/concat multiple columns in a Flat-File from within ODI, a functionality kinda similar to Essbase load rule files for join/split columns etc.
Please let me know.
ThanksLooks like the CONCAT function works only with Memory Engine. Could someone please confirm this? If this is true, I guess applying the patch is the only resolution.
Thanks -
Hi everyone,
I currently need to process about 400 txt files, equivalent to about 1TB of data in maximum 5-6 hours. I need to run a very simple scrip, where I would capture about 0.10% of the data. Something like a select function with some where clause. Originally I
was thinking to import this data in a SQL server however I'm not sure my computer could handle the workload in a reasonable time frame.
What would you suggest? Is there any other products that Microsoft offers to process extremely large flat files?
Thank you in advance.
Cheers,Hi AlexB0865,
The solution you have mentioned should be the best one in my opinion. The performance bottleneck in this case are:
Load data from 400 text files, which is about 1 TB large. Each file should be about 2.5GB
Bulk insert data to SQL Server
Filter data from the 1TB data
For the first chanllange, it won't be an issue in SSIS per my testing. I have a file with 23504761 rows, which is 2.5GB large. This file can be imported into a SQL Server table within 1 minute.
For the second chanllange, we can split the bulk insert to improve the performance.
For the third chanllange, with proper index(es) created, the performance won't be a problem.
Henk details the testing result regarding this same scenario in blog:
http://henkvandervalk.com/speeding-up-ssis-bulk-inserts-into-sql-server
If you have any more question, please feel free to ask.
Thanks,
Jinchun Chen -
How to export data from DB to flat file with ODI?
we want to export DB2 tables to flat files.
Who can tell me how to do it with ODI?
Please give me a simple example or steps.
Thank you.There are two ways
Either use IKM Sql to File Append
(or)
OdiSqlUnload
For OdiSqlUnload you can use this technique to provide the connection parameters - http://odiexperts.com/?p=1985 -
How to Process flat File in Oracle Apps through Concurrent Program
Hello Everyone,
My client has a request, to process a bank file (Lockbox) which is a flat file that will be copied on UNIX box and I will have to create a new concurrent request that will process this flat file and will update receipt information in Oracle Apps database tables.
Could you please suggest, if there are any other standard Oracle Apps functions (Example FND) available which can be used through Concurrent program that can be used to open a file from a particular directory and can be read from the flat file and after processing this file can be closed.
Please let me know, if you have a small example, that would help me a lot.
ThanksThere are base concurrent programs in Accts Receivable that do consume lockbox flat files. Pl see the AR Setup/User Guides at
http://download.oracle.com/docs/cd/B40089_10/current/html/docset.html
Srini -
Can SAP XI AS2 Adapter process flat files? Pl. tell me how to configure it
I have a requirement to read and write delimited ASCII flat files using the AS2 adapter in SAP XI. The AS2 adapter should be able to read from delimited ASCII Flat files and also write to delimited ASCII flat files. Kindly let me know how to configure the AS2 adapter so that it can read and write delimited flat files. I badly need to know this configuration for a client requirement which requires file processing by the AS2 adapter in XI 7.0. Thanks in advance
Hi,
Is the requirement is to just read the flat file and process it ? if so, u can use File Adapter provided , ur source system provides u the data in the file format in the some directory
following thread discussions may help u -AS2 adapter-
Seebuger AS2 adapter for XI as sender for multiple messages
Seeburger AS2 Adapter
Hope this helps,
Regards,
Morthy -
Friends,
I have typical scenario here and wonder if anyone can help/suggest any solution:
I got 2 different files types being loaded into 2 different tables and data needs to be compared before further processing. each file (type A) corresponds to other file (type B) when it comes to DATA. Now I am able to to load the file and compare the files all right but my problem is as follows:
Each file of TYPE A corresponds to file from TYPE B (w.r.t data).
For example:
I have one set of files contains
STD_010908.TXT
STD_020908.TXT
STD_030908.TXT
Other set contains:
RST_010908.TXT
RST_020908.TXT
RST_030908.TXT
Now STD_010908.TXT corresponds to RST_010908.TXT, STD_020908.TXT corresponds to RST_020908.TXT etc...
All files are located in same directory and gets loaded by ODI package.
NOW QUESTION:
USING ODI, how would I make sure that file STD_010908.TXT gets compared to RST_010908.TXT and likewise futher ?
Regards
KSHi,
You can use Joins for comparing your source files. Drag and drop your fliles A and B and put a join in the PK of that files, then map that PK to your target column.
Thanks,
G -
I am trying to extract metadata from essbase into a flat file using ODI.
I have 2 questions in this regard :
Some of the account members storage property is missing in the extract.the reason which i suspect is due to the parent child sorting that did not happen while extracting.How do i do this.I do not see this option when i select IKM hyperion Essbase Metadata to SQL....
I have many account members that have more than one UDA's upto 5 UDA's.But in my extrcat only one UDA appears.How do i incorporate all the UDA's in a single column,sperated by a comma,the extract file as such is semicolon seperated,mainly because of this reason and some alias descriptions have comma's in the source system
ODi is extracting metadata in a descending order.How do i change it to sort records in parent child order
Thanks,
Lingaraj
Edited by: user649227 on 2009/06/10 6:50 AMHi,
There was an issue with early versions of the KM around the storage property, this has since been resolved. I recommend upgrading to the latest release of ODI or have a look through the patches.
Cheers
John
http://john-goodwin.blogspot.com/ -
Loading data from flat file...
Hello,
I am actually experiencing a problem where I cant load data from a flat file into a table. I have actually reverse engineered the flat file into ODI. But the thing is I don't know how to load the data that is reversed into a RDBMS table. Added, I don't know how create this RDBMS table form within ODI to be reflected on the DB. Added, don't know how to load the data from the flat file onto this table without having to add the columns in order to map between the flat file and the table.
In conclusion, I need to know how to create an RDBMS table from within ODI on the database. And how to automatically map the flat file to the DB table and load the data into the DB table.
Regards,
HossamHi Hossam,
We can used ODI procedure to create table in the DB.
Make sure you keep the column name in the table name same as the column name in FLAT FILE so that it can automatically map the column.
and regarding Loading data from FLAT File i.e. our source table is FLAT FILE till ODI 10.1.3.4 we need to manually insert the datastore since the file system cannot be reversed.
Please let me know Hossam if i can assis you further.
Thanks and Regards,
Andy -
Idoc data in flat file structure
Dear Experts,
We have idoc data in flat file structure. We need to fetch it using ftp and map it to an idoc.
please tell me how to proceed .
Thanks,
AjuHi,
For flat file you need to use the File content conversion parameters.
Refer the blog,
SAP Network Blog: How to process flat files with multiple documents like POs, SOs etc. in a File to IDoc scenario
How to process flat files with multiple documents like POs, SOs etc. in a File to IDoc scenario
Thanks
Swarup
Maybe you are looking for
-
I bought additional mega bytes for my Ipad2. I was trying to backup my stuff and each time it claimed about availability of space. after buying the space using my credit card,I got an email from apple saying that 'the following changes to your apple
-
Are Multiple Classes with Same name in a single Class valid ?
package inheritance; interface Foo int bar(); public class Inh6 { class A implements Foo // THis is the first class public int bar() return 1; public int fubar(Foo foo) return foo.bar(); pu
-
I have 2 separate charges for $20 each on my bank statement from Itunes. In my purchase history - it isn't showing up. How do I get reimbursed for these charges?
-
Crystal report blocks bex query
Hi, gurus. I'm in such situation - i need to delete BEX query wich is used in crystal reports. Changing database in my report i simply made new crystal report. So i need to delete it from bw. If you know the way please tell me. Any help ll be apprec
-
MM_validateForm errors in Firefox
Hi, I have a website that uses the MM_validateForm script from Dreamweaver. It would seem that it is working fine to validate all the fields, except for the email input field. In IE it ignore it completely (you can submit the form without filling in