Processing flat file within ODI

Friends,
I need to read a flat file consisting of 'detail or main record' and 'trailier Record
1) Detail Record consist of each transaction details including amount etc.
2) Trailer Record consist of total value on detail records ( number of records and total amount)
How can I verify the total amount in trailer against sum of all detail records?
Thanks
KS

hehehehe
Was a "bad way" to say "execute the calculations".
An way to do it is:
1) create a ODI variable
2) at refresh tab put something like
select t.col_trailier_sum_amount - sum(d.col_amount)
from loaded_tabledetail d, loaded_table_trailier t
where d.record_type = 'detail' and t.record_type = 'trailer"
group by t.col_trailier_sum_amount
* I'm using 2 tables at this example but is the same with just one...
3) create a package and drag and drop:
a) the interface that load the temp table
b) the variable in Refresh mode
c) the variable in evaluate mode ( set the evaluate operator to "=" and value to "0")
d) choose the OK (true) and KO (false) path that you need to each case
Does it make sense to you?

Similar Messages

  • Unable to process flat-files without delimiters in ODI 11.1.1.7

    Hi,
    We have a requirement to process a flat file using ODI 11.1.1.7 (installed on Windows and mounted on Oracle 11g Database).  The flat-file is a normal ASCII file coming from mainframes machine and has no delimiters (column as well as rows).  The first 48 characters is ROW 1, from character 49 to 96 is ROW 2 and so on.  I am unable to create a DataStore for this file in ODI.  If I reverse engineer using the file RKM, it is creating a datastore with over 2000 columns which is not the case (actual file layout given below).  The absence of a row delimiter is making ODI think that the entire file has only one single row. 
    I am also unable to create a DataStore manually. If I am not providing any value for the "row delimiter" it is throwing me an error.  However, I am able to process this file using SQL*Loader (there is a FIX option available where we can specify the length of a row). 
    Is there a way to do it from within ODI?  I tried to modify a KM to create a control file for SQL*Loader and execute it but the control file is not getting generated as expected.
    File Layout:
    Column 1 - Warrant Number (Position 1 - 9) - Number
    Column 2 - Type (Position 10-10) - Number
    Column 3 - Warrant Amount (Position 11-18) - Packed Decimal Signed
    Column 4 - Issue Fund (Position 19-22) - String
    Column 5 - Issue Sub Fund (Position 23-24) - String
    Column 6 - Filler (Position 25-48) - String
    Thanks
    Srivatsan P

    Hi LuizFilipe,
    I tried also with your method but i am getting below error when i am trying to view data.
    See com.borland.dx.dataset.DataSetException error code:  BASE+62
    com.borland.dx.dataset.DataSetException: Execution of query failed.
    Chained exception:
    java.sql.SQLException: ODI-40439: Could not read heading rows from file
      at com.sunopsis.jdbc.driver.file.FileResultSet.<init>(FileResultSet.java:164)
      at com.sunopsis.jdbc.driver.file.impl.commands.CommandSelect.execute(CommandSelect.java:57)
      at com.sunopsis.jdbc.driver.file.CommandExecutor.executeCommand(CommandExecutor.java:33)
      at com.sunopsis.jdbc.driver.file.FilePreparedStatement.executeQuery(FilePreparedStatement.java:135)
      at com.borland.dx.sql.dataset.o.f(Unknown Source)
      at com.borland.dx.sql.dataset.QueryProvider.e(Unknown Source)
      at com.borland.dx.sql.dataset.JdbcProvider.provideData(Unknown Source)
      at com.borland.dx.dataset.StorageDataSet.refresh(Unknown Source)
      at com.borland.dx.sql.dataset.QueryDataSet.refresh(Unknown Source)
      at com.sunopsis.graphical.frame.edit.AbstractEditFrameGridBorland.initialize(AbstractEditFrameGridBorland.java:628)
      at com.sunopsis.graphical.frame.edit.AbstractEditFrameGridBorland.<init>(AbstractEditFrameGridBorland.java:869)
      at com.sunopsis.graphical.frame.edit.EditFrameTableData.<init>(EditFrameTableData.java:50)
      at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
      at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
      at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
      at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
      at oracle.odi.ui.editor.AbstractOdiEditor$1.run(AbstractOdiEditor.java:176)
      at oracle.ide.dialogs.ProgressBar.run(ProgressBar.java:656)
      at java.lang.Thread.run(Thread.java:662)
    Well i created file datastore with selected below things
    File format : fixed
    Heading(number of lines) : 0
    field separator : null
    record separator : by default --> MS-DOS
    text delimiter: (blank)
    decimal separator : (blank)
    then when i did reverse engineering, able to see the row of data and then i separate the data with c1,c2,c3,c4
    please let me know did i miss something.
    Thanks
    Himanshu

  • Error when exporting to flat file in ODI 11g

    This works ok in ODI 10g. I'm using IKM SQL to File Append on Windows Server 2008 R2
    Getting the following error when exporting to a flat file in ODI 11g: ODI-40406: Bytes are too big for array
    I've seen a couple of threads like this on the forum, but I've never seen an answer to the problem.

    Problem is with the difference in behaviour for the IKM SQL to File Append KM between 10g and 11g.
    Our 10g target file datastore had a mixture of fixed string and numeric columns. Mapping from the source to target was simple one to one column mapping. It generated the desired fixed format flat file; numerics were right adjusted with embedded decimal point and leading spaces. Each numeric column in the generated flat file occupied the exact space allocated to it in the layout. This gave the desired results, even though documentation for the 10g IKM states that all columns in the target must be string.
    When we converted to 11g and tried to run this interface, it generated an error on the "numeric" columns because it was wrapping them in quotation marks. The result column was being treated as string, and it was larger than the defined target once it acquired those quotation marks.
    In order to get 11g to work, I had to change all the numeric columns in the target data store to fixedstring 30. I then had to change the mapping for these numeric columns to convert them to right adjusted character strings (i.e. RIGHT(SPACE(30) + RTRIM(MyNumericColumn),30).
    Now it works.

  • How to extract data from Essbase to Flat File using ODI

    Hi,
    Anyone know how to extract data from Essbase to Flat File using ODI?
    Thanks in advance.
    Regards,
    Sumardi

    Hi,
    Have you read through :-
    Oracle Data Integrator Adapter for Hyperion Essbase Getting Started - http://download.oracle.com/docs/cd/E10530_01/doc/epm.931/odiess_getting_started.pdf
    Oracle Data Integrator Adapter for Hyperion Essbase User's Guide - http://download.oracle.com/docs/cd/E10530_01/doc/epm.931/odiess_users.pdf
    If you have read them and are still have a problem at what stage are you having the issues?
    Cheers
    John
    http://john-goodwin.blogspot.com/

  • Extracting data from Essbase & loading into flat file through ODI

    Hi,
    I want to extract data from Essbase and load it into a flat file through ODI(for extraction from essbase I'm using a report script) and I’m using these KM’s:- LKM Hyperion Essbase data to SQL,IKM SQL to FILE Append & for reversing I’m using RKM Hyperion Essbase.All the mappings have been done and the interface has been made. But when I’m executing the interface it is throwing the error below:-
    ODI-1217: Session ESS_FILEI (114001) fails with return code 7000.
    ODI-1226: Step ESS_FILEI fails after 1 attempt(s).
    ODI-1240: Flow ESS_FILEI fails while performing a Loading operation. This flow loads target table ESS_FILE.
    ODI-1228: Task SrcSet0 (Loading) fails on the target FILE connection FILE_PS_ODI.
    Caused By: java.sql.SQLException: ODI-40417: An IOException was caught while creating the file saying The system cannot find the path specified
    at com.sunopsis.jdbc.driver.file.impl.commands.CommandCreateTable.execute(CommandCreateTable.java:62)
    at com.sunopsis.jdbc.driver.file.CommandExecutor.executeCommand(CommandExecutor.java:33)
    at com.sunopsis.jdbc.driver.file.FilePreparedStatement.execute(FilePreparedStatement.java:178)
    at oracle.odi.runtime.agent.execution.sql.SQLCommand.execute(SQLCommand.java:163)
    at oracle.odi.runtime.agent.execution.sql.SQLExecutor.execute(SQLExecutor.java:102)
    at oracle.odi.runtime.agent.execution.sql.SQLExecutor.execute(SQLExecutor.java:1)
    at oracle.odi.runtime.agent.execution.TaskExecutionHandler.handleTask(TaskExecutionHandler.java:50)
    at com.sunopsis.dwg.dbobj.SnpSessTaskSql.processTask(SnpSessTaskSql.java:2906)
    at com.sunopsis.dwg.dbobj.SnpSessTaskSql.treatTask(SnpSessTaskSql.java:2609)
    at com.sunopsis.dwg.dbobj.SnpSessStep.treatAttachedTasks(SnpSessStep.java:537)
    at com.sunopsis.dwg.dbobj.SnpSessStep.treatSessStep(SnpSessStep.java:453)
    at com.sunopsis.dwg.dbobj.SnpSession.treatSession(SnpSession.java:1740)
    at oracle.odi.runtime.agent.processor.impl.StartSessRequestProcessor$2.doAction(StartSessRequestProcessor.java:338)
    at oracle.odi.core.persistence.dwgobject.DwgObjectTemplate.execute(DwgObjectTemplate.java:214)
    at oracle.odi.runtime.agent.processor.impl.StartSessRequestProcessor.doProcessStartSessTask(StartSessRequestProcessor.java:272)
    at oracle.odi.runtime.agent.processor.impl.StartSessRequestProcessor.access$0(StartSessRequestProcessor.java:263)
    at oracle.odi.runtime.agent.processor.impl.StartSessRequestProcessor$StartSessTask.doExecute(StartSessRequestProcessor.java:822)
    at oracle.odi.runtime.agent.processor.task.AgentTask.execute(AgentTask.java:123)
    at oracle.odi.runtime.agent.support.DefaultAgentTaskExecutor$2.run(DefaultAgentTaskExecutor.java:82)
    at java.lang.Thread.run(Thread.java:619)
    Please let me know what I'm missing and how I can resolve this error.
    Thanks

    It seems that you are trying to use the file as your staging areas. Hyperion LKM extracts essbase data into a DB staging area which can then be used by your file IKM to load it into file.
    You need to use a RDBMS for your staging area.

  • DB2 Version 8 to Flat Files using ODI

    Hi,
    We are trying to load data from DB2 V8 to flat files using ODI.
    Can anybody help me in this to configure server in Topology.
    Thanks
    Ashwini

    Hi,
    How are you? Let me try contribute a little...
    A)
    1) The OS isn't a problem, once you link it at topology will be OK.
    2) If you have a fix name just create a datastore to it, if not, there are techniques to got dynamic file names
    3) When you say "*.dat, *.cvs" maybe you refer to the internal format of file (record separator, decimal point, etc) all of this is configurable on datastore that represents the file at ODI.
    B) Yes, the only requisite is to have the write and read rights.
    C) The most simple way is create a refresh variable that get the data in any format you wish..
    Does it make any sense to you?
    Cezar Santos

  • Merge/Concat Multiple Columns in a Flat-File wthin ODI

    Hi,
    I was wondering if there is a quick way to merge/concat multiple columns in a Flat-File from within ODI, a functionality kinda similar to Essbase load rule files for join/split columns etc.
    Please let me know.
    Thanks

    Looks like the CONCAT function works only with Memory Engine. Could someone please confirm this? If this is true, I guess applying the patch is the only resolution.
    Thanks

  • Process Flat Files

    Hi everyone, 
    I currently need to process about 400 txt files, equivalent to about 1TB of data in maximum 5-6 hours. I need to run a very simple scrip, where I would capture about 0.10% of the data. Something like a select function with some where clause. Originally I
    was thinking to import this data in a SQL server however I'm not sure my computer could handle the workload in a reasonable time frame. 
    What would you suggest? Is there any other products that Microsoft offers to process extremely large flat files?
    Thank you in advance.
    Cheers,

    Hi AlexB0865,
    The solution you have mentioned should be the best one in my opinion. The performance bottleneck in this case are:
    Load data from 400 text files, which is about 1 TB large. Each file should be about 2.5GB
    Bulk insert data to SQL Server
    Filter data from the 1TB data
    For the first chanllange, it won't be an issue in SSIS per my testing. I have a file with 23504761 rows, which is 2.5GB large. This file can be imported into a SQL Server table within 1 minute.
    For the second chanllange, we can split the bulk insert to improve the performance.
    For the third chanllange, with proper index(es) created, the performance won't be a problem.
    Henk details the testing result regarding this same scenario in blog:
    http://henkvandervalk.com/speeding-up-ssis-bulk-inserts-into-sql-server
    If you have any more question, please feel free to ask.
    Thanks,
    Jinchun Chen

  • How to export data from DB to flat file with ODI?

    we want to export DB2 tables to flat files.
    Who can tell me how to do it with ODI?
    Please give me a simple example or steps.
    Thank you.

    There are two ways
    Either use IKM Sql to File Append
    (or)
    OdiSqlUnload
    For OdiSqlUnload you can use this technique to provide the connection parameters - http://odiexperts.com/?p=1985

  • How to Process flat File in Oracle Apps through Concurrent Program

    Hello Everyone,
    My client has a request, to process a bank file (Lockbox) which is a flat file that will be copied on UNIX box and I will have to create a new concurrent request that will process this flat file and will update receipt information in Oracle Apps database tables.
    Could you please suggest, if there are any other standard Oracle Apps functions (Example FND) available which can be used through Concurrent program that can be used to open a file from a particular directory and can be read from the flat file and after processing this file can be closed.
    Please let me know, if you have a small example, that would help me a lot.
    Thanks

    There are base concurrent programs in Accts Receivable that do consume lockbox flat files. Pl see the AR Setup/User Guides at
    http://download.oracle.com/docs/cd/B40089_10/current/html/docset.html
    Srini

  • Can SAP XI AS2 Adapter process flat files? Pl. tell me how to configure it

    I have a requirement to read and write delimited ASCII flat files using the AS2 adapter in SAP XI. The AS2 adapter should be able to read from delimited ASCII Flat files and also write to delimited ASCII flat files. Kindly let me know how to configure the AS2 adapter so that it can read and write delimited flat files. I badly need to know this configuration for a client requirement which requires file processing by the AS2 adapter in XI 7.0. Thanks in advance

    Hi,
    Is the requirement is to just read the flat file and process it ? if so, u can use File Adapter provided , ur source system provides u the data in the file format in the some directory
    following thread discussions may help u -AS2 adapter-
    Seebuger AS2 adapter for XI as sender for multiple messages
    Seeburger AS2 Adapter
    Hope this helps,
    Regards,
    Morthy

  • Comparing Files within ODI

    Friends,
    I have typical scenario here and wonder if anyone can help/suggest any solution:
    I got 2 different files types being loaded into 2 different tables and data needs to be compared before further processing. each file (type A) corresponds to other file (type B) when it comes to DATA. Now I am able to to load the file and compare the files all right but my problem is as follows:
    Each file of TYPE A corresponds to file from TYPE B (w.r.t data).
    For example:
    I have one set of files contains
    STD_010908.TXT
    STD_020908.TXT
    STD_030908.TXT
    Other set contains:
    RST_010908.TXT
    RST_020908.TXT
    RST_030908.TXT
    Now STD_010908.TXT corresponds to RST_010908.TXT, STD_020908.TXT corresponds to RST_020908.TXT etc...
    All files are located in same directory and gets loaded by ODI package.
    NOW QUESTION:
    USING ODI, how would I make sure that file STD_010908.TXT gets compared to RST_010908.TXT and likewise futher ?
    Regards
    KS

    Hi,
    You can use Joins for comparing your source files. Drag and drop your fliles A and B and put a join in the PK of that files, then map that PK to your target column.
    Thanks,
    G

  • I am trying to extract metadata from essbase into a flat file using ODI.

    I have 2 questions in this regard :
    Some of the account members storage property is missing in the extract.the reason which i suspect is due to the parent child sorting that did not happen while extracting.How do i do this.I do not see this option when i select IKM hyperion Essbase Metadata to SQL....
    I have many account members that have more than one UDA's upto 5 UDA's.But in my extrcat only one UDA appears.How do i incorporate all the UDA's in a single column,sperated by a comma,the extract file as such is semicolon seperated,mainly because of this reason and some alias descriptions have comma's in the source system
    ODi is extracting metadata in a descending order.How do i change it to sort records in parent child order
    Thanks,
    Lingaraj
    Edited by: user649227 on 2009/06/10 6:50 AM

    Hi,
    There was an issue with early versions of the KM around the storage property, this has since been resolved. I recommend upgrading to the latest release of ODI or have a look through the patches.
    Cheers
    John
    http://john-goodwin.blogspot.com/

  • Loading data from flat file...

    Hello,
    I am actually experiencing a problem where I cant load data from a flat file into a table. I have actually reverse engineered the flat file into ODI. But the thing is I don't know how to load the data that is reversed into a RDBMS table. Added, I don't know how create this RDBMS table form within ODI to be reflected on the DB. Added, don't know how to load the data from the flat file onto this table without having to add the columns in order to map between the flat file and the table.
    In conclusion, I need to know how to create an RDBMS table from within ODI on the database. And how to automatically map the flat file to the DB table and load the data into the DB table.
    Regards,
    Hossam

    Hi Hossam,
    We can used ODI procedure to create table in the DB.
    Make sure you keep the column name in the table name same as the column name in FLAT FILE so that it can automatically map the column.
    and regarding Loading data from FLAT File i.e. our source table is FLAT FILE till ODI 10.1.3.4 we need to manually insert the datastore since the file system cannot be reversed.
    Please let me know Hossam if i can assis you further.
    Thanks and Regards,
    Andy

  • Idoc data in flat file structure

    Dear Experts,
             We have  idoc data in flat file structure. We need to fetch it using ftp and map it to an idoc.
    please tell me how to proceed .
    Thanks,
    Aju

    Hi,
    For flat file you need to use the File content conversion parameters.
    Refer the blog,
    SAP Network Blog: How to process flat files with multiple documents like POs, SOs etc. in a File to IDoc scenario
    How to process flat files with multiple documents like POs, SOs etc. in a File to IDoc scenario
    Thanks
    Swarup

Maybe you are looking for