Can we cleanse and transform data at flat file or external table level?

Hi,
I have some data that I want to cleanse and transform. I don't want to cleanse it after i populate the external table, I want to get done with it at flat file level or while populating the external table. Can we cleanse and transform data at flat file or external table level through Oracle or OWB 11.2? Is it possible to run a conditional load (i.e. having a where clause or if-else-then) for an external table? Can we call oracle functions for an external table at the time of creation?
Thanks in advance.
Regards,
Ann.

Hi Oleg,
Thanks a lot for the clarification. :)
So is there a way that I can clease the data within the text file through Oracle or OWB? I have datatype mismatches in the data and most of my data is getting rejected because of that. The way I can think of, for solving this problem, is to create the external table with all fields with datatype varchar and then cleansing the data. But it doesn't seem very effecient plus it will get very complicated because I have almost 80-90 fields.
Any help?
Thanks and regards,
Ann.

Similar Messages

How can I get data in flat file from Pool table and cluster table ?

Hi,
I am working in one Achiving project. My requirement is to get data into flat file from Cluster table and pool table.
Is there any tool avilable to download data into flat file from pool table and cluster table ?
if table name given in the selection screen then data will be downloaded into flat file.
waiting for quick response.
Best Regards,
Bansidhar

Data cannot be retrived directly form the cluster table
as the Cluster results are stored in Cluster Key say for example PCLkey
and form that Key we need to fetch the data
these clustes are not the part of PNP or PNPCE tables
for ur info kindly check

How to create the Export Data and Import Data using flat file interface

Hi,
Request to let me know based on the requirement below on how to export and import data using flat file interface.....
Please provide the steps involved for the same.......
BW/BI - Recovery Process for SNP data.
For each SNP InfoProvider,
create:
1) Export Data:
1.a) Create an export data source, InfoPackage, comm structure, etc. necessary to create an ASCII fixed length flat file on the XI
ctnhsappdata\iface\SCPI063\Out folder for each SNP InfoProvider.
1.b) All fields in each InfoProvider should be exported and included in the flat file.
1.c) A process chain should be created for each InfoProvider with a start event.
1.d) If the file exists on the target drive it should be overwritten.
1.e) The exported data file name should include the InfoProvider technical name.
1.f) Include APO Planning Version, Date of Planning Run, APO Location, Calendar Year/Month, Material and BW Plant as selection criteria.
2) Import Data:
2.a) Create a flat file source system InfoPackage, comm structure, etc. necessary to import ASCII fixed length flat files from the XI
ctnhsappdata\iface\SCPI063\Out folder for each SNP InfoProvider.
2.b) All fields for each InfoProvider should be mapped and imported from the flat file.
2.c) A process chain should be created for each InfoProvider with a start event.
2.d) The file should be archived in the
ctnhsappdata\iface\SCPI063\Archive directory. Each file name should have the date appended in YYYYMMDD format. Each file should be deleted from the \Out directory after it is archived.
Thanks in advance.
Tyson

Here's some info on working with plists:
http://developer.apple.com/documentation/Cocoa/Conceptual/PropertyLists/Introduc tion/chapter1_section1.html
They can be edited with any text editor. Xcode provides a graphical editor for them - make sure to use the .plist extension so Xcode will recognize it.

How to write data to text file using external tables

can anybody tell how to write data to text file using external tables concept?

Hi,
Using external table u can load the data in your local table in database,
then using your local db table and UTL_FILE pacakge u can wrrite data to text file
external table
~~~~~~~~~~~
http://download-east.oracle.com/docs/cd/B19306_01/server.102/b14200/statements_7002.htm#i2153251
UTL_FILE
~~~~~~~~~
http://download-east.oracle.com/docs/cd/B19306_01/appdev.102/b14258/u_file.htm#sthref14093
Message was edited by:
Nicloei W
Message was edited by:
Nicloei W

Upload data from flat file into internal table

Hi friends,
I want to upload the data from a flat file into internal table , but the problem is that all the columns in that flat file are seperated by "|" character instead of tabs.
Plz help me out.........

HEllo,
DO like this.
CALL FUNCTION 'GUI_UPLOAD'
EXPORTING
FILENAME = LV_FILENAME
FILETYPE = 'ASC'
HAS_FIELD_SEPARATOR = 'X' " Check here
* HEADER_LENGTH = '1'
* READ_BY_LINE = 'X'
* DAT_MODE = ' '
* CODEPAGE = ' '
* IGNORE_CERR = ABAP_TRUE
* REPLACEMENT = '#'
* CHECK_BOM = ' '
* IMPORTING
* FILELENGTH =
* HEADER =
TABLES
DATA_TAB = IT_COJRNL
EXCEPTIONS
FILE_OPEN_ERROR = 1
FILE_READ_ERROR = 2
NO_BATCH = 3
GUI_REFUSE_FILETRANSFER = 4
INVALID_TYPE = 5
NO_AUTHORITY = 6
UNKNOWN_ERROR = 7
BAD_DATA_FORMAT = 8
HEADER_NOT_ALLOWED = 9
SEPARATOR_NOT_ALLOWED = 10
HEADER_TOO_LONG = 11
UNKNOWN_DP_ERROR = 12
ACCESS_DENIED = 13
DP_OUT_OF_MEMORY = 14
DISK_FULL = 15
DP_TIMEOUT = 16
OTHERS = 17
IF SY-SUBRC <> 0.
MESSAGE ID SY-MSGID TYPE SY-MSGTY NUMBER SY-MSGNO
WITH SY-MSGV1 SY-MSGV2 SY-MSGV3 SY-MSGV4.
ENDIF.
VAsanth

Load data from flat file to target table with scheduling in loader

Hi All,
I have requirement as follows.
I need to load the data to the target table on every Saturday. My source file consists of dataof several sates.For every week i have to load one particular state data to target table.
If first week I loaded AP data, then second week on Saturday karnatak, etc.
can u plz provide code also how can i schedule the data load with every saturday with different state column values automatically.
Thanks & Regards,
Sekhar

The best solution would be:
get the flat file to the Database server
define an External Table pointing to that flat file
insert into destination table(s) as select * from external_table
Loading only single state data each Saturday might mean troubles, but assuming there are valid reasons to do so, you could:
create a job with a p_state parameter executing insert into destination table(s) as select * from external_table where state = p_state
create a Scheduler chain where each member runs on next Saturday executing the same job with a different p_stateparameter
Managing Tables
Oracle Scheduler Concepts
Regards
Etbin

SSIS - Exporting Data into flat files from Oracle Table as batchwise process

Hi All,
Thanks in advance.
I have a Large Table in Oracle Database with some 3 Lakhs record. I need to fetch the 10,000 records for every iteration and export it into the flat file. This process should occur recursively until the table becomes empty.
Hence, For every iteration on flat file to be generated with 10,000 records.
Please help how to proceed further in SSIS.
Thanks
Pyarajan.S

Yes, it always helps if your question doesn't specify the actual requirements...
Use the FOR loop container to control the iterations of the data flow. For each run you read 10,000 rows from the table and dump them in a flat file. Either move the flat file, or use an expression on the flat file connection manager to give them dynamic
file names.
30 million rows is also not a problem by the way, it just takes a bit longer.
MCSE SQL Server 2012 - Please mark posts as answered where appropriate.

Sending data from flat file or oracle table view to a IBM MQ

Hi,
We need a help in developing solution.
We have a scenario, where data(multiple records) in source (table/file) needs to populated into a queue(JMS IBM MQ) as a single message. To achieve this we are trying a two step process.
1)     We created a temp table to store multiple records from file, this temp table is having one column of *‘clob’* data-type which will store data from file into a single row. We are facing issue while loading data from file to staging area using LKM FILE TO ORACLE (SQLLDR)
When we are executing interface, while creating C$ table it is going error out . Instead of taking clob (target write data type) it is taking varchar with +5000+ size whereas varchar supports only *4000*.
create table SNPM.C$_0SINGLERECORD
C1_C1 VARCHAR2(5000) NULL
NOLOGGING
Error message:
910 : 42000 : java.sql.SQLException: ORA-00910: specified length too long for its datatype
java.sql.SQLException: ORA-00910: specified length too long for its datatype
Need help in sending data into destination table.
2) After populating this data into destination(temp table) as a single record we need to send each row as a single message to JMS MQ. currently we are using LKM FILE to SQL to load into Staging area and IKM SQL TO JMS APPEND to put message in JMS MQ. We are succeeding in putting message of length < 4000 as these KMs are converting the data using varchar2, but we have data of large size+(>4K)+.
Pointer/ solution will be of great help.
Please let me know in case you need more description.

The error message that is showing while using clob as datatype(which we created as user data type in data types section of respective technology)
java.lang.NumberFormatException: For input string: "4294967295"
     at java.lang.NumberFormatException.forInputString(Unknown Source)
     at java.lang.Integer.parseInt(Unknown Source)
while using varchar2 in creating C$ table following is the error:
910 : 42000 : java.sql.SQLException: ORA-00910: specified length too long for its datatype
java.sql.SQLException: ORA-00910: specified length too long for its datatype
     at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:125)
     at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:316)

Unable to read chinese characters in a flat file to external table

Hi All,
We have a flat file containing data in chinese. We are using external table to read data in files. When i do select <coulmn-name> from <table> it displays box for the chinese characters. The column is of type varchar2.
The NLS_LANGUAGE is AMERICAN and NLS_CHARACTERSET is AL32UTF8.
The character set is mentioed as UTF8 in external table defnition.
Is the external table not reading these charcters properly? Should any patch be installed or some settings should be changed for sql developer to to display chinese character?

The NLS_LANG environment variable/registry string variable (You are just one of the 1000s of 'anonymous' users refusing to post platform and version), so NOT NLS_LANGUAGE, on the server hosting the file, should be set to anything ending in .AL32UTF8
Sybrand Bakker
Senior Oracle DBA

Flat file created ,external table created, deployed successfully but...

hi everyone,
I have a problems :
I have created a flat file through flat file module from a text file.
i've done it successfully,
then i create an external table through external table module for the same
flat file , went successfully,
then velidation went successfull,
and i carried out deployement for the same, that also done successfully.
but when i tried to check the database through SQL-Plus :
i came across with the following error :
SQL> CONNECT TARGET_SCHEMA/TARGET_SCHEMA
Connected.
SQL> SELECT * FROM CUMMINS_EXT;
SELECT * FROM CUMMINS_EXT
ERROR at line 1:
ORA-29913: error in executing ODCIEXTTABLEOPEN callout
ORA-29400: data cartridge error
KUP-04040: file Cumminsparts.txt in TARGET_LOC_CUMMINS_LOC not found
ORA-06512: at "SYS.ORACLE_LOADER", line 19
ORA-06512: at line 1
Please someone guide me to resolve this prob.
Thanking you,

Thanks to you all,
I had created a new file location in OWB to create
a flat file module but I dont know somewhere
i missed something while creating the new file location
that is why it was not loading data inspite of deploy successed ,
but when i put the same file to one of existing file location
and i deployed it and i got the data.
now i am exploring the existing file location for what was that
i missed during creation of new location.
Guys i am new to OWB so i dont know how to go about , if you have any
easy way to get hands on OWB, please let me know
Once again thanks everybody.

Sqlldr flat file to external table using shell scripts

Hi,
Has anyone done this before? Please give me a hand.
Thanks!

Thanks Justin.
When do I need to create the external table EMP_STAGING ?
These are my steps so far:
- shell script to crate the flat file (but I need to change the table name to EMP_STAGING)
- use a script to call sqlldr to load the flat file into the external table
- then the script will call the MERGE sql script to merge the data from the external table into the database table
Am I on the right track?
In which stage should I create and drop the external table?
Thanks!

How to pass parameters to the Control file while loading the data from flat file to staging table

Thanks in advance

Hi ,
LOADDATA statement is required at the beginning of the control file.
INFILE: INFILE keyword is used to specify location of the datafile or datafiles.
INFILE* specifies that the data is found in the control file and not in an external file. INFILE ‘$FILE’, can be used to send the filepath and filename as a parameter when registered as a concurrent program.
INFILE ‘/home/vision/kap/import2.csv’ specifies the filepath and the filename.
Hope this will help you......

Given data is (10,20,.,30)how to load the data from flat file to base table by eliminate the dot from data by using sql loader

pls send ans for this

1b5595eb-fcfc-48cc-90d2-43ba913ea79f wrote:
pls send ans for this
use any text editor to eliminate the dot before loading

How we will change the lowercase data in flat file to uppercase

How can i change the lowercase date in flat file after uploaded, out put should be uppercase.

Hi ramana,
See the example.
DO.
   READ DATASET s_filename INTO I_TEMP.
   IF SY-SUBRC <> 0.
    EXIT.
   ELSE.
     move I_TEMP to I_FINALTAB.
     append I_FINALTAB.
     clear I_FINALTAB.
ENDIF.
ENDDO.
     In the above code we are moving the data form file to i_temp and i_temp to I_FINAL.
Here before moving to i_FINAL.
use TRANSLATE i_temp to UPPSERCASE.
Then move to i_final.
Pls. reward if useful.

Moving data from flat file

Hi,
I am moving data from flat file to oracle table. while populating the oracle table if I get any errors in flat file those errors should populate in ODI error table.
Is this is possible? if yes. Could you please let me know the set up in ODI.

CKM is the dedicated for checking the constraints while doing transformation. The constraints includes, PK,FK, conditions etc.,
There are two types/ways of checking the constraints.
Flow Control: Triggered CKM after data is loaded in to I$.
Static Control : Triggered CKM after data is loaded in to target table.
If you opt for any one the above ODI will create E$ and SNP_CHECK_TAB (summary table for logging the errors) and load the error records.
ODI will also provide you an option of loading the corrected error records by RECYCLE ERROR feature. In this phase/run ODI will load ONLY the error records in to target table (assuming its been corrected/cleaned).
**how to set up flow control could you please provide steps**
**Appreciate your help**

Can we cleanse and transform data at flat file or external table level?

Similar Messages

Maybe you are looking for