OWB Flat File Management

Hello,
Right now we are using OWB 10g R1. In this we created some mappings to get get data from flat files to Oracle database. After that we created Process Flows to execute these mappings. In this process flows we needed lot of file management. For this we wrote shell scripts. Ex: Flat files created with different names. But in mappings we use source name as fixed. For this we changed Flat file name to fixed name which we used in mapping. Then sending this name to process flows. Sending original falt file to Archieve. All this is done by writing shell scripts.
Here my question is there any good file management or file handler in OWB 10g R2 ? If any body familiar with this please guide me or tell the documents where can I find about file management in process flows? I hope you guys understand my question. If not please let me know.
Thanks
Sri

Hi Sri,
I don't know if the following help but they are worth a look -
http://blogs.oracle.com/warehousebuilder/newsItems/viewFullItem$160
http://blogs.oracle.com/warehousebuilder/newsItems/viewFullItem$149

Similar Messages

Issue with characterset setting in OWB flat file target

an OWB mapping reads from the database source and writes to flat file target in unix os but junk characters are displayed for non english characters in the target file . The database table contains french,spanish,german,arabic characters.The nls db parameter setting is AL32UTF8. The same setting has been applied to OWB target also but still junk values are appearing for non english characterset.different charactersets like al32utf8,utf8,utf18,us7ascii have been tried at owb target setting but nothing is wroking out to remove junk characters. Please suggest
Edited by: 943807 on 30 Jun, 2012 10:43 PM

Please provide some input on the issue

OWB FLAT FILE MAPPING NOT ABLE TO DEPLOY

Hi,
I've recently started studying data warehousing and exploring the Datawarehouse builder version 3i.
I've created a simple mapping from a csv file to database table ie my source is a flat file and target is a database table. The mapping gets validated successfully and the ctl scrip also gets generated. But the 'Deploy' and "Run' buttons on the "Transformations" tab of the "Generation Results" window are displabled(greyed out) and hence I cannot deploy the generated script.
Please do suggest the cause and solution for the same.
Regards,
Barkha

Many Thanks to Winfred Deering from Oracle who replied:
"This is the expected behavior for flat file mappings. You can register the tcl script with Oracle Enterprise Manager and run it from there. Also you can save the control file and execute via sql loader. "
Thanks and Regards,
Barkha

Reject columns which exceed a particular length during OWB flat file import

Hi,
I have a file in csv format. This file has a lot of columns out of which i need to use only a few. I created a file in OWB using the given csv file using the file import wizard. I am using this file to create an external table.
The import is working fine, But the problem is that certain columns (which are not required to be processed) have very huge data. This is leading to rejection of certain rows (when i deploy the external table) which have valid data for the required columns, but too huge data was columns which i do not need.
But these rows must not be rejected.
I wanted to know if there any way by which we can truncate/ insert null in these unwanted columns (either during flat file import / or during creation of external table), so that the row is taken in partially(with the wanted columns)?
Thanks a lot!
NS

Hi Julie,
As Jim posted, you could try a RAW file instead of a CSV file. In SSIS, the Raw File destination writes raw data to a file. Because the format of the data is native to the destination, the data requires no translation and little parsing. So does the Raw
File source. Besides, the Raw File destination/source can improve the package performance because they write data more quickly than other destinations such as the Flat File, the OLE DB, and Recordset destinations.
References:
http://www.katieandemil.com/ssis-raw-file-source-example-ssis-2012
http://www.jasonstrate.com/2011/01/31-days-of-ssis-raw-files-are-awesome-131/
Regards,
Mike Yin
TechNet Community Support

OWB - Flat file to oracle table

Second message, please read.
OS- Microsoft windows 2000 5.00.2195 Service Pack 2
OWB client---9.0.3.33.0
OWB Repository--9.0.3.0.1
Database--oracle 9i Enterprise Edition 9.0.1.3.1 with the partioning option Jserver Release9.0.1.3.0
OEM--9.0.1.0.0
I have created mapping between the source(flat file)and target(oracle)using Oracle Warehouse Builder. Mapping was validated, configured and it successfully generated without errors. The ‘Deploy’ button and the ‘Run’ button are disabled. How can I run/deploy this mapping ?
The above OWB copy was downloaded from the Oracle corp website. Is this version an evaluation copy? Alternatively Is this a beta copy ?
Please help.

Dear All,
I have a problem with loading flat files to oracle tables using OWB 9.2
i created a map with my flat file as a source and my oracle table as a target,and i created a connector between the target location and the flat file location.And i deployed the connector and registered the flat file location using my PC information and the path of the files on my pc.
I generated the map and saved the .ctl file and deployed the map,and hence trying to load the ctl file using the sql loader as the following:
C:\Documents and Settings\CORAL 2000>sqlldr owb_rt_rep/owb_rt_rep@ofsa
control = D:\oracle2\ofsa\owb\codegen\BM_DEBIT_CARDS_DAT_MAP.ctl
---where owb_rt_rep is my runtime repository user.
I got the following error:
SQL*Loader-350: Syntax error at line 18.
Expecting "(", found ".".
INTO TABLE "{{TRG_LOC1.Schema}}"."BM_DEBIT_CARDS_DAT"
--where trg_loc1 is my target location and bm_debit_cards_dat is my target table.
I tried to grant from the target schema the privelages to the runtime user incase its the problem,but still it didn't work.
Any ideas??
Dina Nagia
Message was edited by:
Dina Nagia

Flat File as Source in OWB Mapping.

Hi All,
Configuration Details is as follows:
Oracle 9i Warehouse Builder Client: 9.2.0.2.8
Oracle 9i Warehouse Builder Repository: 9.2.0.2.0
Windows XP Professional
I developed a mapping which involves a flat file as source & Oracle Table as target steps followed were as follows.
1. Created Flat File module, selected Connector & Location Details.
2. Done mapping for source file & target table.
3. Validated & Generated the mappings.
4. Deployed the mapping in Deployment Manager.
5. Following was Error Message after execution of the mapping.
Completion Status: Completed with errors.
Meesage Log: Starting Execution of VHM_MAP2. Starting Task VHM_MAP2. RPE-01013:SQL Loader reported error condition,number 4. Completing Task VHM_MAP2.
Following was Control File generated by the above mapping.
OPTIONS ( ERRORS=50, BINDSIZE=50000, ROWS=200, READSIZE=65536)
LOAD DATA
CHARACTERSET WE8MSWIN1252
INFILE '{{TEXT_DATA_LOCATION.RootPath}}dept.txt'
BADFILE '{{TEXT_DATA_LOCATION.RootPath}}dept.bad'
DISCARDFILE '{{TEXT_DATA_LOCATION.RootPath}}dept.discard'
INTO TABLE "{{TARGET_LOCATION.Schema}}"."DEPT"
APPEND
REENABLE DISABLED_CONSTRAINTS
FIELDS
TERMINATED BY ','
OPTIONALLY ENCLOSED BY '"'
"DEPTNO" POSITION (1) INTEGER EXTERNAL ,
"DNAME" CHAR ,
"LOC" CHAR
Please note that no message is displayed in Row Activity & output parameters in Deployment Manager.
I worked on lower version of the OWB & in that version there was no concept of Location, Connector, Deployment Manager.
Not sure why the above mapping is failing?
Also I would like to know where can I get more information about the mapping errors which are different from oracle error messages.
e.g. In above case error message is RPE-01013: SQL Loader reported error condition, number 4 .
Thanks in Advance.
Regards,
Vidyanand

Hi
I'm using single host implementation OWB10g/Oracle on same host.
My flat file location is
c:\source\dept.txt
Pl tell me if I have to save the same file on any particular location for OWB to read.
Implementation in OWB10g
1. Created Flat File module, selected Connector & Location Details.
2. Done mapping for source file & target table.
3. Validated & Generated the mappings.
4. Deployed the mapping in Deployment Manager.
5. Following was Error Message after execution of the mapping.
Can not read data from file
c:\souredept.txt
OPTIONS ( ERRORS=50, BINDSIZE=50000, ROWS=200, READSIZE=65536)
LOAD DATA
CHARACTERSET WE8MSWIN1252
INFILE '{{TEXT_DATA_LOCATION.RootPath}}dept.txt'
BADFILE '{{TEXT_DATA_LOCATION.RootPath}}dept.bad'
DISCARDFILE '{{TEXT_DATA_LOCATION.RootPath}}dept.discard'
INTO TABLE "{{TARGET_LOCATION.Schema}}"."DEPT"
APPEND
REENABLE DISABLED_CONSTRAINTS
FIELDS
TERMINATED BY ','
OPTIONALLY ENCLOSED BY '"'
"DEPTNO" POSITION (1) INTEGER EXTERNAL ,
"DNAME" CHAR ,
"LOC" CHAR
Thx in advance

Standardize a flat file format w.r.t OWB

Hi Pat,
or anyone,
Can u help me to standardize a flat file format with respect to OWB?
I have 15 financial source systems. The client agreed in giving the data in csv format. Now I need to standardize the feed file so that it would be easy for me handle it in OWB. Please note depending on my standardization the client would give me the data and in turn it would be easy for me to handle it OWB External Tables. The records size for a single feed file is 1 million. So my questions are
1.How should I standardize a date field in flat file?. Please note target date field is default date in Oracle.
2.How should I standardize an address field? I.e. normally here I have 3 address fields. But I want the output in 1 field. What is the best possible methodology to do this? Or can I handle this issue in OWB front.
3.How should I standardize a feed file name (generic)?
4.How should I standardize a feed file folder (generic)? Please note the folder will reside in UNIX environment. The concept is, there would be respective folders in UNIX and the feed files would automatically fall into these respective folders in a fortnight basis. So what should I do to achieve this? I know I have to write a shell script for the automatic process. If so how will I do this and schedule it.
5.And w.r.t above question, how will I tell OWB to go and fetch the files in those folders. I know it is done while mapping. But my question is Is that the only best way I can achieve it?
6.If I left anything related to feed file, please tell me?
Please note that based on my standardization, the client would extract the data and give it to me. So if I standardize the feed file which would be easy for me to handle it in OWB, then that would be a great success. And also note this is for a long run. Minimum 10 years.
Regards,
rem.

Good afternoon Rem,
It's all very theoretical indeed, talking about standards and all. I'll try to answer some of your questions according to my experiences.
@1, Usually we load flat files into external tables using varchar2-fields only, occasionally number fields; dates however are always loaded into varchar2's and then when reading the external table into the staging area they are converted to Oracle dates using a simple TO_DATE transformation.
@2, I don't think there's a real standard for that, it should simply meet YOUR needs, so when loading EXT into STG simply perform a transformation concatenating those 3 fields to form one.
@5, you have to register the location where you have defined your flat files, this should be some Unix location; after that the connector between EXT and STG will know what to do when running the mappings from EXT to STG (thus: no other action is needed to 'tell OWB' where to fetch data from flat files).
As far as I know there a no standards in naming conventions for files and folders, just go with what seems logical/practical in your situation.
Regards, Patrick

How to remove the date extensions from a filename in SSIS Flat File Connection Manager dynamically at run time

Hello,
I have to load data from a csv file to SQL Database. The file is placed into a directory by another program but the file name being same, has different extensions based on time of the day that it is placed in the directory. Since I know the file name
ahead of time, so, I want to strip off the date/time extension from the file name so that I can load the file using Flat File Connection Manager. I am trying to use 'variable' and 'expression editor' so that I can specify the file name dynamically. But I
don't know how to write it correctly. I have a variable 'FileLocation' that holds the folder location where the file will be placed. The file for example: MyFileName201410231230 (MyFileName always the same, but the date/time will be different)
Thanks,
jkrish

I don't want to use ForEach Loop because the files are placed by a FTP process 3 times a day at a specific time, for ex. at 10 AM, 12 PM and 3 PM. These times are pretty much fixed. The file name is same but the extension will have this day time stamp. I
have to load each file only once for a particular reason, meaning I don't want to load the ones I already loaded. I am planning on setting up the SSIS process to load at 10:05, 12:05 and 3:05 daily. The files will be piling up in the folder. As it comes,
I load them. At some point in time, I can remove the old ones so that they won't take up space in the server. In fact, I don't have to keep the old ones at all since they are saved in a different folder anyways. I can ask the FTP process to
remove the previous one when the new one arrives. So, at any point in time, there will be one file, but that file will have different extensions every time.
I am thinking of removing the extensions before I load every time. If the file name is 'MyFileNamexxxxxxx.csv', then I want to change it to 'MyFileName.csv' and load it.
Thanks,
jkrish
You WILL need to use it eventually because you need to iterate over each file.
Renaming is unnecessary as one way or another you will need to put a processed file away.
And having the file with the original extension intact will also help you troubleshoot.
Arthur
MyBlog
Twitter

Loading CLOB data to a flat file in owb 11.2.0.3

Hi,
Also,I have one more question.I want to load a table with clob data into a flat file using my owb client 11.2.0.3.But when I run this simple mapping ,I am getting this error:
ORA-06502:pl/sql:NUMERIC OR VALUE ERROR:CHARACTER STRING BUFFER TOO SMALL.
Please suggest what needs to be done.

I was up all night as I had a dead line doing this....
Turned out my Mapping was corrupted.
If you do a series of synchronizations by object name, position, id the mapping would end up corrupted.
I had to delete my staging table operator within the mapping, drag and drop and reconnect and now my loads work just fine..
Thanks

Urgent.. Characterset of flat file target in OWB

OWB mapping reads from the database source and writes to flat file target in unix os but junk characters are displayed for non english characters in the target file . The database table contains french,spanish,german,arabic characters.The nls db parameter setting is AL32UTF8. The same setting has been applied to OWB target by editing flat file defintion also but still junk values are appearing for non english characterset.different charactersets like al32utf8,utf8,utf18,us7ascii have been tried at owb target setting but nothing is working out to remove junk characters. Please suggest

Yes this should be fine. Are you using the gateway to access SQLServer or code template mappings? You need to track which columns are throwing the data type conversion errors.
Cheers
David

Working with Flat files in OWB

I am getting flat files from mainframe tables and I created external tables in OWB for mapping. I get data from flat files sometimes and I dont get data even though the file exists and there is no way to debug where the problem is in the files.
When we import the file metadata do we have to specify the column attributes or we should leave it 0 so that it take default of 255.
If you have any process on how files should be created which is practised in your place without problems pl let me know.
thnks...

Hi,
Do you mean that you are not getting flat file data into the external tables ? in that case see the bad file and try to analyze the reason. Most of the lengths of the fields set up in the external table creates a problem. In that case increase the length of the external table fields to a comfortably large value and then try.
Regards
-AP

Problem Experts OWB 10gR2 - Mapping with OMB - Give a name to the flat file

I'm trying to create a mapping in OWB by using the Experts (OMB),
where the data source is a database table and the target is a flat file (to transform a table to a flat file).
The tasks of my Experts :
This task create a mapping, and add a existing table on it.
     OMBCC '$thepath'
     catch {OMBDROP MAPPING 'M1'}
     OMBCREATE MAPPING 'M1'
     OMBALTER MAPPING 'M1' \
     ADD $INITIAL_LAUNCH_CONTEXT_TYPE OPERATOR '$thetable' \
     BOUND TO $INITIAL_LAUNCH_CONTEXT_TYPE \ '$INITIAL_LAUNCH_CONTEXT' \
This task change the context and create a flat file
     OMBCC '/PRJ_LABO_PHARMA/SRC_FILE'
     catch {OMBDROP FLAT_FILE 'NEW_FILE'}
     OMBCREATE FLAT_FILE 'NEW_FILE' \
     SET PROPERTIES (DATA_FILE_NAME) VALUES('fichier_genere.txt')
This task add the flat file to the mapping, and create a connection between the source (table) and the target (flat file)
     OMBCC '/PRJ_LABO_PHARMA/MOD_SRC_LABO'
     OMBALTER MAPPING 'M1' \
     ADD FLAT_FILE OPERATOR 'NEW_FILE'\
     BOUND TO FLAT_FILE '/PRJ_LABO_PHARMA/SRC_FILE/NEW_FILE' \
     OMBALTER MAPPING 'M1' \
     ADD CONNECTION FROM GROUP 'INOUTGRP1' OF OPERATOR '$thetable' \
     TO GROUP 'NEW_FILE' OF OPERATOR 'NEW_FILE' COPY ALL
This task update the flat file structure
     OMBRECONCILE MAPPING 'M1' OPERATOR 'NEW_FILE' \
     TO FLAT_FILE '/PRJ_LABO_PHARMA/SRC_FILE/NEW_FILE' \
     USE (RECONCILE_STRATEGY 'REPLACE', MATCHING_STRATEGY 'MATCH_BY_OBJECT_ID')
Then I have a task which execute this mapping.
Everything is good, but during the execution I have a error VLD-2357 : The target file's name is not indicated for the file NEW_FILE.
I don't know how to give a name to the flat file in OMB.
Help me please.
Best regards
Samy

Just like if you built this in the UI where you would have to configure the flat file operator and define a target data file name and possibly the location these properties should be set in scripting.
Set the property TARGET_DATA_FILE_NAME to the name of the data file.
Something like....
OMBALTER MAPPING 'M1' \
MODIFY FLAT_FILE OPERATOR 'NEW_FILE' SET PROPERTIES (TARGET_DATA_FILE_NAME) VALUES ('my_datafile.csv')
Cheers
David

How to move flat file after loading in owb to other directory with othernam

Hi All,
I wast struck in problem . after loading the flat file in owb that file should be move or rename or zip with Date everyday .
I dont know how to achieve this issue in owb via proceudre or any other way ... .
how to zip files from pl/sql procedure ...... with sysdate.zip.
any one please help me in this problem to come out soon.
thanks ,
murari

One way is to use a Java stored procedure, this can then be called from a map or a process flow. The Java stored procedure will use the Zip capabilities in Java to do what you need. For example you will need the following procedure and you will have to grant privileges for your file to ZIP and the entire directory for zipping into (since the file name will be dynamic):
For example grant YOURSCHEMA (change to your tgt schema) read on your extract directory where your file to zip is and write on your zips directory (where the zips will go);
exec dbms_java.grant_permission('YOURSCHEMA', 'SYS:java.io.FilePermission', 'c:\staging\extracts\*','read');
exec dbms_java.grant_permission('YOURSCHEMA', 'SYS:java.io.FilePermission', 'c:\staging\zips\*', 'write' )
You can then create your procedure which you can invoke
CREATE OR REPLACE AND COMPILE JAVA SOURCE NAMED "otnZipFile"
AS
import java.util.zip.ZipEntry;
import java.util.zip.ZipOutputStream;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
public class otnZipFile
public static void zipFile(String zip, String filename)
byte[] buf = new byte[1024];
try {
String outFilename = zip;
ZipOutputStream out = new ZipOutputStream(new FileOutputStream(outFilename));
FileInputStream in = new FileInputStream(filename);
out.putNextEntry(new ZipEntry(filename));
int len;
while ((len = in.read(buf)) > 0) {
out.write(buf, 0, len);
out.closeEntry();
in.close();
out.close();
} catch (IOException e) {
create or replace PROCEDURE zip_file(
dir IN VARCHAR2,
file_name IN VARCHAR2) IS
LANGUAGE JAVA NAME 'otnZipFile.zipFile(java.lang.String, java.lang.String)';
For example here I zip a file MYCSV.csv into the zip file MYZIP.zip
exec zip_file('c:\staging\zips\MYZIP.zip', 'c:\staging\extracts\MYCSV.csv');
Cheers
David

OWB export flat file with align to right

Hi All
I want to know if is possible to configure a metadata POSITIONAL FILE (Flat File) where every data is align to right, in a Mapping.
I don't want to use a oracle function RPAD with Paletta Expression.
Regards

Hi,
Based on my test, I can reproduce the similar issue in my environment. When we create the Flat File Connection Manager with the default settings, the data in flat file stops when goes to the Chinese characters.
To fix the issue that makes the Chinese characters to actually get written into the flat file, we should check the Unicode checkbox on the right hand side of Locale property in Flat File Connection Manager. In this way, the flat file can display Chinese
characters.
The following screenshot is for your reference:
Thanks,
Katherine Xiong
Katherine Xiong
TechNet Community Support

Manage Weblogic users through Flat file in obiee11g

Dear Experts
Need information about the following :
I have a requirement to manage users and groups (roles - like BI Author, Consumer etc) through FLAT FILE . The users in Weblogic will be populated through LDAP.
To elaborate more say there is some change/new addition of users in their groups and roles. SO instead of managing it through Weblogic , I wish to upload some flat file with the change and it shpould be done automatically rather than doing it manually.
Any pointer/help will be highly appreciated.

Hi,
Weblogic stores all this information by default in a .xml file. So, uploading some excel/xml manually for this might be quite tough. But, how about uploading the excel/storing the Group information in some database tables, which is quite good.
Hope this helps.
Thank you,
Dhar

OWB Flat File Management

Similar Messages

Maybe you are looking for