Is File Redirection in ODI Possible?
Hi All,
I'm still new to ODI and would like to ask if a file redirection is possible.
Scenario is:
I receive an XML file that contains fields, data, etc,... from a source and the file has the exact information that is needed by the Target. Technically, no transformation, validation needed.
Can this be done without creating tables for staging processes? If yes, how? A straightforward receive file, send file.
Regards,
Dave Null
Can you give some more background about the systems where you are copying from/to (windows or unix?)
We implemented a scenario that copies a file from windows to unix or from unix to unix.
on one of the servers, we have an agent available which calls a script (via an ODI procedure) that resides on the same server.
This script will rsync the file from one server towards the other.
The cool thing about rsync is that it will copy over only the delta's of the file and shrinks the data, so it's really fast. You also have rsync versions for windows available.
Similar Messages
-
Understanding XML file handling in ODI
Hi People,
I've just completed a simple Interface to upload an XML file into an Oracle 10g database table. The interface copes well with both inserts and updates, so I'm very pleased about that. ;)
This exercise has raised some questions though. I tried an experiment, which involved changing an element value in the XML file. I wanted to see if this change would be reflected in the file's associated Source Table in ODI. Unfortunately, it wasn't; despite me clicking the REFRESH button about 4 times. I guess my first question is:
*1. How can I ensure a change to an element value in the XML file is reflected in ODI, after reverse engineering the associated data store from the file? Is this possible?*
Moving on, I'm trying to understand the architecture ODI creates in the background, in order to support a source datasource/table that represents the XML file. I have created the following Topology to support the file:
!http://img2.imageshack.us/img2/6895/odiscr375.jpg!
As you can see, I've associated the XML file with a physical schema called ODI_XML. This represents a schema owned by my Oracle Database.
Now, when I check the XML file data in ODI Designer, I notice that it's based on this query:
select * from ODI_XML.TRADE
I've checked the ODI_XML database schema, and there is neither a table nor any other objects named TRADE. My next question is therefore:
*2. Was I correct to assign a database schema (named ODI_XML) to my XML Data Server, or should this have been something else? If so, what?*
Thanks in advance for any assistance.
JamesSince you are using the Variable ,when you right click and view datastore its missing the required path of the file, as value of the variable is not determined , to do so you need to either refresh or set the variable and call the interface or other ODI objects and in loop keep reading XML files one by one and that should work.
-
I want to use my Time Capsule as a central 'hub/hard-drive' in order to keep all my working files in one place. I then plan to use my laptops/desktops as peripheral devices for editing and creation of these files. Is it possible? To create a personal cloud?
Can someone advise as to whether i can do the following:
Future Internet connection -------------------TIME CAPSULE (containing all files) -------------- Backed up on the WD 2T Hard-drive I have
(not connected yet) I (*connected to timecapsule physically)
I
I I I I I
Macbook Pro iMac HP (PC) OLD Macbook Pro iPhone
(used for remote working) (Used for home working) (used for heavy CAD and rendering) (not being used for anything at the moment) (& iPods)
I am looking to have all my files in one place as i am hopelessly disorganised. I know the online clouds are a good solution (used Dropbox at work and uni for 3 years) however i am wanting to create my own 'dropbox/icloud' at home. So whenever i get back home with my laptop, any work i have been working on whilst out that day is updated to the timecapsule, and then ultimately as i turn on the other devices, they update to those newer versions of the files. Please tell me that the 3TB time capsule i have can do this, otherwise it feels rather overpriced as a wireless storage device?
Another note (to those in the know) If i am to be working on large files (REVIT/SOLIDWORKS/KEYSHOT/CREO/AUTOCAD) - is the timecapsule connection good enough to support editing and updating these files?
I know i may be asking a question that many have before, but as a bit of a technical novice I wanted a clear-cut answer to my specific circumstances. Your help is greatly appreciated.
(*can i use this WD hard-drive that is connected to the Time Capsule as a back up? so that the time machine back-ups/any back ups are also backed up onto this one? can the WD be a backup for the TC?)
Kind Regards
JoeThe diagram was supposed to look more like this......
Internet ---------TIME CAPSULE(containing all files) --------WD 2T Harddrive
I
I I I I I
Macbook Pro iMac HP (PC) OLD Macbook Pro iPhone
Sorry!
Regards
Joe -
Passing Dynamic File Name to ODI nterface for processing to another system
Hi,
I need help regarding passing a Dynamically Name changing fixed length Flat File in ODI Interface. This interface is built for taking the Flat File as Input and process it to SQL Server by applying Data Mapping and transformations... The input Flat File Name is sequence generated for eg: OEORD1123.txt and next file will be OEORD1124.txt and it sits in Oracle Conc tier. How to pass the latest file name to ODI interface for processing
Regards,
Anil..Hi Guys...
I would like to suggest a way.
a) create a single interface with dynamic resouce name (a ODI variable) with a filter to the month column like:
month_column = '#vCountMonth'
b) in the refresh tab of a first variable (I named "vMonth"), use the following query: (varialbe should be alphanumeric, "not persistent")
select to_char(to_date('#vCountMonth','MM'),'month') from dual
c) create one more ODI variable (I named "vCountMonth"), alphanumeric, not persistent and at its refresh tab write:
select lpad(to_char(#vCountMonth + 1), 2, '0') from dual
d) now just create a package, drag and drop the objects in the following order:
d.1) vCountMonth in set mode and set = 0 (zero)
d.2) vCountMonth in refresh mode
d.3) vMonth in refresh mode
d.4) the interface
d.5) vCountMonth in evaluate mode, evaluating "= 12"
==> if NO (KO, red line) link the KO line to d.2 step
==> a OK line is not necessary unless you have others steps after finish the evaluating
Make any sense? That is a single loop to have the interface developed only one time.
Please, remember to check each thread reply as Useful or Correct if they are useful to you... -
When I try to open any docx file from the SharePoint it opens on web via office web apps, and when I choose edit document > edit in Word , Microsoft word show me this error "Sorry, we couldn't find your file.
Is it possible it was moved, renamed or deleted?" followed by half a url to the end users document on onedrive.
I noted that this problem only happens with Docx files, with specific users and only when you try to open the Docx file from OneDrive folder ( SkyDrive ) which is hosted in the MySite.
Any Help?Hi,
According to your post, my understanding is that you failed to open the Docx file from OneDrive folder ( SkyDrive ).
Since this problem only happens with Docx files, with specific users, I recommend to delete and recreate the personal site for the specific users.
More information:
Configure My Sites in SharePoint Server 2013
Best Regards,
Linda Li
Linda Li
TechNet Community Support -
How to do file validation with ODI 10.1.3.5
Hi Team,
Please help me, how to handle file validation with ODI?
My source is files.
Requirement:
With ODI i have to do the file validation and file data loading successfully. Once it is successfully done then i have to move the particular file and move to different directory.
Any internal tool is there in ODI package to watch the file.
Regards,
SureshHi Suresh,
You can do your file loading and validation in an ODI interface.
Once you are done with the interface, add it to a package, you can set up subsequent steps in the package to move the file to another location using the built in ODI file utilities in the package.
Terrence. -
Hi,
we have a requirement to read huge amount of data from an Oracle DB view into a flat file. but instead of creating one big file we want to debatch it into a number of smaller files. Does ODI provide any such kind of functionality ?I think you'll have to do this either by using filters in the interface to split the data flow, or using some customized jython scripts to "slice" the file after processing.
Regards,
-FX -
Linux file name into ODI variable
Hello All,
I have a situation where we have files in one directory in linux ,I want to import the file name into a ODI variable.
Ex : A_B_C_2014_01_20.zip (inside this zip file are the flat files with data) is the file name in /u02/source_files location , i want this into a ODI variable and once the data loading is complete ,i want to log it saying
A_B_C_2014_01_20.zip complete
A_B_C_2014_01_21.zip complete.
So my other shell script will look for the complete flag and copy the next file into the ODI pick up location.In this case I need to pick up A_B_C_2014_01_22.zip and start the data loading.
How do i implement this functionality in ODI. Please let me know.Issue was due to a syntax error. Case closed.
Thanks!
-OS -
What is the file size that ODI can handle
Hi all,
What is the file size that ODI can handle ?????????
can it handle 1GB files without any issues?????????
Thanks
PhaniHi Phani,
It also depends on the type of file you are transfering. If have used the XML file around 1GB and the performance is very low. But we used the Database schema to load the XML data and the performance had increased very well.
I never tried the large flat files load around 1GB.
Thanks,
Yellanki -
Hello Friends,
Can we reverse engineer a text file?
Using ODI-Designer (on Win2003 Server) we are having issue to reverse a file source which is on another server(Unix).
Steps followed:
- Defined a File source in Topology (/export/home/odi/input/files) with logical & context.
- Defined a model with File technology
- Performed a customized-reverse (Which RKM should we use here? (RKM file (from Excel))?
Appreciate your feedbacks.
thank you.If you have scheduler agent running on Unix and if the file is visible to the agent then there is no need to map the directory to Windows.
However, please try doing the same with an agent (not a scheduler agent). Try reversing a file using the agent.
I assume that /export/home/odi/input/files/xxxxx.txt is present and is visible to the agent.
If that doesnt work still, try to do it the way I do.
I have designer on Windows. I have a drive mapped to unix FS on where the file exists and I access it using
//myserver/file/path/file.txt
I have created a physical data server using FILE technology *//myserver/file/path/*
and then a file model using file.txt
Then, I am able to reverse-engineer a file using local agent.
Hope that helps. Let us know how it goes. -
I have a program that uses file redirection to read using System.in. Is there anyway to throw an exception if the file redirection isn't in the command line
Here is my code:
public static void main(String[] args) throws IOException {
Scanner sc = new Scanner(System.in);
InetAddress addr = InetAddress.getLocalHost();
String msgReply, hostName, fromAddress, toAddress, temp;
String data = "";
int portNum = 25;
hostName = args[0];
fromAddress = args[1];
toAddress = args[2];
Socket theSocket = new Socket(hostName, portNum);
BufferedReader in = new BufferedReader(new InputStreamReader(theSocket.getInputStream()));
OutputStreamWriter out = new OutputStreamWriter(theSocket.getOutputStream());
BufferedWriter wrt = new BufferedWriter(out);
msgReply = in.readLine();
Main.msgCheck(msgReply);
System.out.println(msgReply);
wrt.write("HELO " + addr.getHostAddress() + "\r\n");
wrt.flush();
msgReply = in.readLine();
Main.msgCheck(msgReply);
System.out.println(msgReply);
wrt.write("MAIL FROM:" + fromAddress + "\r\n");
wrt.flush();
msgReply = in.readLine();
Main.msgCheck(msgReply);
System.out.println(msgReply);
wrt.write("RCPT TO:" + toAddress + "\r\n");
wrt.flush();
msgReply = in.readLine();
Main.msgCheck(msgReply);
System.out.println(msgReply);
wrt.write("DATA \r\n");
wrt.flush();
in.readLine();
while (sc.hasNext()) {
temp = sc.nextLine();
if (temp.startsWith(".")) {
data += "." + temp + "\r\n";
} else {
data += temp + "\r\n";
wrt.write(data + "\r\n");
wrt.flush();
wrt.write(".\r\n");
wrt.flush();
msgReply = in.readLine();
Main.msgCheck(msgReply);
System.out.println(msgReply);
System.out.println(data);
wrt.write("QUIT \r\n");
wrt.flush();
msgReply = in.readLine();
Main.msgCheck(msgReply);
System.out.println(msgReply);
System.exit(0);
public static void msgCheck(String msgReply) {
if (msgReply.startsWith("5") || msgReply.startsWith("4")) {
System.exit(0);
} //end Main classCommand line should look like:
java Main smtp.google.com [email protected] [email protected] <test.txt
Is there anyway to check for:
java Main smtp.google.com [email protected] [email protected]
ThanksNo. And why should you care about that anyway? If I don't redirect from the file, that gives me the opportunity to type in the message I want to send, instead of having to put it into a file beforehand. Or I could write a program that produces the message and pipe its output into your program:
java PoliteMessageGenerator | java Main smtp.google.com [email protected] [email protected]
No, that's a feature and not a bug. -
How can I work with my adobe Lightroom Version 5 which was added to my Samsung NX3000. If I want to load an .SRW File it is not possible, only the .JPG File. How and from where can I the Module and how is it to install ?
You need Lightroom 5.5 or later (the most current version is 5.7.1). This is a free update if you have already purchased an earlier version of Lightroom 5.
Product updates -
How can I convert a wave file to a jpeg file (picture) is it possible???
Hello:
How can I convert a wave file to a jpeg file (picture) is it possible???zuckini;
I am pretty sure you know, but just to be completely sure: a wave is (usually) sound signal and a jpeg file is an image.
If you mean how to display the wave signal in a graph and then save the graph as a jpeg file, then just drop a graph indicator in the control panel, wire the signal. Check any of the example shipped with LabVIEW on how to do this. To get the image, drop a graph indicator's invoke node in the block and select "Get Image". Check this page for detailed information.
If you have something else in mind, please reply to this message with additional information about the wave signal, the image, etc.
Regards;
Enrique
www.vartortech.com -
Hi ,
is it possible to load a binary file using ODI ?
Basically i'm looking for file to table and file to file conversion .
Regards.DBMS_LOB.LOADBLOBFROMFILE (
dest_lob IN OUT NOCOPY BLOB,
src_bfile IN BFILE,
amount IN INTEGER,
dest_offset IN OUT INTEGER,
src_offset IN OUT INTEGER);
you can specify the src. Also, you can ftp the file using ODI sftp tool and then load.
There are other ways to load as well,
1. Sql loader aslo supports loading BLOBs. Odi has got IKM Sql loader however I have never tried with BLOB and need to check if it supports. The below link might be helpful,
ORACLE-BASE - Loading LOB Data Using SQL*Loader
2. External tables could be used.
3. Java program could be called in ODI proc. -
Error while using Oracle Table as source file :- ODQ for ODI
Hi All,
I am getting some errors while working on ODQ with Oracle Tables as source file.
If I am trying with text files (*.txt) as source and output it works fine.
Please let me know how we can connect to an oracle table which is my source file.
In the exported project -> “settings” folder
In the file named eN_transfmr_pXX.stx
For
/CATEGORY/INPUT/PARAMETER/INPUT_SETTINGS/ARGUMENTS/EN
TRY/DATA_FILE_NAME=
What I need to give? (URL – source file)
I tried with
1. jdbc:oracle:thin:@xxx.xxx.x.xx:1521:ORCL
2. jdbc:oracle:thin:UserName/Password@// xxx.xxx.x.xx:1521:ORCL
i am not sure , is there any thing missing ???
(Note: for text file I am giving “D:\Sourcefolder\customer.txt”)
If I am running the batch file directly from CMD Prompt it’s displaying the error message
“Cannot open file”
If I am connecting with ODI it’s displaying the error
com.sunopsis.dwg.function.SnpsFunctionBaseException: OS command returned 3503. …………………….
Thanks in advance…
Rathish A MHi Ratish,
ODQ supports files as inputs not Oracle tables, what you should do is:
- define an ODQ process that takes a file as an input.
- create an ODI process that dumps your Oracle table into a file that will be used by ODQ. (interface or OdiSqlUnload step)
- run the ODQ process in ODI (in a package)
- create an ODI interface that will load your ODQ output file into a DB.
You can profile Oracle tables directly using Oracle Data Profiling.
Thanks,
Julien
Maybe you are looking for
-
SQL*Loader : ORA-19007 with missing Schema Location Hint
Hi, These days I'm stuck with a 10.1.0.4 database (OS Linux Red Hat), and I'm trying to find a workaround for the following situation : XML schema (sfgtest.xsd) <?xml version="1.0" encoding="UTF-8"?> <xs:schema xmlns:xs="http://www.w3.org/2001/XMLSch
-
TS1365 My Macbook air turns on but is stuck on the apple icon screen
Mu Macbook air powers up and it gets stuck on the Apple Icon screen and will not go any farther. Any suggestions?
-
Concatenate 2 columns from 2 different dimension tables
Hi, I have 2 columns one from each dimension table. Column A from Dimension 1 and Column B from Dimension 2. In my answers i want the results as "dim1.Col A / Dim2.Col B". I used the concatenate function and it dint give me the results. Both the colu
-
Enbedded pictures on AOL email will not display, yet does on Internet Explorer?
Enbedded pictures on AOL email will not display, yet does on Internet Explorer?
-
Install button on photoshop cs5 installer won't work
I've downloaded the dmg file of photoshop cs5 to reinstall on my mac and when I click on install nothing happens. Suggestions?