How to load thousands of data files?

My database is installed on windows, and I have thousands of data files that I need to load. Trying to figure out how to do it without manually editing my CTL file every time I load one data file.
Can you specifiy your control file to load directories instead of just a file?
Or can some looping script be written that makes the changes?
Thanks!
Message was edited by:
user447781

Don't list the data file in the CTL file ... use the "data=" option on the command line to pass in the data file.
A wrapper BAT file should be easy to create/generate.
D:\>sqlldr
SQL*Loader: Release 9.2.0.6.0 - Production on Mon Aug 29 13:52:54 2
Copyright (c) 1982, 2002, Oracle Corporation. All rights reserved.
Usage: SQLLDR keyword=value [,keyword=value,...]
Valid Keywords:
    userid -- ORACLE username/password
   control -- Control file name
       log -- Log file name
       bad -- Bad file name
      data -- Data file name
   discard -- Discard file name
discardmax -- Number of discards to allow          (Default all)
...

Similar Messages

How to load a huge data file (export from mysql) to an oracle table?

I have a data file which contains 1.3G bytes table data from mysql. It's format is csv.
How can i load these data to an oracle table in java way?
I thought about using sqlldr. But it requires a oracle client which may not be permitted in my environment.
If i generate "insert " statement and use batch update the batch update will stop if it comes across an exception when executing the insert one by one. I find no "insert ingore" statement in oracle like mysql.
Is there anyone can give me a solution with high performance? It's urgent and Thanks!

hsy541 wrote:
I have a data file which contains 1.3G bytes table data from mysql. It's format is csv.
How can i load these data to an oracle table in java way?
Why must it be a Java way? The point is just to move the data.
I thought about using sqlldr. But it requires a oracle client which may not be permitted in my environment.
Why not just import the .csv file into Oracle? I'm sure there's a script or utility that will do it.
If i generate "insert " statement and use batch update the batch update will stop if it comes across an exception when executing the insert one by one. I find no "insert ingore" statement in oracle like mysql.
You want to turn off transactions? I'm not an Oracle expert, but I'd be shocked if there was something you could do in MySQL that didn't have an analog in Oracle. Find a better resource, like somebody that actually knows Oracle. Who set up that database for you? Consult with them.
Is there anyone can give me a solution with high performance? It's urgent and Thanks!Urgent to you....
%

How to load images from css file in JavaFX 8

I have this css file which loads images in JavaFX 8 application:
#pill-left {
    -fx-padding: 5;
     -fx-border-image-source: url("/com/dx57dc/images/left-btn.png");
    -fx-border-image-slice: 4 4 4 4 fill;
    -fx-border-image-width: 4 4 4 4;
    -fx-border-image-insets: 0;
    -fx-border-image-repeat: stretch;
     -fx-background-color: null !important;
#pill-left:selected { -fx-border-image-source: url("/com/dx57dc/images/left-btn-selected.png"); }
#pill-left .label {
    -fx-text-fill: #d3d3d3;
    -fx-effect: dropshadow( one-pass-box , rgba(0,0,0,0.75) , 0, 0.0 , 0 , -1 );
#pill-left:selected .label {
    /* -fx-text-fill: black; */
    -fx-text-fill: white;
    -fx-effect: dropshadow( one-pass-box , white , 0, 0.0 , 0 , 1 );
#pill-center {
    -fx-padding: 5;
     -fx-border-image-source: url("/com/dx57dc/images/center-btn.png");
    -fx-border-image-slice: 4 4 4 4 fill;
    -fx-border-image-width: 4 4 4 4;
    -fx-border-image-insets: 0;
    -fx-border-image-repeat: stretch;
     -fx-background-color: null !important;
#pill-center:selected { -fx-border-image-source: url("/com/dx57dc/images/center-btn-selected.png"); }
#pill-center .label {
    -fx-text-fill: #d3d3d3;
     -fx-effect: dropshadow( one-pass-box , rgba(0,0,0,0.75) , 0, 0.0 , 0 , -1 );
#pill-center:selected .label {
    -fx-text-fill: black;
    -fx-effect: dropshadow( one-pass-box , white , 0, 0.0 , 0 , 1 );
#pill-right {
    -fx-padding: 5;
    -fx-border-image-source: url("/com/dx57dc/images/right-btn.png");
    -fx-border-image-slice: 4 4 4 4 fill;
    -fx-border-image-width: 4 4 4 4;
    -fx-border-image-insets: 0;
     -fx-border-image-repeat: stretch;
    -fx-background-color: null !important;
#pill-right:selected { -fx-border-image-source: url("/com/dx57dc/images/right-btn-selected.png"); }
#pill-right .label {
     -fx-text-fill: #d3d3d3;
    -fx-effect: dropshadow( one-pass-box , rgba(0,0,0,0.75) , 0, 0.0 , 0 , -1 );
#pill-right:selected .label {
    -fx-text-fill: black;
    -fx-effect: dropshadow( one-pass-box , white , 0, 0.0 , 0 , 1 );
The images are located at the Java package com.dx57dc.images
In Java 7_25 this code works as expected but in JavaFX 8 b99 I get this error:
ava.lang.NullPointerException
at com.sun.javafx.sg.prism.NGRegion.renderContent(NGRegion.java:1129)
at com.sun.javafx.sg.prism.NGNode.doRender(NGNode.java:1598)
at com.sun.javafx.sg.prism.NGNode.render(NGNode.java:1520)
at com.sun.javafx.sg.prism.NGGroup.renderChildren(NGGroup.java:233)
at com.sun.javafx.sg.prism.NGGroup.renderContent(NGGroup.java:199)
at com.sun.javafx.sg.prism.NGRegion.renderContent(NGRegion.java:1249)
at com.sun.javafx.sg.prism.NGNode.doRender(NGNode.java:1598)
at com.sun.javafx.sg.prism.NGNode.render(NGNode.java:1520)
at com.sun.javafx.sg.prism.NGGroup.renderChildren(NGGroup.java:233)
at com.sun.javafx.sg.prism.NGGroup.renderContent(NGGroup.java:199)
at com.sun.javafx.sg.prism.NGRegion.renderContent(NGRegion.java:1249)
at com.sun.javafx.sg.prism.NGNode.doRender(NGNode.java:1598)
at com.sun.javafx.sg.prism.NGNode.render(NGNode.java:1520)
at com.sun.javafx.tk.quantum.ViewPainter.doPaint(ViewPainter.java:99)
at com.sun.javafx.tk.quantum.AbstractPainter.paintImpl(AbstractPainter.java:210)
at com.sun.javafx.tk.quantum.PresentingPainter.run(PresentingPainter.java:95)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
at com.sun.javafx.tk.RenderJob.run(RenderJob.java:58)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at com.sun.javafx.tk.quantum.QuantumRenderer$PipelineRunnable.run(QuantumRenderer.java:129)
at java.lang.Thread.run(Thread.java:724)
D3D Vram Pool: 13,331,480 used (5.0%), 13,331,480 managed (5.0%), 268,435,456 total
20 total resources being managed
4 permanent resources (20.0%)
1 resources locked (5.0%)
7 resources contain interesting data (35.0%)
0 resources disappeared (0.0%)
D3D Vram Pool: 13,331,480 used (5.0%), 13,331,480 managed (5.0%), 268,435,456 total
20 total resources being managed
4 permanent resources (20.0%)
1 resources locked (5.0%)
7 resources contain interesting data (35.0%)
0 resources disappeared (0.0%)
D3D Vram Pool: 13,331,480 used (5.0%), 13,331,480 managed (5.0%), 268,435,456 total
20 total resources being managed
4 permanent resources (20.0%)
1 resources locked (5.0%)
7 resources contain interesting data (35.0%)
0 resources disappeared (0.0%)
What is the proper way to load images from css in Java 8?
Ref
How to load images from css file in JavaFX 8 - Stack Overflow

There is nothing special to do - you execute the statement from your program just like any other SQL statement. The only thing to be aware of are the privilege/permission issues:
When loading from a file on a client computer:
READ CLIENT FILE privilege is also required for the database user.
Read privileges are required on the directory being read from.
The allow_read_client_file database option must be enabled.
The read_client_file secure feature must be enabled.
Revoking these privileges is also the only way you can prevent a user from executing the statement.

How do i move a data file from my pc to my ipad

How do I move a data file from my computer to my ipad? It is my product catalogue and it would help if it were on my ipad.

Plug the iPad into iTunes and you can use file sharing to move it into an app that supports file sharing and that document format.
see http://support.apple.com/kb/ht4094
Or, use an online storage service (iCloud, dropbox, box, google drive, sugarsync, etc) and upload the file to that. Then on the iPad, from within the app you plan to use the file with, link to that service and download the file.
The app will again have to work with that file type, and support the particular service you used.

How to load external storage html file in web view

hi all,
how to load external storage html file in web view, please help me
" ms-appdata://local/index.html" not working
veerasuthan veerakesan

It need be read as string. Then load the string by Webview.NavigateToString.
Sample as below
string htmlstring = string.Empty;
try
var htmlfile = await Windows.Storage.ApplicationData.Current.LocalFolder.OpenStreamForReadAsync("a.html");
using (System.IO.StreamReader streamReader = new System.IO.StreamReader(htmlfile))
htmlstring = streamReader.ReadToEnd();
webview.NavigateToString(htmlstring);
catch(Exception ex)
Debug.WriteLine(ex.ToString());
在現實生活中，你和誰在一起的確很重要，甚至能改變你的成長軌跡，決定你的人生成敗。和什麼樣的人在一起，就會有什麼樣的人生。和勤奮的人在一起，你不會懶惰；和積極的人在一起，你不會消沈；與智者同行，你會不同凡響；與高人為伍，你能登上巔峰。

How may load an external (html) file in swf? Anyone!!!!!

hi all,
How may load an external html file in swf? such as google.com in a movieclip.
anyone may help me.........????????

With Flash, you can't do this.
Just with FlashBuilder and Air (something like HTMLLoader , not sure now)
I mean loading in a movieclip..
Of course you can open a page to display in a browser.

How do I encrypt a data file so that only I can retrieve the info?

How do I encrypt a data file so that it cannot be read without the proper authority?
I have an application where the customer should not have access to the data I need to record for troubleshooting purposes. (there are industrial secrets I wish to protect) My plan is to record a datalog (currently I am producing a tab-delimited spreadsheet format) whenever the device is running and hide the files where they will probably not be found. But some sort of encryption or at least password protection would be better.
I've never tried to do this before, but thought it would be fairly easy. Maybe I'm just not looking in the right places.
Thanks
Solved!
Go to Solution.

Well, you could look into something like DES, Triple DES, AES, etc. There are libries floating around for these written in LabVIEW. I'm not sure about the cost though.
If you want a really simple way to protect your data, just invert all or some of the bits in each byte of your file. It is super simple and turns a nice ASCII text file into noise when read from a text file.
There are only two ways to tell somebody thanks: Kudos and Marked Solutions
Unofficial Forum Rules and Guidelines

How to design SQL server data file and log file growth

how to design SQL DB data file and log file growth- SQL server 2012
if my data file is having 10 GB sizze and log file is having 5 GB size
what should be the size in MB (not in %) of autogrowth. based on what we have to determine the ideal size of file auto growth.

It's very difficult to give a definitive answer on this. Best principal is to size your database correctly in advance so that you never have to autogrow, of course in reality that isn't always practical.
The setting you use is really dictated by the expected growth in your files. Given that the size is relatively small why not set it to 1gb on the datafile(s) and 512mb on the log file? The important thing is to monitor it on an on-going basis to see if that's
the appropriate amount.
One thing you should do is enable instant file initialization by granting the service account Perform Volume Maintenance tasks in group policy. This will allow the data files to grow quickly when required, details here:
https://technet.microsoft.com/en-us/library/ms175935%28v=sql.105%29.aspx?f=255&MSPPError=-2147217396
Also, it possible to query the default trace to find autogrowth events, if you wanted you could write an alert/sql job based on this
SELECT
[DatabaseName],
[FileName],
[SPID],
[Duration],
[StartTime],
[EndTime],
CASE [EventClass]
WHEN 92 THEN 'Data'
WHEN 93 THEN 'Log' END
FROM sys.fn_trace_gettable('c:\path\to\trace.trc', DEFAULT)
WHERE
EventClass IN (92,93)
hope that helps

How to load the exist Data Mart Request in to cube.

Hi Friends,
I have a scenario... i got DSO and one Cube , iam loading the data from DSO to Cube .
The number of records are more in the DSO and due to some clculation in the routine i need to load the data year by year(i got 2007 and 2008 year daya). I have loaded the 2007 data to the Infocube and in the DSO i can see the Data Mart Symbal against the Request Id when th load is finished successfully.
When iam trying to load the 2008 data from DSO to Cube iam getting the "0" records. I realised that the Data Mart symbal is exist in the DSO.
How to load the 2008 data in this scenario. If i delete the data mart sysbel means iam deleting the Cube request.
Can any have an idea on this.
Thaks in advance.

HI,
Things are not clear.
How is loading happening if its delta or full load through DTP or you using 3.5 flow??
In any cases if you do a full load or full repair based on the year selection it should pick the records from the source...there is nothing to do with the data mart status in the case of full loads.
Data mart status will come into picture only when you schedule the delta loads.
Do a full load based on selections on year from the DSO to the cube...no need to delete the data mart or it will bring that request again delta is scheduled...does that request in DSO contains the data for 2008 only ...if yes then you can just delete the data mart status for that and do a delta...if not then do full loads as said.
Thanks
Ajeet

How to load Matrix report data into basic table data using ODI

Hi,
How to load Matrix report data into basic table data using oracle Data Integrator ?
Requirement Description:
Following is the matrix report data:
JOB                       DEPT10                DEPT20
ANALYST                                           6000
CLERK                   1300                     1900 Need to convert it into below format:
JOB                             Dept                        Salary
ANALYST                  DEPT10
ANALYST                  DEPT20                     6000
CLERK                       DEPT10                    1300
CLERK                       DEPT20                    1900
    Thanks for your help in advance. Let me know if any further explanation is required.

Your list seems to be a little restrictive, you can do a lot more with ODI procedures.
If you create new procedure, and add a step. In the 'command on source' tab set you technology and schema as per your source database. Use the unpivot functionality as described in the link, please, rather than using 'SELECT *' use the appropriate column names and alias them for eg:
SELECT job as job,
deptsal as deptsal,
saldesc as saledesc
FROM pivoted_data
UNPIVOT (
deptsal --<-- unpivot_clause
FOR saldesc --<-- unpivot_for_clause
IN (d10_sal, d20_sal, d30_sal, d40_sal) --<-- unpivot_in_clause
Then in your 'command on target' tab set the technology and schema to your target db, then put your INSERT statement for eg:
INSERT INTO job_sales
(job,
deptsal,
saledesc
VALUES
:job,
:deptsal,
:saledesc
Therefore you are using bind variables from source to load data into target.
Obviously if the source and target table are in the same database, then you can have it all in one statement in the 'command on target' as
INSERT INTO job_sales
(job,
deptsal,
saledesc
SELECT job as job,
deptsal as deptsal,
saldesc as saledesc
FROM pivoted_data
UNPIVOT (
deptsal --<-- unpivot_clause
FOR saldesc --<-- unpivot_for_clause
IN (d10_sal, d20_sal, d30_sal, d40_sal) --<-- unpivot_in_clause
also set the log counter as 'Insert' on the tab where your INSERT statement is, so you know how many rows you insert into the table.
Hope this helps.
BUT remember that this feature only came out in Oracle 11g.

How do I use old data files with a new installation in Oracle 9i

I am in an unenviable position, with an unsupported database.
We are running Oracle 9i on Windows XP. We are upgrading soon to Oracle 11g on a newer platform, but need to get our development environment working first.
We lost a system that was running our development database without having a database export. The C drive was placed into a new system as the D drive.
I have loaded Oracle 9i on the C drive, but I have been unable to determine how to point it to the existing data files on the D drive. My search skills may be the limiting factor here...
We cannot simply load the drive as C, since the hardware is different.
What are the steps to point the new database software at the data files on the D drive? Or, how do I copy the old data files into the new Oracle Home and have them recognized properly?
Thanks for any advice.

user3930585 wrote:
I am in an unenviable position, with an unsupported database.
We are running Oracle 9i on Windows XP. We are upgrading soon to Oracle 11g on a newer platform, but need to get our development environment working first.
We lost a system that was running our development database without having a database export. The C drive was placed into a new system as the D drive.
I have loaded Oracle 9i on the C drive, but I have been unable to determine how to point it to the existing data files on the D drive. My search skills may be the limiting factor here...
We cannot simply load the drive as C, since the hardware is different.
What are the steps to point the new database software at the data files on the D drive? Or, how do I copy the old data files into the new Oracle Home and have them recognized properly?
Are you stating that you don't know how to use COPY command?
Can you recreate same directory structure on new C drive as exist on old C drive?
Can you then drag & drop copies of the files?

How to Edit a XML data file from AS3 class...

Hi...
             I need to edit an XML data file I have at the exit of application.... I have a demo code which is not working....
This is the code :
public function UpdateData():void
               trace("Closing....");
               var FileData:XML=<Hello>Hai</Hello>
               var urlLdr:URLLoader = new URLLoader();
               var urlRqst:URLRequest = new URLRequest ("AppData.xml");
               urlRqst.data=FileData;
                urlRqst.contentType = "text/xml";
               urlRqst.method = URLRequestMethod.POST;
               urlLdr.load(urlRqst);
This AppData file already exist and has some data in it.
    Plus :
   How to find the application is exiting.

use: fscommand('Quit');
to exit your app.
and this is how you would use the urlloader class to save data:
public function UpdateData():void
         trace("Closing....");
          var FileData:XML=<Hello>Hai</Hello>
var urlVar:URLVariables=new URLVariables();
urlVar.fileData=FileData;
            var urlLdr:URLLoader = new URLLoader();
           var urlRqst:URLRequest = new URLRequest ("this must be an executable to parse and write data");
           urlRqst.data=urlVar;
           urlRqst.contentType = "text/xml";
               urlRqst.method = URLRequestMethod.POST;
              urlLdr.load(urlRqst);
but i don't think you can use an executable on a fl4 device so you will need to use a different class. i don' know how you're storing an xml file on a fl4 device but you should be able to use the sharedobject to read and store xml.

HOW TO LOAD R/3 DATA INTO SAP BI USING PROCESS CHAINS?

Hi,
Can we load R/3 data into BI using process chains?... I loaded data from R/3 into Infocube using generic extraction using view... took 2 tables EBKN and EBAN and ceated view.
In PSA I can find all the 2388 records but when I load into datatarget in transferred tab there are 2388 records but in added colum i could find only 2096.....
I deleted the request and want to load through process chains....... but how to do ?????? without flat file ...can we laod using process chains?
I appreciate any inputs.......
Regards,
Prasanthi.

did you even bother looking at the links in my previous posts???
read the docs...try yourself...if you encounter specific issues, you can post them on the forum...
if you're really expecting somebody to post a step by step for process chain, i think you can wait a long, long time...

Does anybody know how Oracle load large N-Triple file into Oracle 11g R1?

Does anybody know how Oracle load large N-Triple(NT) file into Oracle 11g R1 by using sql*loader according to their benchmark results?
Their benchmark results indicate they have over come the large data set problem.
http://www.oracle.com/technology/tech/semantic_technologies/htdocs/performance.html
It means they have loaded LUBM 8000(1.068 Billion+ Triples) into Oracle successfully, but there is no detailed steps provided. For instance, 32-bit or 64-bit platform they used, only one NT file being used corresponding to one dataset or several NT files?
Is there any exception occured during the loading process if the NT file beyond 60GB? When using jena to generate NT file against LUBM(8000), the size of the NT file would definitely beyond 60GB.
We are dividing such large NT file into several small ones? Is it a good approach? I'm hesitating to do so!

A Linux 32-bit platform was used for bulk-load of LUBM-8000 1.106 billion (before duplicate elimination) RDF triples into Oracle.
Multiple gzipped N-Triple files were used to hold the LUBM-8000 data. zcat was used on all these files together to send the complete data into a named pipe. SQL*Loader used this named pipe as the input data file to load the data into a staging table in Oracle. Once the staging table was loaded, the sem_apis.bulk_load_from_staging_table API was used to load the data into Oracle Semantic Store.
(Additional details in http://www.oracle.com/technology/tech/semantic_technologies/htdocs/performance.html )
Thanks.

How to load a long flat file field into BW

Hi all,
I have a field from flat file with 256 characters to be loaded into BW. As we all know the max length an info object can take is only 60. How can I split the flat file field so that I can load that data into BW and how can I then concatenate them in the BEx.
Can anyone suggest a way?
Thanks in advance.
Vinay

Hi Vinay,
we already had this discussion in this forum. In short:
1. create a number range object
2. create a table with a number field a key and a text field
3. while loading the data check your text against the table. If the text already exist, get the number out of the table. If the text doesn't exist, get a new number out of your number range object and post this number + the text into your table.
4. in your loaded data always work with the number you assigned to the text
5. create a infoobject for the number-field
6. create a variable that takes it's value out of the table.
regards
Siggi

How to load thousands of data files?

Similar Messages

Maybe you are looking for