Oracle Replication and SQL*Load direct Path

We are setting up Oracle replication and have a few tables which are loaded using SQL*Loader Direct path. I have following questions:
1. Can MultiMaster replication replicate direct patch sqlloaded data.
2. If the answer to the above question is no, should we set up a new snapshot replication group . All other tables are in the Multi Master replication.
3. Another question is on the number of replication groups. How many of these we should create. We have total of about 500 tables and database size is about 450 Gig. We plan to replicate all the tables and the refresh interval we want to be one minute for all the tables. Does having more/less replication groups help??
Thanks for your help

We are setting up Oracle replication and have a few tables which are loaded using SQL*Loader Direct path. I have following questions:
1. Can MultiMaster replication replicate direct patch sqlloaded data.Yes, I don't think it should matter how the table is getting updated.
2. If the answer to the above question is no, should we set up a new snapshot replication group . All other tables are in the Multi Master replication.see above
3. Another question is on the number of replication groups. How many of these we should create. We have total of about 500 tables and database size is about 450 Gig. We plan to replicate all the tables and the refresh interval we want to be one minute for all the tables. Does having more/less replication groups help??
Thanks for your help I believe what Oracle recommends is that you split up your tables into replication groups transactionally. I personally have 6 replication groups supporting 700+ tables. They are broken up more by business function than by number and size.

Similar Messages

Cannot get Oracle sequence to be used with SQL Loader direct path load.

I attempted to load approximately 3 million rows from a flat file into a 10g database using SQL Loader direct path load. In order to generate a primary key for each of these rows I used a simple SQL expression which gets the next number of an Oracle sequence and assigned that to my primary key column inside the control file. The strange thing is no primary key was generated for each of these columns as I see the sequence was not advanced. Furthermore when I try to run a query to find the count on those rows which have this column as NOT NULL ,I get the total of all the rows even though on inspection this column is empty in each of the rows.
As far as I know, in the control file for a direct path load I am able to use a SQL Expression that returns simple scalar data which a NEXTVAL on a sequence does. (Oreilly Oracle SQL Loader The Definitive Guide p.184)
Does anybody why my primary key was not generated and furthermore why is this column in a query appear to be NOT NULL when in fact it has nothing in it?

Daniel,
I appreciate your response alot.
Now a follow on. You say that SQL*Loader is the fastest way to get data into Oracle Spatial even though the conventional path. How does SQL*Loader do this? Doesn't it use OCI?
I just can't help but think that coverting my data to SQL*Loader format, and then having it parse it and send it to the database in some form must be slower than me just sending whatever SQL Loader sends to the DB myself using OCI. Am I missing something?
The SQL Loader documentation seems to suggest that SQL*Loader just turns the input into alot of INSERT stateemnts. If so, can't I just do this?
My performance with OCI and INSERT statements isn't very good, but I believe I am doing one transaction per insert. Might I be as well off to concentrate on fine tuning my OCI based code using INSERTs?
I will actually do some time tests myself, but I would appreciate your opinion.
Once again thanks for the great info you have provided.

SQL Loader direct path loads and unusable indexes

sorry about all the questions. I am researching several issues. I am reading the Utilities document. It says that in certain circumstances indexes will become unusable. I have some questions about my scenario.
1. tables partitioned by range
2. local indexes
3. all tables have 1 primary key and other indexes are non-unique
4. all sql loads will go into the most recent partition
5. users will be querying tables while sql loader is occurring
6. only one sql loader session will run per table
7. no foreign keys, triggers, or other constrants other than primary keys.
The docs are not clear. Do I have a concern about unusable indexes with direct path loads? Will indexes function while the sql loader direct path is occurring(this I can't test since I have small data files now and they load fast, but I will have larger ones in production).
My understanding is that Extertnal tables using Insert append is exactly the same as sql loader direct path load. Is this true?

if you dont have anything productive to say how about you don't post at all? you have made ignorant posts like this for years.
as far as reading the docs what do you think "the docs are not clear" means? By the docs I am referring to the utilities document.
As far as version number its 10.2 and I forgot that. However, it does not appear that sql loader has really changed all that much over the last few versions.
Finally I plan on testing it out and its more than a 2 minute test. I wanted to make sure I don't miss anythng in my tests.
don't respond to any threads or posts I make from now on.

Sql loader direct path

Hi All,
Can anyone confirm whether SQL loader DIRECT path option can load in clustered tables?
Also what is the difference between Clustered Server and Clustered Table? Will the clustered Server by default have clustered tables?

That is 1/2 of the questions asked.
What do you mean by clustered server? Do you mean RAC or HACMP or Veritas something else?
What do you mean by clustered table? Do you mean like a single table hash cluster or something else?
Please be specific ... you don't want an answer based on a guess.
And what is 10g? Is that 10.1.0.3, or 10.2.0.1, or 10.2.0.5? Again this is computers please be precise.I am sorry. version is 10.2.0.1. its Oracle RAC. I am not sure what kind of table clustering exists. I believe its table cluster using a cluster key(what kind of table clustering is this?)
I just want to know if sql loader direct path option can load data in Oracle RAC / Clustered tables? If not, any workaround for that?

SQL Loader Direct Path issue

Hi,
I am trying to load data using SQL Loader Direct Path. One of the tables has a REC_UPD_DT column which is of timestamp datatype. The table was created with default value of SYSTIMESTAMP on this column. However when I load the table, the REC_UPD_DT is null due to the behavior(bypasses the sqlfunctions, etc) of the Direct Path method.
What is the way out to achieve my objective?
I do not want to use the conventional path since that is proven to be slow for my case.
Thanks in advance
Pavin

Hi,
My apologies for not providing requested information before. Here are the scripts:
I am testing from my windows m/c.
Database version: 10.2.0.3
Table creation script:
CREATE TABLE TXN_DEFAULT
(MAPPER_Q_ID NUMBER(1,0) NOT NULL,
TRANSACTION_SEQUENCE NUMBER(10,0) NOT NULL,
RECORD_COUNT NUMBER(4,0),
STATUS VARCHAR2(1 CHAR),
USER_ID VARCHAR2(20 CHAR),
USER_IP_ADDRESS VARCHAR2(35 CHAR),
REC_UPD_DT TIMESTAMP (6) DEFAULT SYSTIMESTAMP
Loader Control file script:
OPTIONS (DIRECT=TRUE)
load data
CHARACTERSET WE8ROMAN8
into table TXN_DEFAULT
append
FIELDS TERMINATED BY '     '
trailing nullcols
MAPPER_Q_ID,
TRANSACTION_SEQUENCE,
RECORD_COUNT,
STATUS,
USER_ID,
USER_IP_ADDRESS,
REC_UPD_DT
Sample Dat file contents:
0     251939112     3     N     chowdsr     10.222.79.20
1     251939113     140     N     zhuanre     146.222.52.176
2     251939114     41     N     sombupo     146.222.187.155
0     251939116     75     N     BaseAdmin     146.222.4.246
Log File contents:
SQL*Loader: Release 10.2.0.1.0 - Production on Mon Dec 7 13:22:23 2009
Copyright (c) 1982, 2005, Oracle. All rights reserved.
Control File: Control.ctl
Character Set WE8ROMAN8 specified for all input.
Data File: Control.dat
Bad File: Control.bad
Discard File: none specified
(Allow all discards)
Number to load: ALL
Number to skip: 0
Errors allowed: 50
Continuation: none specified
Path used: Direct
Table TXN_DEFAULT, loaded from every logical record.
Insert option in effect for this table: APPEND
TRAILING NULLCOLS option in effect
Column Name Position Len Term Encl Datatype
MAPPER_Q_ID FIRST * WHT CHARACTER
TRANSACTION_SEQUENCE NEXT * WHT CHARACTER
RECORD_COUNT NEXT * WHT CHARACTER
STATUS NEXT * WHT CHARACTER
USER_ID NEXT * WHT CHARACTER
USER_IP_ADDRESS NEXT * WHT CHARACTER
REC_UPD_DT NEXT * WHT CHARACTER
Table TXN_DEFAULT:
4 Rows successfully loaded.
0 Rows not loaded due to data errors.
0 Rows not loaded because all WHEN clauses were failed.
0 Rows not loaded because all fields were null.
Bind array size not used in direct path.
Column array rows : 5000
Stream buffer bytes: 256000
Read buffer bytes: 1048576
Total logical records skipped: 0
Total logical records read: 4
Total logical records rejected: 0
Total logical records discarded: 0
Total stream buffers loaded by SQL*Loader main thread: 1
Total stream buffers loaded by SQL*Loader load thread: 0
Run began on Mon Dec 07 13:22:23 2009
Run ended on Mon Dec 07 13:22:23 2009
Elapsed time was: 00:00:00.75
CPU time was: 00:00:00.08
If I run using Conventional path, the REC_UPD_DT is populated with the a systimestamp value. However, this column values are null when I use the Direct path. Log files have no errors. What is the way out to achieve my objective?
Thanks in advance.
- Pavin

Sql loader direct path leaving indexes unusuable

10.2.0.3
redhat 4.5
temp tablespace 6.5 GB
datafile size 33k
I thought in version 10 you can do direct path loads without indexes going unusuable?
i saw this, but we have plenty of temp space.
http://dbasolve.com/OracleError/ora/ORA-01502.txt
Oracle Error:
ORA-01502 Oracle Index in Unusable State
Cause and Solution:
When trying to perform query on Oracle tables with select SQL statement, Oracle returns this error.
The error indicates an attempt has been made to access an index or index partition that has been marked unusable by a direct load or by a DDL operation.
The problem usually happens when using the Direct Path for the SQL*Loader, Direct Load or DDL operations. This requires enough temporary space to build all indexes of the table. If there is no enough space in TEMP tablespace, all rows will still be loaded and imported, but the indices are left with STATUS = 'INVALID'.
Invalid indexes can be checked with down SQL statement.
SQL>SELECT * from USER_INDEXES WHERE STATUS = 'INVALID';
Solution to this error is simple. You can:
1. Drop the specified index and/or recreate the index
2. Rebuild the specified index
3. Rebuild the unusable index partition
Generally, the following SQL manipulation language will be able to rebuild the unusable index:
SQL>ALTER INDEX index_name REBUILD online

Hi Alan,
How big is this table and what sort of indexes are they?
An index update using SQL*Loader unrecoverable direct path load is achieved by an isolated sort followed by a nologging merge of the old index and the new mini-index into a new index segment (this according to seminar notes by Jonathan Lewis). This will conceivably take a long time for a large table / large index.
Performance improvements? Are you loading all records into a new partition?
Cheers,
Colin

SQL Loader, direct path and indexes on partition tables

Hi,
I have a big partitioned table and I have two indexes on it.
When I use SQL Loader to upload data to my table with using OPTIONS (DIRECT=TRUE) UNRECOVERABLE , then it takes almost half hour to load just one record!!
When I remove OPTIONS (DIRECT=TRUE), then it takes just two seconds to upload the same one record.
Am I missing anything? Can I use direct path load on indexed partitioned tables and have reasonable load time ?
An scheduled external job loads almost 100,000 records into this table every hour and I am trying to make the sql*loader performance it as fast as possible.
Any help would be appreciated,
Alan

Hi Alan,
How big is this table and what sort of indexes are they?
An index update using SQL*Loader unrecoverable direct path load is achieved by an isolated sort followed by a nologging merge of the old index and the new mini-index into a new index segment (this according to seminar notes by Jonathan Lewis). This will conceivably take a long time for a large table / large index.
Performance improvements? Are you loading all records into a new partition?
Cheers,
Colin

SQL Loader Direct Path and Nulls

I'm using Sql Loader with Direct Path and I cannot find a way to remove trailing spaces in varchar2 fields.
I'm trying to avoid updating the tables after loading.
Any ideas?

Hi Alan,
How big is this table and what sort of indexes are they?
An index update using SQL*Loader unrecoverable direct path load is achieved by an isolated sort followed by a nologging merge of the old index and the new mini-index into a new index segment (this according to seminar notes by Jonathan Lewis). This will conceivably take a long time for a large table / large index.
Performance improvements? Are you loading all records into a new partition?
Cheers,
Colin

Loading VARRAY's with SQL loader, direct path in 9i?

Isn't it possible to load VARRAY's with SQLloader and direct path in a Oracle 9i database?
/Magnus Hornstrom
mailto:[email protected]

Daniel,
I appreciate your response alot.
Now a follow on. You say that SQL*Loader is the fastest way to get data into Oracle Spatial even though the conventional path. How does SQL*Loader do this? Doesn't it use OCI?
I just can't help but think that coverting my data to SQL*Loader format, and then having it parse it and send it to the database in some form must be slower than me just sending whatever SQL Loader sends to the DB myself using OCI. Am I missing something?
The SQL Loader documentation seems to suggest that SQL*Loader just turns the input into alot of INSERT stateemnts. If so, can't I just do this?
My performance with OCI and INSERT statements isn't very good, but I believe I am doing one transaction per insert. Might I be as well off to concentrate on fine tuning my OCI based code using INSERTs?
I will actually do some time tests myself, but I would appreciate your opinion.
Once again thanks for the great info you have provided.

Errors when using sql loader direct path with a nvl functin on a date field

I thought I read in 10g that all sql functions now work in direct path mode.
the following function works without direct path mode. When we turn direct path mode on the load fails. When we take the function out in direct path mode it works.
mydate DATE "YYYY-MM-DD" "nvl(:mydatedate,to_date('9999-01-01','YYYY-MM-DD'))"
with direct path mode I get:
EVERY record gets rejected with this error.
Record 10: Rejected - Error on table MYTABLE
ORA-00604: error occurred at recursive SQL level 1
ORA-01861: literal does not match format string
again works fine without direct=true, works fine with direct=true if i get rid of the function.
any ideas?
Message was edited by:
Guess2
Message was edited by:
Guess2

like usual your posts are completely useless daniel.
I could have swarn that oracle advertised that you can use functions with 9i or 10g release in the new features guys or the first releases.

Sql Loader Direct Path Upload

I am trying DIRECT path upload to insert data from flat file to oracle table which has some indexes.The table is used by a procedure for dml operations on another table.
Previously,we used conventional path upload with PARALLEL=TRUE option.
I read that DIRECT upload affects indexes,and since I use the table being populated by direct upload for some other dml operation I don't want any performance degradation to other operations in the procedure.
How do I inspect the impact of DIRECT upload on my tables and procedure execution.
Also,are there any other side-effects of DIRECT Path Upload.
Any ideas?

since I use the table being populated by direct upload for
some other dml operationOne of the restrictions for Direct Path Loads is
that the tables to be loaded do not have any active transactions pending.
For other restrictions on using direct path loads and conditions causing a direct path load to leave an index in usables state see
http://download-west.oracle.com/docs/cd/B14117_01/server.101/b10825/ldr_modes.htm#i1008815

Oracle RAC and sql loader

hi all,
We have Oracle 10g RAC on IBM AIX
Today i will be uploading data in a table through sql loader.
I want that at the time of insertion only data which i am inserting goes into the table,As i am using sequence on particular column A.
Problem is If i shutdown database then i will not to able to insert data and if i doesnt shutdown db value for that column A is generated automatically when application works.
i have recommanded that we should stop connectivity of application with database then i will perform insertion but some people are saying no.Find other method
What steps should i do?

Ok i thing my question was not clear
I have table XYZ
A B C
1 2 3
4 4 5
if application is connected to database value to Coulmn is incremented
A B C
1 2 3
4 4 5
5 6 7
i have to insert into table XYZ using sql loader giving sequence (4,1) on column A.
But if it is connected to application it will give me error of PK.
as Value 5 is already inserted by application so how can i stop this.
so that i can have the max value of coulmn and start sequence
Hope this time make u clear.
sorry for bad english

Importing to a Oracle Table from SQL Loader Fails

Hi ,
When I try to upload one xml file from my server to my table in oracle server using sql loader it fails at times.Some times it works perfectly.
This is a daily process which automatically dumps data to my oracle.
Please find the error log :
SQL*Loader: Release 10.2.0.4.0 - Production on Thu Dec 5 04:07:32 2013
Copyright (c) 1982, 2007, Oracle. All rights reserved.
Control File:   xmlFeedDelta.ctl
Data File:      xmlFileNames_Delta.txt
Bad File:     xmlFileNames_Delta.bad
Discard File: none specified
(Allow all discards)
Number to load: ALL
Number to skip: 0
Errors allowed: 1000
Bind array:     50000 rows, maximum of 256000 bytes
Continuation:    none specified
Path used:      Conventional
Table XMLFEEDDELTA, loaded from every logical record.
Insert option in effect for this table: APPEND
   Column Name                  Position   Len Term Encl Datatype
FILENAME                            FIRST 4000   ,       CHARACTER
FILECONTENT                       DERIVED     * EOF      CHARACTER
    Dynamic LOBFILE. Filename in field FILENAME
value used for ROWS parameter changed from 50000 to 63
SQL*Loader-643: error executing INSERT statement for table XMLFEEDDELTA
ORA-03113: end-of-file on communication channel
SQL*Loader-2026: the load was aborted because SQL Loader cannot continue.
Table XMLFEEDDELTA:
0 Rows successfully loaded.
0 Rows not loaded due to data errors.
0 Rows not loaded because all WHEN clauses were failed.
0 Rows not loaded because all fields were null.
Space allocated for bind array:                 252378 bytes(63 rows)
Read   buffer bytes: 1048576
Total logical records skipped:          0
Total logical records read:             1
Total logical records rejected:         0
Total logical records discarded:        0
Run began on Thu Dec 05 04:07:32 2013
Run ended on Thu Dec 05 04:08:42 2013
Elapsed time was:     00:01:10.05
CPU time was:         00:00:00.28
My Control File Looks like this :
LOAD DATA
INFILE xmlFileNames_Delta.txt
INTO TABLE xmlFeedDelta APPEND
fields terminated by ','
filename CHAR(4000),
filecontent LOBFILE(filename) terminated by eof
My Database version :
Oracle Database 11g Enterprise Edition Release 11.2.0.2.0 - 64bit Production
PL/SQL Release 11.2.0.2.0 - Production
"CORE 11.2.0.2.0 Production"
TNS for Linux: Version 11.2.0.2.0 - Production
NLSRTL Version 11.2.0.2.0 - Production
I am not sure why this is happening at times . Any help would be appreciated.

Hi,
have you tried with the FILLER command like
LOAD DATA
INFILE xmlFileNames_Delta.txt
INTO TABLE xmlFeedDelta APPEND
fields terminated by ','
filename FILLER CHAR(4000),
filecontent LOBFILE(filename) terminated by eof

Installing Oracle Database and SQL Developer

Hi, I was looking for some help/advice in regards to installing an Oracle Database and SQL Developer onto a Windows 7 32bit machine. I have been asked to create a database to store data that I create through reports in Excel so I am required to create a local database but not sure how to go about doing this. Also not sure if I install the Express Edition or Oracle Database 11g R2. If somebody could please advise on how I can get this started or even any links/tutorials that would guide me in how to get this installed with a localhost.
Thanks in advanc.

318f20b8-a3d0-4fb4-bb0f-73785250b7d4 wrote:
Thanks guys, I have managed to install Oracle 11gR2 and I have setup a database using the dbca tool and I can connect to my new database. One thing that I have noticed is that when i look into my tables in sql developer i see loads of different tables i.e. def$_, logmnr_, etc and I am not sure why this is as I expected an empty tablespace.
Thanks btw for all your help.
One of the key principles of relational databases is that they use their own facilities to manage themselves. That means the database management system uses the database to manage data about the database using exactly the same facilities (tablespaces, tables, user accounts, procedures, etc) that an application uses to keep data for the application. See "data dictionary" So there is no such thing as a completely empty database.
As said by others, you do NOT want to be creating your own tables under oracle's schemas (users) or in oracle's own tablespaces (esp. SYSTEM and SYSAUX). Create your own user account/schema. Create your own tablespace. Make that tablespace the default for that user.

How to provide joins between oracle tables and sql server tables

Hi,
I have a requirement that i need to generate a report form two different data base. i.e Oracle and Sql Server.
how to provide joins between oracle tables and sql server tables ? Any help on this
Regards,
Malli

user10675696 wrote:
I have a requirement that i need to generate a report form two different data base. i.e Oracle and Sql Server. Bad idea most times. Heterogeneous joins do not exactly scale and performance can be severely degraded by network speed and b/w availability. And there is nothing you can do in the application and database layers to address performance issue at the network level in this case - your code's performance is simply at the mercy of network performance. With a single glaring fact - network performance is continually degrading. All the time. Always. Until it is upgraded. When the performance degradation starts all over again.
If the tables are not small (few 1000 rows each) and row volumes static, I would not consider doing a heterogeneous join. Instead I would rather go for a materialised view on the Oracle side, use a proper table and index structure, and do a local database join.

Oracle Replication and SQL*Load direct Path

Similar Messages

Maybe you are looking for