SQL UN loader ????

Hi there
Does anyone know of a SQL UN Loader, that can be run on the client?
Getting data out of oracle into ie CSV, TXT files, BLOBS-> files etc?
The very opposite of SQL Loader in fact?
Mette
PS I know that we can all just write sql*plus scrips, java, c# applications - but a more generic version would be nice - and maybe someone out there knows of one appl. like that

HI
Michaels2: I found a reference to lob.zip on the website you referenced: [http://www.dbatools.net/mytools/unload_upload_oracle_lobs.html] and this solves 50% of our problems. Thanks a lot - this is a cmd prompt tool, which we can automate with the job sceduler from any client. Great ! I also downloaded and have done a little test of the ociuldr - and it works just a perfectly - and even creating SQL Loader statements for the other way sa well. Thank you very much for this info. Now its gonne be a nice weekend after all .....
Others: We dont have - and never will get - access to the server at file level, so the solution above is very nice and just what I was looking for. We do have Enterprise Edition, java etc etc etc enabled - but with no rights to access server this does not get us very far. Thats why we are looking for a reversed "SQL Loader" :-)
regards
Mette
Edited by: mettemusens on 2009-02-06 14:33

Similar Messages

Essbase substitution variable used for SQL Dimension load-URGENT

Hi All,
I am creating one dimension through SQL Dimension load rule writing by parent child build rule. The SQL query is like below---
distinct trim(dist_mobile),trim(ret_mobile) from dist_retailer al1 where al1.proc_dt=to_date(&Proc_date,'dd/mm/yyyy')
When I fixed the value of proc_date (e.g.- '27/09/2008') then it is retrieving data properly, but when I try to make the same as a essbase variable as shown above (to give user the priviledge to select for which date data will be fetched from RDBMS to create the dim) and after setting the aforesaid value of that variable in essbase subs var screen and click 'OK/Retrieve' in rule,it is returning no rows executing the query as well as no error coming.
I guess in query nothing is wrong but the subs variable part is not working. Can anybody please give some insight to resolve the problem.
In this context, I would like to mention that I will have to use substitution variable in some other rule file also to automate the load process. How can I use essbase subs variable in SQL load rule? Could you please show me a proper summarise query illustrating how to use the subs var in aforesaid purpose for the sake of understanding?
Thanks in advance.

Hi,
Sorry for delayed response. Appreciate your inputs. At last I have found out the solution which is as below----
distinct trim(dist_mobile),trim(ret_mobile) from dist_retailer al1 where al1.proc_dt=to_date('&Proc_Dt','dd/mm/yyyy')
after setting the substitution variable as Proc_Dt and set the value as 27/09/2008 (without any quotation mark or such sign) now I checked and now it is retrieving fine.
Thanks & Regards.

ORA-06520: PL/SQL: Error loading external library

Hi
I am trying to use an external C program from sql using following steps.
copy the dll in dir ORACLE/806/BIN
create or replace library outs as
'e$/ORACLE/806/BIN/SmlCrcChecking.dll'
create or replace FUNCTION plsCheck_File_CRC (fname Varchar2)
return boolean
AS LANGUAGE C
library outs
name "Check_File_CRC";
and call the dll using
select plsCheck_File_CRC('c:\mttest.txt') from dual;
The dll is notc alled the message is
ERROR at line 1:
ORA-06520: PL/SQL: Error loading external library
ORA-06522: Unable to load DLL
Please help
Prashant

I am trying to create a Pl/SQL test env using
create or replace procedure file_test
(fname in varchar2,
status out number)
is
begin
If Check_File_CRC (fname) then
status:=0;
else
status:=1;
end if;
end;
However the following errros are displayed
Errors for PROCEDURE FILE_TEST
6/1 PL/SQL: Statement ignored
6/4 PLS-00382: expression is of wrong type
mY TNSNAMES.ORA contains following
additions
EXTPROC_CONNECTION_DATA =
(DESCRIPTION =
(ADDRESS_LIST =
(ADDRESS = (PROTOCOL = IPC)(KEY = EXTPROC0))
(CONNECT_DATA = (SID = PLSExtProc)(PRESENTATION = RO))
extproc_connection_data.world =
(DESCRIPTION =
(ADDRESS =
(PROTOCOL = IPC)
(KEY = extproc))
(CONNECT_DATA =
(SID = extproc)
AND LISTNER.ORA
SID_LIST_LISTENER =
(SID_LIST =
(SID_DESC =
(SID_NAME = PLSExtProc)
(ORACLE_HOME = D:\oracle\ora81)
(PROGRAM = extproc)
Message was edited by:
prashant13

Exel to SQL DFTask loads some Null records

I have an SSIS package to import Excel data into SQL. When I look at Excel I dont see ANY Null rows. However when I run the SSIS package and check SQL, It loads some NULL RECORDS IN SQL. WHy is that It loads Null records in SQL, When i cannot see the Null
recs In Excel?

That's because the person who created the Excel file and added the data to it pressed the "Enter" key on some of the empty cells under that specific column. In excel, such a row might look like any other empty, non-data row but for SSIS, its an
actual data row except that it doesn't have any value. SSIS treats a missing data value as a DBNULL and that's what you saw.
There are multiple ways of fixing this.
1. Educate the creator of the excel to be more careful at the time of data entry.
2. Make that target column in the DB table as NOT NULL.
3. Handle such "empty" data values as exception inside your SSIS (using a data flow task and ignoring such rows).
4. Switch to using CSV file format instead of excel format.
5. All of the above :)
Hope this helps.
Cheers!
Muqadder.

Pull the data from Azure sql and load into the on premises data base

Hi friends,
I have small requirement here, I have to pull the data from Azure sql and load it to the on premises data base.
I am using Azure sync process to do that, but it will take so much time to do that..
I know that azure have some limitations, is there any other way to do this..
Please help me on this...

Hello,
Check the program...
SAPBC480_DEMO.
Check the below threads
https://www.sdn.sap.com/irj/sdn/go/portal/prtroot/docs/library/uuid/88e7ea34-0501-0010-95b0-ed14cfbeb85a
https://www.sdn.sap.com/irj/sdn/go/portal/prtroot/docs/library/uuid/bfbcd790-0201-0010-679d-e36a3c6b89fa
Thanks
Seshu

Sql and load rule file query

Currently loading the data with the script , which it contatins
Retreving data from oracle through SQL interface on to rules files
and loading the data.
just my question is , if i get 0 records from oracle , i dont want to proceed with the loading of data (which is no useful ).
Can i stop the loading if records are zero.

As far as I know there is nothing in MaxLIt that could tell that there were no rows.The MaxL statement would return a zero code saying that the process worked ok. It would be prossible in your batch script to have mutiple calls to MaxL and between them to do something like make a call to your sql DB getting the count of rows for your query. Investigate the result and if it is zero, then get out of your load and calc process (I guess you could do that even before you try to load).
Or how about scheduling a job that creates an output file with the count if it is greater than zero. If it is, then run the batch load process if it is not, then bypass it. There are multiple ways to do this outside of Essbase, just not in it.

SQL Developer load Java error

Hello, I'm trying to load Java code to my Oracle SQL developer. On Load java, I choose Java source from my hdd (from NetBeans destination folder) and I get the below error. Any help?
error code:
    Error in Source Code
    &Exception in Stack Trace 1 of 1
    java.sql.SQLSyntaxErrorException: ORA-24344: success with compilation error
    ORA-06512: at line 1
        at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:447)
        at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:396)
        at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:879)
          ...... here is the code. It works well in NetBeans and I'm trying to load it in SQL Developer in .java file
package c2_01_signhelloworld;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.security.GeneralSecurityException;
import java.security.KeyStore;
import java.security.PrivateKey;
import java.security.Security;
import java.security.cert.Certificate;
import org.bouncycastle.jce.provider.BouncyCastleProvider;
import com.itextpdf.text.DocumentException;
import com.itextpdf.text.Rectangle;
import com.itextpdf.text.pdf.PdfReader;
import com.itextpdf.text.pdf.PdfSignatureAppearance;
import com.itextpdf.text.pdf.PdfStamper;
import com.itextpdf.text.pdf.security.BouncyCastleDigest;
import com.itextpdf.text.pdf.security.DigestAlgorithms;
import com.itextpdf.text.pdf.security.ExternalDigest;
import com.itextpdf.text.pdf.security.ExternalSignature;
import com.itextpdf.text.pdf.security.MakeSignature;
import com.itextpdf.text.pdf.security.MakeSignature.CryptoStandard;
import com.itextpdf.text.pdf.security.PrivateKeySignature;
public class C2_01_SignHelloWorld {
    public static final String KEYSTORE = "C:/Users/myFile/Documents/itext/2/ks";
    public static final char[] PASSWORD = "password".toCharArray();
    public static final String SRC = "C:/Users/myfile/Documents/itext/2/unsigned1.pdf";
    public static final String DEST = "C:/Users/myfile/Documents/itext/2/unsigned1_signed.pdf";
    public void sign(String src, String dest,
            Certificate[] chain,
            PrivateKey pk, String digestAlgorithm, String provider,
            CryptoStandard subfilter,
            String reason, String location)
                    throws GeneralSecurityException, IOException, DocumentException {
        // Creating the reader and the stamper
        PdfReader reader = new PdfReader(src);
        FileOutputStream os = new FileOutputStream(dest);
        PdfStamper stamper = PdfStamper.createSignature(reader, os, '\0');
        // Creating the appearance
        PdfSignatureAppearance appearance = stamper.getSignatureAppearance();
        appearance.setReason(reason);
        appearance.setLocation(location);
        appearance.setVisibleSignature(new Rectangle(36, 748, 144, 780), 1, "sig");
        // Creating the signature
        ExternalDigest digest = new BouncyCastleDigest();
        ExternalSignature signature = new PrivateKeySignature(pk, digestAlgorithm, provider);
        MakeSignature.signDetached(appearance, digest, signature, chain, null, null, null, 0, subfilter);
    public static void main(String[] args) throws GeneralSecurityException, IOException, DocumentException {
        BouncyCastleProvider provider = new BouncyCastleProvider();
        Security.addProvider(provider);
        KeyStore ks = KeyStore.getInstance(KeyStore.getDefaultType());
        ks.load(new FileInputStream(KEYSTORE), PASSWORD);
        String alias = (String)ks.aliases().nextElement();
        PrivateKey pk = (PrivateKey) ks.getKey(alias, PASSWORD);
        Certificate[] chain = ks.getCertificateChain(alias);
        C2_01_SignHelloWorld app = new C2_01_SignHelloWorld();
        app.sign(SRC, String.format(DEST, 1), chain, pk, DigestAlgorithms.SHA256, provider.getName(), CryptoStandard.CMS, "Test 1", "Ghent");
        app.sign(SRC, String.format(DEST, 2), chain, pk, DigestAlgorithms.SHA512, provider.getName(), CryptoStandard.CMS, "Test 2", "Ghent");
        app.sign(SRC, String.format(DEST, 3), chain, pk, DigestAlgorithms.SHA256, provider.getName(), CryptoStandard.CADES, "Test 3", "Ghent");
        app.sign(SRC, String.format(DEST, 4), chain, pk, DigestAlgorithms.RIPEMD160, provider.getName(), CryptoStandard.CADES, "Test 4", "Ghent");

Hi,
This could be one of those cases where using Java 7 versus Java 6 makes a difference, or perhaps there is some conflict with the JDBC driver version. So, a few things to try...
1. Make sure your installation's SetJavaHome line in ...\sqldeveloper\sqldeveloper\bin\sqldeveloper.conf points to a Java 6 JDK.
2. If (1) does not work, also uncheck the Use OCI/Thick driver box in Tools -> Preferences -> Database -> Advanced.
3. If (1) and (2) do not work, also copy the ojdbc6.jar from any Oracle client you might have installed to your installation's ...\sqldeveloper\jdbc\lib directory
I would think doing just (1) should be sufficient. If on Mac OS, use sqldeveloper-Darwin.conf instead.
Hope this helps and please post back here and let us know, one way or the other,
Gary
SQL Developer Team

SQL Developer loading JDBC error

Hi,
I am using SQL Developer to connect Oracle. it was working fine till yesterday, but today when i open SQL Developer and tried to connect it, but it is showing an error msg: "Failure -Error loading native JDBC driver library"
Pls assist me on this. Thanks in advance
SQL Developer version: 1.1.3
Java Platform: 1.5.0_06

Thanks for your reply,
I did not install or remove anything since yesterday.
what i have to do for this to work SQL Developer properly.
Great opportunity to upgrade to 1.5? -- means, am i need to upgrade to 1.5 ?
if yes then how, pls specify me the process for upgrade.

Please help: WebLogic + BI + Bi SQl Group Loader

Hi all, i have big problem with solution on my company project. I please somebody help me.
This is my problem:
I have bifoundation_domain :
WebLogic Server Version: 10.3.5.0
EM 11g
Oracle Business Intelligence 11.1.1.7.0
with this structure:
bifoundation_domain
|- AdminServer
|- bi_cluster
|- bi_server1
So and i need use Weblogic embedded LDAP (DefaultAuthenticator in realms security providers) and i need loading GROUPS from DATABASE. I read and tried a lot of articles, blogs, manuals but
within positive result.
My procedure is:
In WLS console :
- create jdbc datasource with name "bip_apps_DS"
- create BI SQL Group provider (with name BIGroupLoader) with this settings
<sec:authentication-provider xmlns:ext="http://xmlns.oracle.com/weblogic/security/extension" xsi:type="ext:bisql-group-providerType">
<sec:name>BIGroupLoader</sec:name>
<sec:control-flag>OPTIONAL</sec:control-flag>
<ext:data-source-jndi-name>bip_apps_DS</ext:data-source-jndi-name>
<ext:sql-list-member-groups>SELECT ROLE_NAME FROM V_SYS_AUTH_ROLES WHERE LOGIN_NAME = ?</ext:sql-list-member-groups>
<ext:sql-list-groups>SELECT NAME FROM UA_ROLES WHERE NAME LIKE ?</ext:sql-list-groups>
<ext:sql-group-exists>SELECT NAME FROM UA_ROLES WHERE NAME = ?</ext:sql-group-exists>
<ext:sql-is-member>SELECT LOGIN_NAME FROM V_SYS_AUTH_ROLES WHERE ROLE_NAME = ? AND LOGIN_NAME = ?</ext:sql-is-member>
<ext:sql-get-group-description>SELECT DESCRIPTION FROM UA_ROLES WHERE NAME = ?</ext:sql-get-group-description>
</sec:authentication-provider>
(my DB schema is correct)
and i move him on first place in providers list.
So after these steps in WLS console i see in security realm->groups my groups from DB. Everything is OK.
Now i need use GROUPS from my database in EM in the context of create BI users roles (maping BI application roles on GROUPS (enterprise roles)).
So i created a database adapter for the Virtualized Identity Store
this is it:
<?xml version = '1.0' encoding = 'UTF-8'?>
<adapters schvers="303" version="1" xmlns="http://www.octetstring.com/schemas/Adapters" xmlns:adapters="http://www.w3.org/2001/XMLSchema-instance">
<dataBase id="directoryType" version="0">
<root>%ROOT%</root>
<active>true</active>
<serverType>directoryType</serverType>
<routing>
<critical>true</critical>
<priority>50</priority>
<inclusionFilter/>
<exclusionFilter/>
<plugin/>
<retrieve/>
<store/>
<visible>Yes</visible>
<levels>-1</levels>
<bind>true</bind>
<bind-adapters/>
<views/>
<dnpattern/>
</routing>
<pluginChains xmlns="http://xmlns.oracle.com/iam/management/ovd/config/plugins">
<plugins>
<plugin>
<name>VirtualAttribute</name>
<class>oracle.ods.virtualization.engine.chain.plugins.virtualattr.VirtualAttributePlugin</class>
<initParams>
<param name="ReplaceAttribute" value="uniqueMember={cn=%uniquemember%,ou=people,ou=myrealm,dc=bifoundation_domain}"/>
</initParams>
</plugin>
</plugins>
<default>
<plugin name="VirtualAttribute"/>
</default>
<add/>
<bind/>
<delete/>
<get/>
<modify/>
<rename/>
</pluginChains>
<driver>oracle.jdbc.driver.OracleDriver</driver>
<url>%URL%</url>
<user>%USER%</user>
<password>%PASSWORD%</password>
<ignoreObjectClassOnModify>false</ignoreObjectClassOnModify>
<includeInheritedObjectClasses>true</includeInheritedObjectClasses>
<maxConnections>10</maxConnections>
<mapping>
<joins/>
<objectClass name="groupofuniquenames" rdn="cn">
<attribute ldap="cn" table="V_SYS_AUTH_ROLES" field="ROLE_NAME" type=""/>
<attribute ldap="description" table="V_SYS_AUTH_ROLES" field="ROLE_NAME" type=""/>
<attribute ldap="uniquemember" table="V_SYS_AUTH_ROLES" field="LOGIN_NAME" type=""/>
</objectClass>
</mapping>
<useCaseInsensitiveSearch>true</useCaseInsensitiveSearch>
<connectionWaitTimeout>10</connectionWaitTimeout>
<oracleNetConnectTimeout>0</oracleNetConnectTimeout>
<validateConnection>false</validateConnection>
</dataBase>
</adapters>
and run command to register:
./libovdadapterconfig.sh -adapterName BIGroupLoader -adapterTemplate bi_sql_groups_adapter_template.xml
-host localhost -port 7001 -userName weblogic -domainPath /OFM/BI/user_projects/domains/bifoundation_domain
-dataStore DB -root ou=people,ou=myrealm,dc=bifoundation_domain -contextName default -dataSourceJNDIName bip_apps_DS
Adapter is creatted successfully within errors!
I restarted managed server(bi_server1) and AdminServer, all bi commponets etc. BUT WITHOUT RESULT. I still dont see GROUPS in Enterprise manager in
BI->coreapplication->security->application roles
I tried set in security setting of webLogic domain in EM virtualize=true.
This procedure is described on all sites but not funkcionaly for me. Do you know somebody where is mistake? Etc. need i installing OVD server? I dont know. Please helm me.

Hi,
You will not be able to see groups that you have defined in database in EM. To assign a database group to a role in EM go to BI->coreapplication->security->application roles & search for groups locate Advance Option at the bottom and check the box "Check to enter principal name here instead of searching from above. This option can be used for advanced scenarios related to custom authenticators". Now select Groups under Type and enter the name (exactly same) of the database group manually. Now you should be able to assign database groups to users authenticated through weblogic embedded LDAP.
Thanks

SQL server - Load balancing

Hi,
I am looking for a load balancing feature in SQL cluster likely RAC in ORACLE.
Can you please share anything like this ?
Thanks,
Vinodh Selvaraj

There is no such thing as RAC in SQL Server, but there are a few ways you could implement some sort of load balancing.
You could, for an instance, have two copies of your database (in different servers) synchronize by using merge replication, and implement load balancing logic in your application.
There's also the Parallel Data Warehouse solution. Requests are distributed across several SQL Server instances, all of them running the same logical set of databases.
http://blogs.technet.com/b/dataplatforminsider/archive/2013/11/15/how-does-sql-server-parallel-data-warehouse-pdw-deliver-the-performance-that-it-does.aspx
You can also implement some HA solutions such as AlwaysOn AG, or maybe log shipping, but keep in mind that the passive node(s) will be read-only, and you'll have to implement the load balancing yourself.
Just because there are clouds in the sky it doesn't mean it isn't blue. But someone will come and argue that in addition to clouds, birds, airplanes, pollution, sunsets, daltonism and nuclear bombs, all adding different colours to the sky, this
is an undocumented behavior and should not be relied upon.

PL/SQL Bulk Loading

Hello,
I have one question regarding bulk loading. I did lot of bulk loading.
But my requirement is to call function which will do some DML operation and give ref key so that i can insert to fact table.
Because i can't use DML function in select statement. (which will give error). otherway is using autonomous transaction. which i tried working but performance is very slow.
How to call this function inside bulk loading process.
Help !!
xx_f is function which is using autonmous transction,
See my sample code
declare
cursor c1 is select a,b,c from xx;
type l_a is table of xx.a%type;
type l_b is table of xx.b%type;
type l_c is table of xx.c%type;
v_a l_a;
v_b l_b;
v_c l_c;
begin
open c1;
loop
fetch c1 bulk collect into v_a,v_b,v_c limit 1000;
exit when c1%notfound;
begin
forall i in 1..v_a.count
insert into xxyy
(a,b,c) values (xx_f(v_a(i),xx_f(v_b(i),xx_f(v_c(i));
commit;
end bulkload;
end loop;
close c1;
end;
I just want to call xx_f function without autonoumous transaction.
but with bulk loading. Please let me if you need more details
Thanks
yreddyr

Can you show the code for xx_f? Does it do DML, or just transformations on the columns?
Depending on what it does, an alternative could be something like:
DECLARE
   CURSOR c1 IS
      SELECT xx_f(a), xx_f(b), xx_f(c) FROM xx;
   TYPE l_a IS TABLE OF whatever xx_f returns;
   TYPE l_b IS TABLE OF whatever xx_f returns;
   TYPE l_c IS TABLE OF whatever xx_f returns;
   v_a l_a;
   v_b l_b;
   v_c l_c;
BEGIN
   OPEN c1;
   LOOP
      FETCH c1 BULK COLLECT INTO v_a, v_b, v_c LIMIT 1000;
      BEGIN
         FORALL i IN 1..v_a.COUNT
            INSERT INTO xxyy (a, b, c)
            VALUES (v_a(i), v_b(i), v_c(i));
      END;
      EXIT WHEN c1%NOTFOUND;
   END LOOP;
   CLOSE c1;
END;John

SQL Developer Loading Problem

Hi Friends,
I Successfully Install Oracle 11g in my system. When i enter into SQL Developer the screen was opened but it was asking username and password continuously. So, i couldn't work.Please tell me the solution for this. Can anyone help me ?
Thanks and Regards,
Bala ....

What version of SQL Developer are you having? FYI. the latest 1.5.4.59.40 can be downloaded from OTN.
Information regarding the OS, JDK/JRE and Locale will also be a valuable input in zeroing in on the issue. Also if you have 11g client installed can you see if you can connect successfully using SQL*PLUS using the same account information?
Edited by: vasan_kps on Apr 2, 2009 11:42 AM

Sql query/loading help

SQL> create table t1(col1 varchar2(20));
Table created.
i want to insert abc&d into the col1 of t1.
but it is treating $ to eneteer the variable
i have these value abc&d in the flat file.how can i enter this
SQL> insert into t1 values('abc&d');
Enter value for d:
SQL> insert into t1 values('abc'||'&'||'d');
1 row created.

In addition to the responses above, you can set the escape character as well, if you:
- want to enter data containing ampersand characters ("&") as well as
- want to use ampersand for substition variables in your script.
test@ORA10G>
test@ORA10G> show escape
escape OFF
test@ORA10G> show define
define "&" (hex 26)
test@ORA10G>
test@ORA10G> drop table t;
Table dropped.
test@ORA10G> create table t (x varchar2(20));
Table created.
test@ORA10G> insert into t(x) values ('ab&c');
Enter value for c:
old   1: insert into t(x) values ('ab&c')
new   1: insert into t(x) values ('ab')
1 row created.
test@ORA10G>
test@ORA10G> -- set the escape character
test@ORA10G> set escape '\'
test@ORA10G>
test@ORA10G> -- escape character is set and define is ON
test@ORA10G> -- so now you can escape the "&" while inserting
test@ORA10G> -- and use it as it is for reading values of
test@ORA10G> -- substitution variables
test@ORA10G>
test@ORA10G> -- (1) ampersand used with escape character
test@ORA10G> insert into t(x) values ('laurel \& hardy');
1 row created.
test@ORA10G>
test@ORA10G> -- (2) ampersand used with substitution variable
test@ORA10G> insert into t(x) values ('yin&yang');
Enter value for yang: and yang
old   1: insert into t(x) values ('yin&yang')
new   1: insert into t(x) values ('yin and yang')
1 row created.
test@ORA10G>
test@ORA10G> commit;
Commit complete.
test@ORA10G> select * from t;
X
ab
laurel & hardy
yin and yang
test@ORA10G>
test@ORA10G> -- to revert back, set escape off
test@ORA10G> set escape off
test@ORA10G>
test@ORA10G>hth,
pratz

SQL data Load rule with Substitution variable

Hi,
I have a data load rule with MySQL as a data source. I have requirement where user does not want to see Current day data in cube and they keep changing days not to be loaded.
I have setup the substitution variable SUBv as '2009-11-04' and have scripts to update it daily.
Data Load rule in where condition contains
brand = 'HP' and region = 'east' and date <> &SUBv
Now, above where condition does not work for date <> &SUBv but if I set substitution variable SUBv as "brand = 'HP' and region = 'east' and date <> '2009-11-04' " and set load rule where condition with just &SUBv it would just work fine. I dont know whats wrong in 1st place.
I dont want to create multiple where condition in substitution variable (Don't we have limit on them?). I have 40+ load rules which might use this SUBv substitution variable if I find a way to define it so that it work as "brand = 'HP' and region = 'east' and date <> &SUBv"
Thanks,
Vikram

I did some research and came to know when using substitution variable in Load rule need to keep substitution variable first in string..
It worked out magically.

How can I load my data faster? Is there a SQL solution instead of PL/SQL?

11.2.0.2
Solaris 10 sparc
I need to backfill invoices from a customer. The raw data has 3.1 million records. I have used pl/sql to load these invoices into our system (dev), however, our issue is the amount of time it's taking to run the load - effectively running at approx 4 hours. (Raw data has been loaded into a staging table)
My research keeps coming back to one concept: sql is faster than pl/sql. Where I'm stuck is the need to programmatically load the data. The invoice table has a sequence on it (primary key = invoice_id)...the invoice_header and invoice_address tables use the invoice_id as a foreign key. So my script takes advantage of knowing the primary key and uses that on the subsequent inserts to the subordinate invoice_header and invoice_address tables, respectively.
My script is below. What I'm asking is if there are other ideas on the quickest way to load this data...what am I not considering? I have to load the data in dev, qa, then production so the sequences and such change between the environments. I've dummied down the code to protect the customer; syntax and correctness of the code posted here (on the forum) is moot...it's only posted to give the framework for what I currently have.
Any advice would be greatly appreciated; how can I load the data faster knowing that I need to know sequence values for inserts into other tables?
DECLARE
   v_inv_id        invoice.invoice_id%TYPE;
   v_inv_addr_id    invoice_address.invoice_address_id%TYPE;
   errString        invoice_errors.sqlerrmsg%TYPE;
   v_guid          VARCHAR2 (128);
   v_str           VARCHAR2 (256);
   v_err_loc       NUMBER;
   v_count         NUMBER := 0;
   l_start_time    NUMBER;
   TYPE rec IS RECORD
      BILLING_TYPE             VARCHAR2 (256),
      CURRENCY                 VARCHAR2 (256),
      BILLING_DOCUMENT         VARCHAR2 (256),
      DROP_SHIP_IND            VARCHAR2 (256),
      TO_PO_NUMBER        VARCHAR2 (256),
      TO_PURCHASE_ORDER   VARCHAR2 (256),
      DUE_DATE                 DATE,
      BILL_DATE                DATE,
      TAX_AMT                  VARCHAR2 (256),
      PAYER_CUSTOMER           VARCHAR2 (256),
      TO_ACCT_NO          VARCHAR2 (256),
      BILL_TO_ACCT_NO          VARCHAR2 (256),
      NET_AMOUNT               VARCHAR2 (256),
      NET_AMOUNT_CURRENCY      VARCHAR2 (256),
      ORDER_DT             DATE,
      TO_CUSTOMER         VARCHAR2 (256),
      TO_NAME             VARCHAR2 (256),
      FRANCHISES       VARCHAR2 (4000),
      UPDT_DT                  DATE
   TYPE tab IS TABLE OF rec
                  INDEX BY BINARY_INTEGER;
   pltab           tab;
   CURSOR c
   IS
      SELECT   billing_type,
               currency,
               billing_document,
               drop_ship_ind,
               to_po_number,
               to_purchase_order,
               due_date,
               bill_date,
               tax_amt,
               payer_customer,
               to_acct_no,
               bill_to_acct_no,
               net_amount,
               net_amount_currency,
               order_dt,
               to_customer,
               to_name,
               franchises,
               updt_dt
        FROM   BACKFILL_INVOICES;
BEGIN
   l_start_time := DBMS_UTILITY.get_time;
   OPEN c;
   LOOP
      FETCH c
      BULK COLLECT INTO pltab
      LIMIT 1000;
      v_err_loc := 1;
      FOR i IN 1 .. pltab.COUNT
      LOOP
         BEGIN
            v_inv_id := SEQ_INVOICE_ID.NEXTVAL;
            v_guid := 'import' || TO_CHAR (CURRENT_TIMESTAMP, 'hhmissff');
            v_str := str_parser (pltab (i).FRANCHISES); --function to string parse - this could be done in advance, yes.
            v_err_loc := 2;
            v_count := v_count + 1;
            INSERT INTO    invoice nologging
                 VALUES   (v_inv_id,
                           pltab (i).BILL_DATE,
                           v_guid,
                           '111111',
                           'NONE',
                           TO_TIMESTAMP (pltab (i).BILL_DATE),
                           TO_TIMESTAMP (pltab (i).UPDT_DT),
                           'READ',
                           'PAPER',
                           pltab (i).payer_customer,
                           v_str,
                           '111111');
            v_err_loc := 3;
            INSERT INTO    invoice_header nologging
                 VALUES   (v_inv_id,
                           TRIM (LEADING 0 FROM pltab (i).billing_document), --invoice_num
                           NULL,
                           pltab (i).BILL_DATE,                 --invoice_date
                           pltab (i).TO_PO_NUMBER,
                           NULL,
                           pltab (i).net_amount,
                           NULL,
                           pltab (i).tax_amt,
                           NULL,
                           NULL,
                           pltab (i).due_date,
                           NULL,
                           NULL,
                           NULL,
                           NULL,
                           NULL,
                           TO_TIMESTAMP (SYSDATE),
                           TO_TIMESTAMP (SYSDATE),
                           PLTAB (I).NET_AMOUNT_CURRENCY,
                           (SELECT   i.bc_value
                              FROM   invsvc_owner.billing_codes i
                             WHERE   i.bc_name = PLTAB (I).BILLING_TYPE),
                           PLTAB (I).BILL_DATE);
            v_err_loc := 4;
            INSERT INTO    invoice_address nologging
                 VALUES   (invsvc_owner.SEQ_INVOICE_ADDRESS_ID.NEXTVAL,
                           v_inv_id,
                           'BLAH INITIAL',
                           pltab (i).BILL_DATE,
                           NULL,
                           pltab (i).to_acct_no,
                           NULL,
                           NULL,
                           NULL,
                           NULL,
                           NULL,
                           NULL,
                           NULL,
                           SYSTIMESTAMP,
                           NULL);
            v_err_loc := 5;
            INSERT INTO    invoice_address nologging
                 VALUES   ( SEQ_INVOICE_ADDRESS_ID.NEXTVAL,
                           v_inv_id,
                           'BLAH',
                           pltab (i).BILL_DATE,
                           NULL,
                           pltab (i).TO_ACCT_NO,
                           NULL,
                           NULL,
                           NULL,
                           NULL,
                           NULL,
                           NULL,
                           NULL,
                           SYSTIMESTAMP,
                           NULL);
            v_err_loc := 6;
            INSERT INTO    invoice_address nologging
                 VALUES   ( SEQ_INVOICE_ADDRESS_ID.NEXTVAL,
                           v_inv_id,
                           'BLAH2',
                           pltab (i).BILL_DATE,
                           NULL,
                           pltab (i).TO_CUSTOMER,
                           pltab (i).to_name,
                           NULL,
                           NULL,
                           NULL,
                           NULL,
                           NULL,
                           NULL,
                           SYSTIMESTAMP,
                           NULL);
            v_err_loc := 7;
            INSERT INTO    invoice_address nologging
                 VALUES   ( SEQ_INVOICE_ADDRESS_ID.NEXTVAL,
                           v_inv_id,
                           'BLAH3',
                           pltab (i).BILL_DATE,
                           NULL,
                           'SOME PROPRIETARY DATA',
                           NULL,
                           NULL,
                           NULL,
                           NULL,
                           NULL,
                           NULL,
                           NULL,
                           SYSTIMESTAMP,
                           NULL);
            v_err_loc := 8;
            INSERT
              INTO    invoice_event nologging (id,
                                                         eid,
                                                         root_eid,
                                                         invoice_number,
                                                         event_type,
                                                         event_email_address,
                                                         event_ts)
            VALUES   ( SEQ_INVOICE_EVENT_ID.NEXTVAL,
                      '111111',
                      '222222',
                      TRIM (LEADING 0 FROM pltab (i).billing_document),
                      'READ',
                      'some_user@some_company.com',
                      SYSTIMESTAMP);
            v_err_loc := 9;
            INSERT INTO   backfill_invoice_mapping
                 VALUES   (v_inv_id,
                           v_guid,
                           pltab (i).billing_document,
                           pltab (i).payer_customer,
                           pltab (i).net_amount);
            IF v_count = 10000
            THEN
               COMMIT;
            END IF;
         EXCEPTION
            WHEN OTHERS
            THEN
               errString := SQLERRM;
               INSERT INTO   backfill_invoice_errors
                    VALUES   (
                                pltab (i).billing_document,
                                pltab (i).payer_customer,
                                errString || ' ' || v_err_loc
               COMMIT;
         END;
      END LOOP;
      v_err_loc := 10;
      INSERT INTO   backfill_invoice_timing
           VALUES   (
                       ROUND ( (DBMS_UTILITY.get_time - l_start_time) / 100,
                              2)
                       || ' seconds.',
                       (SELECT   COUNT (1)
                          FROM   backfill_invoice_mapping),
                       (SELECT   COUNT (1)
                          FROM   backfill_invoice_errors),
                       SYSDATE
      COMMIT;
      EXIT WHEN c%NOTFOUND;
   END LOOP;
   COMMIT;
EXCEPTION
   WHEN OTHERS
   THEN
      errString := SQLERRM;
      INSERT INTO   backfill_invoice_errors
           VALUES   (NULL, NULL, errString || ' ' || v_err_loc);
      COMMIT;
END;

Hello
You could use insert all in your case and make use of sequence.NEXTVAL and sequence.CURRVAL like so (excuse any typos - I can't test without table definitions). I've done the first 2 tables, so it's just a matter of adding the rest in...
INSERT ALL
     INTO      invoice nologging
                VALUES   (     SEQ_INVOICE_ID.NEXTVAL,
                               BILL_DATE,
                                my_guid,
                                '111111',
                                'NONE',
                                CAST(BILL_DATE AS TIMESTAMP),
                                CAST(UPDT_DT AS TIMESTAMP),
                                'READ',
                                'PAPER',
                                payer_customer,
                                parsed_francises,
                                '111111'
     INTO      invoice_header
          VALUES   (      SEQ_INVOICE_ID.CURRVAL,
                    TRIM (LEADING 0 FROM billing_document), --invoice_num
                    NULL,
                    BILL_DATE,                 --invoice_date
                    TO_PO_NUMBER,
                    NULL,
                    net_amount,
                    NULL,
                    tax_amt,
                    NULL,
                    NULL,
                    due_date,
                    NULL,
                    NULL,
                    NULL,
                    NULL,
                    NULL,
                    SYSTIMESTAMP,
                    SYSTIMESTAMP,
                    NET_AMOUNT_CURRENCY,
                    bc_value,
                    BILL_DATE)
     SELECT
     src.billing_type,
          src.currency,
          src.billing_document,
          src.drop_ship_ind,
          src.to_po_number,
          src.to_purchase_order,
          src.due_date,
          src.bill_date,
          src.tax_amt,
          src.payer_customer,
          src.to_acct_no,
          src.bill_to_acct_no,
          src.net_amount,
          src.net_amount_currency,
          src.order_dt,
          src.to_customer,
          src.to_name,
          src.franchises,
          src.updt_dt,
          str_parser (src.FRANCHISES) parsed_franchises,
          'import' || TO_CHAR (CURRENT_TIMESTAMP, 'hhmissff') my_guid,
          i.bc_value
        FROM        BACKFILL_INVOICES src,
             invsvc_owner.billing_codes i
     WHERE   i.bc_name = src.BILLING_TYPE;Some things to note
1. Don't commit in a loop - you only add to the run time and load on the box ultimately reducing scalability and removing transactional integrity. Commit once at the end of the job.
2. Make sure you specify the list of columns you are inserting into as well as the values or columns you are selecting. This is good practice as it protects your code from compilation issues in the event of new columns being added to tables. Also it makes it very clear what you are inserting where.
3. If you use WHEN OTHERS THEN... to log something, make sure you either rollback or raise the exception. What you have done in your code is say - I don't care what the problem is, just commit whatever has been done. This is not good practice.
HTH
David
Edited by: Bravid on Oct 13, 2011 4:35 PM

SQL UN loader ????

Similar Messages

Maybe you are looking for