Code Template Design - SQL LOADER

Hi,
We extract data from 12 csv files which have 10+ million rows using SQL Loader into staging table. We use each of these tables in at least two different mappings to populate our Dimensions & Facts.
Now moving to CT's -
Using Code Template,
Code Template LCT_FILE_TO_ORACLE_SQLLDR and DEFAULT_ORACLE_TARGET_CT , i am able to load the data into the staging table. But this involves loading the data into work table and then into our Staging table. So we are loading our data twice.
So I decided to write my own code template which loads the data from file directly to staging table bypassing work table.
Developed an Integration CT which write's data directly to my staging table.
But the issue is , I am miissing the audit. The CT does not shows me any record count it processed. This might be issue since i don't have any work tables.
Is there a way we can capture audit ?
Regards,
Samurai.

Hi David,
I was thinking of getting the details from log file (using jytho) but did not know how to update the audit statistics. If you can get me that, it would be great.
I go through all your blogs :-) . I am following the baby step as per your blog :-) I am moving towards bulk loading.
Trying to design a CT
a) SQL Loader
1) Create a named pipe in unix to hold data into memory
2) Bulk load data from MYSQL or SYBASE into a named pipe
3) Then load the data into oracle table using SQLLoader
4) Drop the named pipe
OR
b) External Table
1) Bulk load data from MYSQL or SYBASE into a file ( zip the data)
2) Then load the data into oracle external table with preprocessor option
From our past experience in our current enviornment, quering or transforming data across external table with 10+ million rows was slower than a doing it across a table ( dba's will come after me for issuing such statements). So was trying to use SQL loader first.
Regards,
Samurai.
BTW You know me by other name.
Edited by: Samurai on Mar 4, 2010 3:12 PM

Similar Messages

Code Templates in SQL Developer 2.1

Hi all!
Looks like this feature does not work at all. Even followed the instructions from Sue's book page 99:
In SQL Developer 2.1, simply start typing the code from the
template, and the full code is displayed in the code insight drop list. Select the code
to add it to the SQL Worksheet.
Any ideas?
Thanks,
Radu

Hi,
Really disappointed to see that code template dont work in v 2.1. If it is considered that code insight can replace ctr + shift + T than this is wrong assumption.
ctr + shift + T was so easy to use. Also the code insight dont pop up for templates having PL/SQL blocks. For examples
If a template T1 is like
BEGIN
<Some processing statements>
END;
than T1 can not be used as template in 2.1 unlike 1.5.
Hope the issue is fixed very soon else users would prefer to use 1.5.
Regards,
Sandeep G
Edited by: [email protected] on Jan 28, 2010 3:37 AM

Executing code within a SQL*LOADER session for RLS

We are using RLS, and have a need to load some data using sql*loader. More specifically, we need to do the following:
set the context (sys_context)
load the data, using sys_context('xxx','xxx') in the data.
This would require the execution of a package procedure within the session that loads the data. I do not know if this is possible using sql*loader.
I have a workaround; creating a load user and creating a login trigger that calls the package procedure. This is not a very elegant solution for us and we would like to perform the procedure call within sql*loader.
We are running Oracle 10g release 1 on linux.
Thank you in advance.

Ok -
I have figured out how to do this, although it is somewhat inefficent. I am populating a table that contains carrier information: British Airways for example. The table is similar to the following:
carrier_id number(35)
carrier_code varchar2(3)
carrier_name varchar2(30)
modification_date date
There is a unique id, called a site_id, assigned to each site in our streams environment. The primary key, carrier_id, is built using this ID.
I have a function that will set the context, and then use sys_context to get the value of the siteid. I also have a function that will get the site_id from a table using the global name. Finding the site_id using these functions and inserting it through sql loader is just a matter of putting the following in the controlfile:
carrier_id "get_siteid||:carrier_id"
where carrier_id is a number from 1-n where n is the number of carriers.
If you have a more elegant way of doing this, please let me know.
Jamie McLaughlin

SQL Developer 2.1.1.64 - code template does not work in Windows Vista

I have been trying to import a code template in SQL Developer 2.1.1.64 / Windows Vista using the following steps:
1. Import template (xml) file using "Preferences > Database > SQL Formatter > Import..." option. Pressed "OK" button.
2. Clicked on "Preferences > Database > SQL Formatter > Oracle Formatting".
I expected to see the new profile under "Profile" drop-down-list-box, but it does not appear there.
I tried putting the template (xml) file under various locations, including:
(a) C:\Users\<user_name>\
(b) C:\Users\<user_name>\AppData\Roaming\SQL Developer\
Please let me know any suggestions that you may have.
Thanks in advance.

I got it to work. In step 1 that I mentioned, "OK" button should not be pressed; go to step 2 without pressing "OK" button.

Using a Table Trigger to Trap Error Codes Returned to a SQL Loader seeion

I would be grateful if anyone knows how to go about this.
I am trying to insert into a table, details of SQL Loader's rejected records by possibly using a trigger on the loading table to try and trap the error code returned to SQL Loader.
What I am trying to achieve is to insert in to a table the load or rejected record details that are in SQL Loaders log file after a load session. For example: records read:, records inserted:, records rejected:...etc..
My main concern is that the loading table is regularly loaded, with 3m records, has 90 columns, and to check by referencing each column's new value, in the table trigger, may mean very long load times.
Anyone got any how to ideas?
Many thanks
Regards
Carlos

Folks, thanks for your help.
Previously, I had briefly considered both ideas and they would work.
I dismissed the utl_file package solution because I believed that it would take too long to read in each log file line, search for the text and then inserts it into a table.
I overlooked testing this and when I did (1mb file) it actually was pretty fast. I will go that route.
Thanks for everyones help
regards
Carlos

SQL DEVELOPER 3EA1 -- Doesn't show my Code Template ?

Hi
I wander if someone could help to workout with Code template in SQL Developer 3.0EA1 (0.5.97)
I have many code template in my previous release 3.0 ( Tools --> Preferences --> Database --> SQL Editor Code Template ). When I was typing first word of Code Template, It was appearing but this is not the case in 3.0EA1 , It shows me default Code Template not the one I am looking for.
Thanks
Hemesh.

They have been removed:
http://vadimtropashko.wordpress.com/2011/10/11/documentation-code-snippets/
Here is how you get your programed code template back. In the worksheet invoke your code template the "official" way, that is via abbreviation. Run the statement. Now it's in SQL History and should be shown in code insight.

How to get audit id in coder template mapping

Hello everybody,
i am new in this Forum and new to the Oracle Warehouse Builder. I am Student and currently working on a approach of error handling by ETL processes.
To my question:
I am using a Code Template Mapping to load data in my target tables. In my target table i want to have a column which contains the Execution_Audit_Id of the current Code Template run. Can anyone tell me how i get this Audit ID in a Code Template Mapping? That would be great!
OWB Version 11.2
Regards,
Jens

Hi,
You would need to create a parameter in your report called userloginid select the paramter data type to match that of the datatype this value is stored as in the table.
then the correct syntax would be select * from table1 where userloginid=:userloginid
Hope this helps,
Domnic

Calling SQL*Loader from Forms

Hi,
I was wondering if anyone has called SQL*Loader from Forms?
What I am wanting to do is use Oracle Forms as the interface where you can specify a file that you can import into the database and it will use a set control file. Push the import button and SQL*Loader does the rest.
Is using Java code to call SQL*Loader from Forms a viable option, or is there an easier way to achieve the desired outcome.
Any ideas or guidance will be much appreciated.
Thanks,
Scott.

Scott,
In forms, there's a HOST built-in command which is supposed to execute any o/s commands.
What you have to do is :
1. Bult up the string exacltly in the fashion which you will run in o/s
2. Call the HOST Built-in and pass in the string
Here's a example :
Declare
lOsCmd Varchar2(1000) := Null;
Begin
lOsCmd := 'sqlldr user-id=userid/passwd@connectStr '
|| ' control=c:\temp\abc.ctl log=c:\temp\abc.log '
|| ' bad = c:\temp\abc.log';
Host (lOsCmd, No_Screen);
End;
-- Shailender Mehta --

Change the mapping generation code from sql*loader to pl/sql

I want to use a mapping with a flat file operator to generate pl/sql code even if a mapping generate sql*loader code as default.
I tried to change the Language generation property of the mapping but an API8548 error message is shown. The suggested solution by OWB is to change the language generation code in the property inspector of the mapping.
I can't use external table because I have to work with a remote machine.
What i have to do to change the generation code from SQL*Loader to PL/SQL?

How about breaking this out into 2 mappings? In the first mapping, map a flat file operator to an table using SQL*Loader code. Then define a second mapping using the table as source and therefore generate PL/SQL. Then use process flow to launch the 2nd map to run after completion of first.

Executing sqlldr (sql loader) from java and returning the error code

I'm wandering do sqlldr return any error code whenever it hit error while running in java.
For example, if i run in command prompt using the command below,
C:\ >sqlldr uid/pwd data=abc.dat control=abc.txt
It might give me some indicator that error occurs such as
SQL*Loader-601: For INSERT option, table must be empty. Error on table CURRENCY
or
SQL*Loader-500: Unable to open file (abc.txt)
SQL*Loader-553: file not found
SQL*Loader-509: System error: The system cannot find the file specified.
But when i run in java using the code below,
Runtime rt = Runtime.getRuntime();
Process proc = rt.exec("sqlldr uid/pwd data=abc.dat control=abc.txt");
int exitVal = proc.waitFor();
System.out.println("Process exitValue: " + exitVal);
it will only give me the same exitValue of 1(i presume its error) / 0 (i presume no error) instead of the details of the error.
How can i get the exact error code/message if i were to execute it using java?
Any solution?

mg,
I don't think user576271 wants the exit code, I think [s]he wants the error message.
But wouldn't error messages from SQL*Loader be sent to the stderr stream, and not the stdout stream?
In which case user576271 would need method "getErrorStream()" of class java.lang.Process, no?
Good Luck,
Avi.

Add code to pl/sql pages from web generator designer

Has anyone tryed to put code before pl/sql procedure that
manages web pages with designer ?
I'm more specific:
i want to put pl/sql code on top the procedure "Forminsert"
generated by designer; i know i can use text in the module but
generator transform text in htp.p('text');
Is there any solutions ?
thanx
Francesco Galante

You can put code in the Event PreInsert.
Example: Check web user with access control list.
Define procedure checkACL(p_userid, p_role).
Then in the PreInsert event put:
BEGIN
checkACL( owa_util.get_cgi_env('REMOTE_USER'),
'ADMIN');
END

Problem with SQL Editor Code Template

Hello,
I use Oracle SQL Developer 3.0.04 in a computer is Spanish language. When I insert a SQL Code Template with unicode chars (like ñ, á, etc.) and I go to the tab Preferences -> Database -> SQL Editor Code Template, I've lost all the templates.
I've been investigating about the problem and I've found that the templates have been stored at *%APPDATA%/SQL Developer/CodeTemplate.xml*. When I open that template, I can see that it is codified in ANSI instead of UTF-8. If I recodified the file to UTF8 manually, the SQL Code Template work perfectly.
To sum up, my problem is that everytime I add a code template the file CodeTemplate.xml was recodified to ANSI.
For example: if I add the next template AÑO="2011" lost all the templates the next time that go to the tab SQL Editor Code Template
Thanks in advance.
Lolo

Thanks, logged bug 12691441 for this.

SQL LOADER : RETURN CODE differs from 8i to 10.1.0.5

Customer is using 8.1.7.1.0 and same SQL LOADER script returns RETURN CODE: 2
However the same script returns RETURN CODE: 0 when using 10.1.0.5

You haven't specified so far the circumstances under which a possible error message is shown. The fact that you launch the same script against two different oracle versions doesn't mean the script will find the same environment, some errors could have been generated at 8i, meanwhile at 10g it could have found a clean environment. Could you please also compare the generated log file for both executions?
~ Madrid
http://hrivera99.blogspot.com/

Sample code of how to execute sql loader from visual basic

Hi,
Can anyone please provide me with sample code of how to
run sql loader from inside a visual basic 6.0 application with Oracle 8.1.7.3 database or please point me to documentation/tech books containing such examples?
Thanks very much
John

You have to use the command shell to activate a file.bat. The Shell command has the following parameters :
Shell(Pathname, Windows Style )
[email protected]
Joel P�rez

Code Templates/Completion Insight SQL Developer 3.2.2

Hello, today I got the version 3.2.2. During the first start all setting from my previous version were imported and all work perfect, except the completion insight.
If I write
ssf +[CTRL + Space]
the result in the previous version is
select * from
In the new version, for the same input is:
ssfselect * from
For other code template it is the same, the id for the code template will not be delete while setting the code template.
Is this a bug or is there somethin wrong with my enviroment?

It's a bug that's been fixed for version 4.0.

Code Template Design - SQL LOADER

Similar Messages

Maybe you are looking for