Running a SQL Loaded Job from Schedular

How Can I Run a SQL Loader Job from Schedular ,
Message was edited by:
jus

The trick is to create a wrapper script that is referred to by the PROGRAM object with arguments. The wrapper should do the work.
/usr/local/bin/do_sqlldr.sh:
#!/usr/bin/env ksh
# setup actions
sqlldr control=...
Don't forget to check who is the owner of $ORACLE_HOME/bin/extjob and that the owner is available.
My question is how this can be fitted in an DTAP environment where Developement and Test environment are on the same server and have the same job definitions. Development and Test should use different datafile and logfile locations.
regards,
Ronald
http://ronr.nl/unix-dba

Similar Messages

How Can I Run a SQL Loader Job from Schedular

How Can I Run a SQL Loader Job from Schedular , So that It Runs every Day.

Depends on a couple of factors.
If you are on a UNIX platform, you can create a shell script and schedule it with cron.
If you are on a Windows platform, you can create a batch file and schedule it with the Windows scheduler.
Or, if you are on Oracle 9i or 10g, you could use the external table feature instead of SQL*Loader. Then you could write a stored procedure to process the external table and schedule it using the Oracle scheduler (DBMS_JOB). This would probably be my preference.

Run multiple SQL*Loader control files from command prompt

I have written control files for 15 different tables with specific data files for each table. I have build the control files using toad for each one of them at once. Now I want to run the sql loader for all the tables one time. I know I cannot run multiple files from toad and not sure If I can run from the command prompt.
Please let me know If I can.

skvaish1 wrote:
Hi,
On unix you can create a script to runn all sqlldr at once like as followes.
1. Create a script load_sqlldr.sh with following content.
export ORACLE_SID=<YOUR DB_SID>
export ORACLE_HOME=<Your Oracle Home>
export PATH=$ORACLE_HOME/bin:$PATH
sqlldr (Full command with parameter for 1st table) &
sqlldr (Full command with parameter for 2nd table) &
sqlldr (Full command with parameter for 3rd table) &
sqlldr (Full command with parameter for 4th table) &
sqlldr (Full command with parameter for 15th table) &
2. Then execute the script.
Regards1. Can I also write script load_sqlldr.sh with following content
sqlldr user/psswd@db_name control=controlfile1.ctl
sqlldr user/psswd@db_name control=controlfile2.ctl
sqlldr user/psswd@db_name control=controlfile3.ctl
sqlldr user/psswd@db_name control=controlfile15.ctl
2. Then execute the script.

Run SQL Loader script from Unix env

Hi,
We are using HP-UNIX server. The control file of SQL Loader is placed in oracle home\bin folder in the unix server. I created Executable for this SQL Loader control file in concurrent program and attached into a request group.
Then I ran it from a responsibility, I am getting the below error.
+-----------------------------
| Starting concurrent program execution...
+-----------------------------
SQL*Loader-350: Syntax error at line 16.
Expecting "," or ")", found "TIMESTAMP".
GL_DATE TIMESTAMP 'yyyy MM dd hh:MI:SS:AM',
^
SQL*Loader: Release 8.0.6.3.0 - Production on Sun Sep 5 15:28:45 2010
(c) Copyright 1999 Oracle Corporation. All rights reserved.
SQL*Loader-350: Syntax error at line 16.
Expecting "," or ")", found "TIMESTAMP".
GL_DATE TIMESTAMP 'yyyy MM dd hh:MI:SS:AM',
^
My Actual Control file is,
LOAD DATA
INFILE '/usr/........xyz.txt'
BADFILE '/usr/........xyz.bad'
DISCARDFILE '/usr/........xyz.dsc
APPEND
INTO TABLE gl_interface_stg
FIELDS TERMINATED BY "|" TRAILING NULLCOLS
GL_DATE TIMESTAMP 'yyyy MM dd hh:MI:SS:AM',
CATEGORY_NAME,
JOURNAL_DESCRIPTION,
JOURNAL_NAME,
BATCH_NAME,
BATCH_DESCRIPTION,
ACCOUNT_CODE,
PROCESSING_STATUS CONSTANT 'N'
If I run the above control file from my local machine (windows) using SQLLDR username/pwd @servername control='c:/abc.ctl' from the command prompt, then all the records from xyz.txt file pushed into staging table.
Where as, If I schedule the above control file and trying to run from a GL responsibility, I am getting the above error message.
Do I need to modify the control file scripts in order to run from UNIX environment?
Pls advice how to do it from UNIX environment.
Thanks in advance.

Hi,
SQL*Loader-350: Syntax error at line 16.Please see if these docs help.
SQL*LOADER-350 SYNTAX ERROR [ID 1019271.102]
LDR-00350 TOKEN LONGER THAN MAX ALLOWABLE LENGTH ERROR WHEN LOADING ASCII FILE [ID 1020091.6]
Do I need to modify the control file scripts in order to run from UNIX environment?
Pls advice how to do it from UNIX environment.See these docs.
How to Use 9i or 10g Features in SQL*Loader for Apps? [ID 423035.1]
11i FND:How to specify Record Terminator In Sql*Loader type of concurrent program [ID 252850.1]
Thanks,
Hussein

SQL*Loader job exits unexpectedly and causes table to be locked with NOWAIT

I have a weekly report job that I run where I have to load about 48 logs with about 750k rows of data in each log. To facilitate this, we have been using a Java job that runs SQL*Loader as an external Process (using ProcessBuilder), one after the other. Recently however, this process has been terminating abnormally during the load which is causing a lock on the table and basically causes the process to grind to a halt until we can open a ticket with the DB team to kill the session that is hung. Is there maybe a better way to handle this upload process than using SQL*Loader or is there some change I could make in either the control file or command line to stop it from dying a horrible death?
At the start of the process, I truncate the table that I'm loading to and then run this command line with the following control file:
COMMAND LINE:
C:\Oracle\ora92\BIN\SQLLDR.EXE userid=ID/PASS@DB_ID load=10000000 rows=100000 DIRECT=TRUE SKIP_INDEX_MAINTENANCE=TRUE control=ControlFile.ctl data=logfile.log
CONTROL FILE:
UNRECOVERABLE
Load DATA
INFILE *
Append
PRESERVE BLANKS
INTO TABLE MY_REPORT_TABLE
FIELDS TERMINATED BY ","
     filler_field1          FILLER char(16),
     filler_field2          FILLER char(16),
     time               TIMESTAMP 'MMDDYYYY-HH24MISSFF3' ENCLOSED BY '"',
     partne          ENCLOSED BY '"',
     trans               ENCLOSED BY '"',
     vendor          ENCLOSED BY '"' "SUBSTR(:vendor, 1, 1)",
     filler_field4          FILLER ENCLOSED BY '"',
     cache_hit_count,
     cache_get_count,
     wiz_trans_count,
     wiz_req_size,
     wiz_res_size,
     wiz_trans_time,
     dc_trans_time,
     hostname          ENCLOSED BY '"',
     trans_list          CHAR(2048) ENCLOSED BY '"' "SUBSTR(:trans_list, 1, 256)",
     timeouts,
     success ENCLOSED BY '"'
Once all of the logs have finished loading, I rebuild the indexes on the table and then start the report process. It seems like it's just dying on random logs now, re-running the process it will fail at a different point each time.
EDIT: The reasons for the UNRECOVERABLE and SKIP_INDEX_MAINTENANCE are to speed the load up. As it is, it still can take 7-12 minutes for each log to load, it's even worse without those on. Overall it's taking about 18 hours for this process to run from start to finish.
Edited by: user6676140 on Jul 7, 2011 11:37 AM

Please note that my post stated: "I have opened a ticket with Oracle support. after 6 days have not had the help that I need."
I also agree that applying the latest PSU is a Best Practice, which Oracle defines as "a cumulative collection of high impact, low risk, and proven fixes for a specific product or component".
With that statement I feel there should not be the drastic issues that we have seen. Our policy is to always apply PSUs, no matter what the product or component, without issue.
Except for now. We did our research, and only open an Oracle ticket when we need expert help. That has not been forthcoming from them, but we are still working the ticket.
Hence, I opened this forum because many times I have found help here, where others have faced the same issue and now have an insight. When having a serious problem I like to use all of my resources, this forum being one of those.
To restate the question:
(1)     97% of our databases reside on RAC. From the Search List for Databases, we do not see the columns Sessions:CPU, Sessions: I/O, Sessions: Other, Instance CPU%, and are told this is working as designed because you must monitor the instance, not the database, with RAC.
     (a) After applying PSU2 the Oracle Load Map no longer showed any databases.
All of this in (1) is making the tool less useful for monitoring at the database level, which we do most of the time.
(2) Within a few days of applying PSU2, we couldn't log into EM and got the error "Authentication failed. If problem persists, contact your system administrator."
    (b) searching through the emoms.trc files we found the errors listed above posting frantically.
After rolling back PSU we are back in business.
However, there is still the need to remain current with the components of EM.
I am looking for suggestion, insights, experience. While I appreciate Akanksha answering so quickly, a recommendation to open an SR is not what I need.
Sherrie

SSIS Package will only run as SQL Agent Job when I have remote desktop to server open.

Hey guys, so I have another problem to add to the already massive 'SSIS/SQL Server Agent Job' pile. After days of searching, I can't seem to find anything specific to my problem though.
The setup is as follows: a SSIS package that refreshes and saves excel files that are hosted on a server. The package runs fine on the local machine, using BIDS on the server, and will even work as a SQL Agent Job on the server IF there is a remote
desktop connection to the server. To elaborate, if I simply run the job as you would normally do it will fail and give the below error. If I run the job while either myself, or a different machine, has a remote desktop connection to the server where the
job is scheduled - it will run successfully.
Below is the error from the History File of the job. Any help would be greatly appreciated.
Source: Refresh Excel and Save      Description: System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.Runtime.InteropServices.COMException (0x8000401A): Retrieving
the COM class factory for component with CLSID {00024500-0000-0000-C000-000000000046} failed due to the following error: 8000401a.     at ST_79772452677f4de1852d5ffbba3e5232.csproj.ScriptMain.ExcelRefresh(String FileName)
at ST_79772452677f4de1852d5ffbba3e5232.csproj.ScriptMain.Main()     --- End of inner exception stack trace ---     at System.RuntimeMethodHandle._InvokeMethodFast(Object target, Object[] arguments, SignatureStruct&
sig, MethodAttributes methodAttributes, RuntimeTypeHandle typeOwner)     at System.RuntimeMethodHandle.InvokeMethodFast(Object target, Object[] arguments, Signature sig, MethodAttributes methodAttributes, RuntimeTypeHandle typeOwner)
at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture, Boolean skipVisibilityChecks)     at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags
invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)     at System.RuntimeType.InvokeMember(String name, BindingFlags bindingFlags, Binder binder, Object target, Object[] providedArgs, ParameterModifier[] modifiers, CultureInfo
culture, String[] namedParams)     at System.Type.InvokeMember(String name, BindingFlags invokeAttr, Binder binder, Object target, Object[] args, CultureInfo culture)     at Microsoft.SqlServer.Dts.Tasks.ScriptTask.VSTATaskScriptingEngine.ExecuteScript()
UPDATE:
In my previous post the Identity in DCOM Config file for Microsoft Excel was set as The Interactive User. The job was working only when I had a remote connection to the server open.
If I set the Identity to 'This User' and use the username and password of the server login account, it will work as a scheduled job without needing an open remote connection to the server. So it works, great! but I have reservations setting this
for all instances of Excel for the server. I'm sure other users have different accounts they use for running Excel. Any suggestions around this?

Hi LiamSexton,
It should be the server-side Automation of Office issue described in the following KB article:
http://support.microsoft.com/kb/257757
Microsoft does not currently recommend, and does not support, Automation of Microsoft Office applications from any unattended, non-interactive client application or component (including ASP, ASP.NET, DCOM, and NT Services), because Office may exhibit
unstable behavior and/or deadlock when Office is run in this environment.
User Identity: Office applications assume a user identity when the applications are run, even when Automation starts the applications. The applications try to initialize toolbars, menus, options, printers, and some add-ins based on settings in the user
registry hive for the user who launches the application. Many services run under accounts that have no user profiles (such as the SYSTEM account or the IWAM_[servername] accounts). Therefore, Office may not initialize correctly on startup. In this situation,
Office returns an error on the CreateObject function or the CoCreateInstance function. Even if the Office application can be started, other functions may not work correctly if no user profile exists.
To work around the issue, you can refer to the following alternative introduced in the article:
Most server-side Automation tasks involve document creation or editing. Office 2007 supports new Open XML file formats that let developers create, edit, read, and transform file content on the server side. These file formats use the System.IO.Package.IO
namespace in the Microsoft .NET 3.x Framework to edit Office files without using the Office client applications themselves. This is the recommended and supported method for handling changes to Office files from a service.
Regards,
Mike Yin
TechNet Community Support

Running sqlldr (sql*loader) as an concurrent program executeable in 11.5.10

Running 11.5.10.2 on Linux
I have a .ctl file written for my linux environment but I am having trouble with the following.
Properly configuring the concurrent program executable. Is there documentation on this setup? I have looked through OTN, MetaLink and the database utils guide, but no luck.
How do you pass the variable of the control file to sqlldr using the executable/concurrent program approach?
How do you pass the variable of the username/password and database to sqlldr using the executable/concurrent program approach?
my sqlldr script is listed below. All other activity takes place in the .ctl file and this is working good.
sqlldr apps/password@dev control='/sea/apps/dev/ora/8.0.6/rates.ctl'

Please see these docs.
11i FND:How to specify Record Terminator In Sql*Loader type of concurrent program [ID 252850.1]
How to Register a Host Concurrent Program in Applications [ID 156636.1]
How To Create A Custom Concurrent Program With Host Method and Pass Parameters To The Shell Script [ID 266268.1
How to Use 9i or 10g Features in SQL*Loader for Apps? [ID 423035.1]
Is there a Method for Returning a 'Warning' Status from Host Language Concurrent Program? [ID 866194.1]
Use Encrypt To Prevent Apps Pwd Being Displayed In Log/Sql Script [ID 377858.1]
Thanks,
Hussein

Sql*loader called from stored procedure

Does anyone know how to invoke sql*loader
(or any other command line type utility) from within a stored procedure?

<BLOCKQUOTE>quote:<HR>Originally posted by CHARLES SAUER ([email protected]):
Does anyone know how to invoke sql*loader
(or any other command line type utility) from within a stored procedure? <HR></BLOCKQUOTE>
Hi,
I did something like that on 8i with java.
Wrote a simple java proc that runs unix commands, which gets called by a plsql proc.
If you want details, please post back.
Cheers,
Paulo
null

Error with SSIS package running as SQL Agent job

I have a strange issue.
I have a SQL Agent job that execute 5 steps, each step is an SSIS package that imports a CSV file to a temp table, then executes a SQL script to update a production table.
This job runs every 3 mins, every day, and has been for over a year.
Every once in a while it will start failing on step 2. Sometime it only happens for a couple of runs, other times I have to restart the SQL server. Once it corrects itself, it could be months before it happens again, but usually happens every
few weeks.
The error in the job history is: "Unable to bulk copy data. You may need to run this package as an administrator"
Obviously I have all the permissions correct, as it was running successfully every 3 mins before this error suddenly starts. I'm at a loss as to what the problem is. It is always on step 2, which is identical to step 1 except its a different CSV file
going to a different table. As I typed this, it failed twice, then was successful afterwards.
Any ideas where to look for further information? I've checked the event viewer and looked in the SQL logs, nothing stands out at this time.

Hi Kerberos35,
It seems to be caused by the UAC feature which makes SSIS use the low-permissions token of the administrator Windows account under which the job runs. This issue is described in the following KB article:
http://support.microsoft.com/kb/2216489/en-us
To resolve the issue, you can install the latest service pack for your SQL Server. To work around this issue, you can also use one of the following two methods:
Method 1: Replace the SQL Server Destination components in the Data Flow Tasks that are failing with OLE DB Destination components that point to the same SQL Server connection manager.
Method 2: Create a SQL Server Agent proxy by using a Windows account that is not a member of the local Administrators group after you assign Create Global Objects permissions to that account.
To do this, follow these steps:
Click Start, point to Administrative Tools, and then click Local Security Policy.
Expand Local Policies, and then click User Rights Assignment.
In the details pane, double-click Create global objects.
In the Local Security Policy Setting dialog box, click Add.
In the Select Users or Group dialog box, click the user accounts that you want to add, click Add, and then click OK two times.
Regards,
Mike Yin
TechNet Community Support

Automate a sql loader job

Hi,
I need to automate schedule job to import excel files into Oracle tables using SQL Loader.
Can any one please tell me how to do this?
thanks
Jenny

Depends on a couple of factors.
If you are on a UNIX platform, you can create a shell script and schedule it with cron.
If you are on a Windows platform, you can create a batch file and schedule it with the Windows scheduler.
Or, if you are on Oracle 9i or 10g, you could use the external table feature instead of SQL*Loader. Then you could write a stored procedure to process the external table and schedule it using the Oracle scheduler (DBMS_JOB). This would probably be my preference.

Running a SQL*Plus command from Java

I have a requirement to run SQL* Plus commands from Java. The results as they would appear in SQL*Plus window or spool file should be captured and stored in a table.
For example,
SQL> select * from dual;
D
X
1 row selected;
SQL>
All the lines above should be stored for later use.
Could any one give pointers?
Thanks,
Ravi


Hi,



public class RuntimeExecApp {
public static void main(String args[]) throws IOException
Runtime r = Runtime.getRuntime();
r.exec("C:\\Oracle\\sqlplus.exe");
}


Kuba

Message was edited by:
KUBA

Error when running package to load data from InfoProvider

I encounter the following error when using the standard package to load data from a InfoProvider to BPC. I tried loading the data from both DSO and InfoCube, but I get the same error.
The log message where the error occurs is as follows:
Task name CLEAR CUBE DATA:
Could not perform write (INHERITED ERROR)
The "convert" step of the process chain completed successfully. This error is happening when the "clear cube according data" step of the process chain gets executed.
Any ideas on why this is happening?
Thanks.

Hi Laeral:
You should be able to just open one VISA session then write and read both commands then close the session at the end. This sounds to me like an error that comes from trying to open two VISA sessions to the instrument at the same time. I have attached some very basic LabVIEW VISA code that writes and reads two different commands. Hopefully it will get you started.
Regards,
Emilie S.
Applications Engineer
National Instruments
Attachments:
Basic VISA Example Two Commands.vi ‏39 KB

Does SQL Developer have any way to run the SQL*PLUS COPY FROM command?

set arraysize 100;
set copycommit 100;
COPY FROM SCOTT/TIGER@OtherDBinstance -
CREATE COMPANY -
USING SELECT * from COMPANY;

Thanks. I have been using the create table ABC as select * from ABC@otherdb but this often runs into temp segment issues for very large tables. The SQL*PLUS version allows me to have the copy commit save after each xxxxx lines and this seems to be able to handle any size table. Can copy commit be used from SQL Developer?

Running a SQL Stored Procedure from Power Query with Dynamic Parameters

Hi,
I want to execute a stored procedure from Power Query with dynamic parameters.
In normal process, query will look like below in Power Query. Here the value 'Dileep' is passed as a parameter value to SP.
    Source = Sql.Database("ABC-PC", "SAMPLEDB", [Query="EXEC DBO.spGetData 'Dileep'"]
Now I want to pass the value dynamically taking from excel sheet. I can get the required excel cell value in a variable but unable to pass it to query.
    Name_Parameter = Excel.CurrentWorkbook(){[Name="Table3"]}[Content],
    Name_Value = Name_Parameter{0}[Value],
I have tried like below but it is not working.
Source = Sql.Database("ABC-PC", "SAMPLEDB", [Query="EXEC DBO.spGetData Name_Value"]
Can anyone please help me with this issue.
Thanks
Dileep

Hi,
I got it. Below is the correct syntax.
Source = Sql.Database("ABC-PC", "SAMPLEDB", [Query="EXEC DBO.spGetData '" & Name_Value & "'"]
Thanks
Dileep

How to schedule a load job from bi to r3 daily ? What are the available opt

I think we can create a PC for the same so should we code an ABAP routine with include to do the process. So can it be scheduled daily, what are the other option in BI7 other than PC ?

Hi Manohar,
To make the data source Real-Time enabled you need to run a small program which updates the REALTIME flag in ROOSOURCE table. Program is as below.
report z_rda_flag.
tables : roosource.
parameters : p_oltpsr like roosource-oltpsource.
update roosource set realtime = 'X'
where oltpsource = p_oltpsr
and objvers = 'A'.
Enter the data source name in the selection screen and run the program.
Now go back to SE16 and see the contents of 0FI_GL_4 inROOSOURCE.
You can see that the program has updated REALTIME flag for the datasource 0FI_GL_4.
hope this answer solves ur issue
u can do this only for an ODs and not for a cube. will let u know if there are any other options for a cube.
assign values to the answers as its the way of thanking in SDN.
regards,
Ranjith

Running a SQL Loaded Job from Schedular

Similar Messages

Maybe you are looking for