SQL and Spool File: low performance  (Oracle 8)

Hi,
I've got some difficulties with the spool function: it works very slowly when I have to write on a file.
In details, a shell script starts the PL/SQL procedure, which starts the spool function that produces a file.dat .
Oracle solves the query and sends out the data immediately. This with 850 million of records for a 20gb final file. The total time to write this file is 8 hours.
May it be a solutions with could to improve this file writing procedure, trying not to use the spool function, like a C program which could work on the memory? In this way it could take fewer time.
Thank you for your suggestions and solutions.
A

Oracle RDBMS does not write to the spool file. The client does (SQL*Plus in this case).
SQL*Plus uses standard kernel calls.
To open/create the file and write data to it:
- open call with flags O_WRONLY|O_CREAT|O_TRUNC (see open (C System Call) - Code Wiki)
- write call to file descriptor (see write (C System Call) - Code Wiki)
These are standard calls supported by all modern kernels (Unix, Linux, Windows, OS/X, etc). The performance of which is entirely dependant on how fast the kernel can service the call. Which in turn is dependant on the speed of the file system on which these calls operate.
Speeding up a spool type process typically means using async I/O (called overlap I/O in the Windows kernel). The approach being to fetch the next set of data from the database cursor, while writing the previous fetch's data to spool file, at the same time.
Of course, better and faster hardware can also improve the performance of the I/O subsystem and make such a spool process faster.

Similar Messages

  • Using archivelog and control file from other Oracle server

    I am still bothered with my backup process.
    I have 2 AIX boxes (same model, say A and B); both have BAAN 5 and Oracle 10g R2 on. Right now my colleague insists to use the export pump (cold backup) from Prod Oracle server (A) to restore the Oracle server on Box B. The Prod server has the archivelog mode turn on. But it will miss any transaction data from import pump till the crash point of Box A. So this is my confusion.
    Can I pass the control files and archivelog files form Box A (prod server) to Box B and use them to restore the Box B as the latest Prod server? How?
    I tried to convince them to use the RMAN backup? But not successful?
    I think the best way is probably to use the Oracle Data guard. However, there is always one concern to my manager and colleague, that such process will cause the data on the restored server (failover, Box B) not recognizable by the BAAN, which define the objects (tables).
    Thanks

    Performing a logical backup is not useful to restore to the point of failure. The only valid and available option is a hot backup/archivelog mode. Your recovery manager backup perform a controlfile and redologfile backup, so those can be restored at the destination. You must take care of the way you perform the backup, and ensure the paths where your backup is being deposited are visible by the second node. A shared storage with same mount points is suitable in this case. A tape robot configured at both nodes is also a suitable solution.
    Recovery manager perfoms a controlfile and spfile restore, too. This rman command perform the action:
    SET DBID <DBID of the database,for which you want to restore the controlfile>;
    RESTORE CONTROLFILE FROM <name_of_backupiece_which_contains_the_controlfile backup>';
    I don't see any problem on the recovery manager side, and technically speaking, on the Oracle side it is perfectly possible to restore your database at a remote location. I don't know what happens on the BAAN side, if you are required to have it configured to be operative on the target node. You could try to clone your database at the node B, configure BAAN and prepare the proceduere in case of failure.
    Configuring a dataguard is also a recomended action. So it is to think about Cold Failed Over clusters. I have recently performed a CFC configuration with BAAN. No problem it works smoothly.
    ~ Madrid.

  • SQL and database file

    Hello.(sorry for my english)
    I have a legacy system , ported to opencobol that uses Berkeley DB. I can successfully open the database files using java both in windows and linux. However i want to create a web front end for the application.
    Data is stored using strange cobol structures (most numbers are stored as text or 4 bits per digit) and other fancy stuff.. so custom parsers should be written. (dont rember BDB classes name for that)
    I have been reading the documentation of Berkeley DB 12 hours totaly today but couldnt find answers to some questions...
    Is it possible to attach the database to dbsql.exe (sqlite) server ?? I tried to do "attach "..pathtofile" as NULL ,but i get multiple databases specified and not supported and other stupid messages....
    I think that what i tried propably doesnt make sense as collumns arent specified and nothing is in order this thing to work ..It is a simple Key value database.
    So i am a little stucked here.
    The only solution i can think is to synchronize with a relational Database. But i dont have triggers or nothing that helps....
    Note that i want one way synchronization. The "other way" will have very limited tasks.
    It seems to me that checking every x minutes if something changed using a cursor is a demanding task. So i am wondering.. Is there a way to track changes???
    Thank you :)
    Edited by: 843912 on 12 Μαρ 2011 7:13 μμ

    Hello,
    I am not sure I completely understand your question. If you are asking about importing
    and exporting data from a Berkeley DB database into an Oracle DB Table in Berkeley DB
    releases prior to 5.*, then this can be done with the Oracle OCI interface. If I am
    misunderstanding the question, please let me know.
    Thanks,
    Sandra

  • Parse JSON records using PL/SQL and store the output in Oracle Database.

    We have JSON data/record stored in a Oracle table as CLOB(database version 10g). We need to read/parse this JSON data and recognize the relevant tags and corresponding values in this JSON data and then insert these id/values into a separate Oracle staging table.
    Please let me know if there is any re-usable code available for this.
    We are looking for solution that does this manipulation in pl/sql. If this is not feasible, even Java code approach is fine.
    Regards,
    Kiranmayi.

    Hello,
    PL/JSON | Free software downloads at SourceForge.net
    ERASME - pl/sql library for JSON
    Regards
    Marcus

  • SQL and Excel File

    HII,
    I am trying to develop an SQL query that will produce an Excel File that can be posted on a WEB site for x amount of transaction.
    I would like to know which ways are there in which I can do this tast.
    If anyone has a suggestion that would be greatly appreciated.
    Thanks
    Pankti

    Hii Justin,
    I was just reading up on UTL_File package althought I am still unable to conclude as to how am I to go about using this method. I hope you don't mind I am a bit new to this enviorment so I have lots of question. Let me explain in detail: I have columns x, y, and z in a perticular table. I want to develop an SQL query which will give data in Excel file.
    How do I use the UTL_File method to acchive this task??

  • Catproc.sql and catalog.sql files

    I am using oracle 10g recently i have runned two files catproc.sql and catalog files by connecting to sys
    after running those files in sys when i select
    select * from tab;
    no rows were returned
    1) these files will show any eeect on data base
    2)From toad when i connect to data base in toad it is showing all the tables in sys user
    is there any effect of running those files

    933900 wrote:
    I am using oracle 10g recently i have runned two files catproc.sql and catalog files by connecting to sys
    after running those files in sys when i select
    select * from tab;
    no rows were returned
    1) these files will show any eeect on data base
    2)From toad when i connect to data base in toad it is showing all the tables in sys user
    is there any effect of running those filesHave you created database newly? so you ran catalog.sql & catproc.sql?
    Of course you can able to see from TOAD but not from SQL, its weird.
    Can you post
    SQL> select * from dba_registry;
    Also any errors from alert log file, when you ran These scripts? if so post here.

  • Delete audit and trace files

    Is anyone aware of why we should not delete audit and trace files under the oracle admin directory that are over 30 days old? I don't know that there is anything in place to do this and they are building up. Tuldcorpadb01:/oracle is at 96%. These files aren't that big, but there are a lot of them. See counts below.
    oracle:tuldcorpadb01:ecmd> cd admin
    oracle:tuldcorpadb01:ecmd>find . -name \*.aud -type f -ctime +30 -ls|wc -l
    25149
    oracle:tuldcorpadb01:ecmd>find . -name \*.trc -type f -ctime +30 -ls|wc -
    2426

    Move older audit files to a different filesystem until you find someone being able to define what files can be deleted.
    Regards
    Gustavo Restuccia

  • Output to spool file only

    Hi,
    I don't want the output of the spool file to be written to a log file while executing the script. My script looks like this,
    #/bin/ksh
    sqlplus test/test 1>>/test/abc.log 2>>/test/abc.log <<EOF
    set termout off
    set feedback off
    set flush off
    spool /test/test.dat
    select 'a' || 'b'|| 'c' from dual;
    spool off
    disconnect;
    exit;
    EOF!
    I want the output to be written only into the spooled file. now it writes to both the log file and spool file. how can i avoid it from writting to the log file(1>>/test/abc.log). pl help

    Actually. Why do you want to write to both anyway?
    You are using the general unix format to write to a file so why do you need another file?

  • How do you extract SQL from Oracle Forms and Reports files?

    I am developing an "as is" data model for a government client for a 14 year old system that has three databases, 20 schemas, over 1500 tables, and over 23,000 columns. Needless to say, I do not plan to perform a manual mapping of data to screens and reports.
    Most of the system has been developed in Oracle Forms and Reports. I am trying to map the live tables and columns to forms and reports.
    The process here has been to save the forms and reports files as .fmb and .rdf files. The client does not have an available copy of Oracle Designer, which I understand could be used to extract the SQL.
    Is there a utility somewhere that can parse the .fmb and .rdf files to extract the SQL?
    Thanks,
    Jim Gearing

    Jim,
    I don't know of any utility that will do this. You can convert and save each fmb as a fmt so you can view/search the contents , but I don't recommend that approach.
    On the other hand, you download a copy of Oracle Designer:
    http://www.oracle.com/technology/software/index.html
    It usually is included with Oracle Forms and Reports.

  • C# application to execute both oracle and sql server script files.

    Hi All,
    I am suppose to develop an application using c# which can execute oracle script files as well as sql server script files based on some user's choice. 
    Both the script files(oracle/sql server) contains scripts to create tables, create views and create functions.
    What is the best way to do this and which existing c# functions should is use.
    Thanks in advance.

    I am suppose to develop an application using c# which can execute oracle script files as well as sql server script files based on some user's choice. 
    You can use SQL Plus for Oracle. You can have a C# program that issues commands  for  scripts for table, views, triggers  creation and the whole 10 yards to SQL Plus,
    http://docs.oracle.com/cd/B19306_01/server.102/b14357/qstart.htm
    https://www.google.com/#q=sql+plus
    I started writing a program to use SQL Plus to run the scripts, but cI ame to the conclusion that it was a waste of my time.
    A tool called OSM, an Oracle, tool gives a list of script files that need to be run do to a change in a script file based on file timestamp. You can copy the list doing a copy/past operation and drop it into SQL Plus's Command line, and SQL Plus runs the
    scripts.
    For MS SQL Server, you have this, which you can run from a C# program too.
    http://www.webfactory-world.de/wfknowledgebase/wfknowledgebase_wf3.3/Content/AdditionalWindowsSettings/Tutorials/UpdatingSQLDBFromCommandLine.htm
    I would consider using the above from a C# program a waste of time too.
    I look at it this way. If the user doesn't have database admin experience and know how to use DB admin tools, then they have no business doing anything with a database - period.
    BTW, the Invoke statement in VB or C# will run SQL Plus or SQL Server command line tool.

  • Extraction of oracle pkg and pkg bodies into spool file??

    Hi,
    I need to extract all packages and its bodies into spool file. can you give me script or guide how to do this.. i have more than 200 packages.. need to create spool file for each package. great thanks for help.
    /mr
    Edit/Delete Message

    From 10g, DBMS_METADATA is designed for ! :)
    [http://download.oracle.com/docs/cd/B19306_01/appdev.102/b14258/d_metada.htm#i1019414]
    Edited by: Leo Anderson on 1 sept. 2008 14:22

  • Date and time in spool file name

    Hi,
    can anyone show me how i can write the code below so that the spool file will automatically take the system time and date?
    thanks again.
    set term off;
    set echo off;
    set heading off;
    set linesize 1500;
    set pagesize 9999;
    set feedback off;
    spool /home/oracle/ATM_Upload/files/CRD000119MMDDYYHHMISS.DAT;
    SELECT rec_ind||source_type||bbdsa_code||seq_num||dt_ti_ext||ver_num
    as atm_header
    FROM atm_dc_hd;
    spool off;

    Hi,
    I do something similar to this in a file called ed.sql which I use in SQL*Plus to give me a history of SQL*Plus buffers in separate files which follow the naming convention $DATABASE_$SID_$TIMESTAMP.sql. I use it from SQL*Plus when editing, rather than typing "ed" I type "@ed" and it does the naming for me:
    -- Turn off terminal output
    set termout off
    -- Save current buffer to a temp file
    save tmp.txt replace
    -- Set editfile name to $DATABASE_$SID_$TIMESTAMP.sql
    column fname new_value fname
    column sid new_value sid
    select trim(sid) sid
    from v$mystat
    where rownum = 1;
    select global_name||'_&sid'||'_'||to_char(sysdate, 'YYYYMMDDHH24MISS')||'.sql' fname
    from global_name;
    set editfile '&fname'
    -- Retrieve our initial buffer
    get tmp.txt
    -- Turn terminal output back on
    set termout on
    -- Fire up the editor
    edYou could probably adapt this in the following way for your needs:
    column fname new_value fname
    select 'CRD000119'||to_char(sysdate, 'MMDDYYHH24MISS')||'.dat' fname
    from dual;
    spool '&fname'I don't know how you get the 'CRD' part of the filename but you may be able to generate that too using the above as a guide.
    cheers,
    Anthony

  • Hello, How do I tell sql+ to spool output file from windows to Unix server?

    Hello, How do I tell sql+ to spool output file from windows to Unix server?
    I am new to SQL+ and just learned how to spool the file. But file is saved in my local windows enviroment and since it's 2GB in size...I want to spool it directly to another remote unix server.
    Pls answer in detail... I have been to most of the thread and didn't see relevant answer to above question.
    Am I suppose to develope some script which FTP the spool file directly to the server I want to
    or
    i Have to use UTL_FILE Package ?
    Thanks for reply

    You may not be able to...
    SQL*Plus can only spool to the local machine. If you have mapped a directory on the Unix server from your Windows machine, you can specify that directory in your SPOOL command.
    You could telnet to the Unix server, run SQL*Plus there, and spool the file to a local (Unix) directory.
    If the Unix server is also the Oracle database server, you could use the UTL_FILE package rather than using SQL*Plus to spool a file.
    If the Unix server is also an FTP server, you could also FTP the file from your local machine to the server.
    Of course, I would tend to re-examine a requirement to regularly generate a 2 GB text file. It seems likely that there is a better way...
    Justin

  • Seek help to format spool file from SQL*PLUS

    I am running a Unix shell script to call a Oracle 11g SQL script from a Oracle database. In this SQL script, I need to connect to many different remote databases to select data, then sool these records as one big text file to a directory. Then email the file to related Group users. In the spool file, there is a line on the top of each page like this:
    DUMMY
    DB_NAME
    I know this is caused by connect to remote database in SQL*PLUS. My connection string is like this:
    Conn system/password@Oracle_SID
    How can I remove these lines or how to skip these lines into spool file? Please advise. Thanks in advnce. Wish all of you Happy New Year!!!

    Hi,
    It sounds like you have some kind of formatting (such as SQL*Plus TTITLE) producing the output you don't want. If that's the case, temporarily stopping the spooling might not help you. Find out what is causing the output that you don't want. You say that you know it is caused by the CONNECT statements, but it must be more than that. I've written scripts with CONNECT statements that don't have anything like what you reported at the top of each page; in fact, they don't even have pages: the output is one continuous stream. Find out what's putting the unwanted output there, and that will be a big clue as to how you can stop it.
    You say that you know the unwanted titles are there because of the CONNECT statements. If so, use database links instead of CONNECT. You don't have to use dbms_scheduler or utl_file; just eliminate the CONNECT statements. (I'm not saying that there's anything wrong with dbms_scheduler or utl_file; you should definitely investigate those tools. I'm just saying that using database links is independent of them.)
    What would happen if you did all your connecting to different databases at the OS level? Can you write a shell script that connects to each database in turn, and runs a SQL*Plus script in each one. Each SQL*Plus script would have a SPOOL or SPOOL ... APPEND command, or maybe you could build the SPOOL into a LOGIN.SQL script.

  • Spool file contains spool off; statement and a / in it .how to remove these

    Hi ,
    I am new to using forums.
    I am a basic beginner user of oracle .
    I need to spool the output of 2 select queries into a file.
    I could achieve it.
    But the file also contains ,
    SPOOL OFF;
    in it . How to avoid these in the file.
    Any help is appreciated .
    thanks

    user11768287 wrote:
    Hi
    I checkout and created a .sql file .
    But the spool file , contains an empty line after every record is printed. These are the heading i am using.
    SET HEADING OFF
    SET ECHO OFF
    SET TERMOUT OFF
    SET VERIFY OFF
    SET FEEDBACK OFF
    The spool file starts with a blank line followed by the record and then blank line after every record .
    What to do to get all records one after the other without blank lines in between the records in the spool file.
    In addition to Toni's mention of PAGESIZE and TRIMSPOOL, look at LINESIZE. Also, what are you using to view the spooled file? Are you sure you aren't just seeing line wrap within that application - like Notepad?
    Thanks ...

Maybe you are looking for