How to load huge size file in database

Question

Hi,
I have a requirement to load files into database and download them as well. Can you tell me which is the best method to do on plsql.
Thanks
Sudhir

Answer 1

The file upload will be a tad ugly.
You need to create an upload form/page in Apex that allows the user to upload the file. The Apache location configuration for that Database Access Descriptor (DAD) will include a parameter called PlsqlDocumentTablename - and this parameter specifies the name of the Oracle (or Apex) table to use for uploads.
The default document table is WWV_FLOW_FILE_OBJECTS$.
The 150Mb file will thus be a LOB in a row in the document table.
You can add a post PL/SQL process to that Apex upload page to process the uploaded document. Because of the size of the uploaded file, I suggest that you start a batch process (via the DBMS_JOB or DBMS_SCHEDULER interfaces) to process the upload file.
Okay, now for the processing. This is the ugly part - as the file is a LOB column in the database and you want to use it via SQL*Loader as an external file or external table. You can write your own CSV parser in PL/SQL. But I do not recommend using this approach for large files - SQL*Loader and external tables are a lot more flexible. I have not yet seen a means to define an external table on a CLOB - so this means having to write code using UTL_FILE and unload the 150Mb LOB contents to an actual external file. Once that is done, you can then use an external table on the external file to load it.
As for sending files to the client via Apex. This is done using the WPG_DOCLOAD.download_file() call to stream the LOB to the client browser. E.g. basic example:
--// URL example: http://my-server.my-domain.com/MyDAD/MySchemaName.StreamFile?fileID=1234
create or replace procedure StreamFile( fileID number ) AUTHID DEFINER is
        mimeType        varchar2(48);
        fileName        varchar2(400);
        lobContent      BLOB;
begin
        --// read the LOB from a table (or dynamically create it)
        select
                f.filename,
                f.mime_type,
                f.blob_content
                        into
                fileName,
                mimeType,
                lobContent
        from    FLOWS_FILES.WWV_FLOW_FILE_OBJECTS$ f
        where   f.id = fileID;
        --// format a basic HTTP header that describes the file stream send
        OWA_UTIL.mime_header( mimeType, FALSE );        -- e.g. text/csv text/plain text/html image/gif
        HTP.p( 'Content-Disposition: attachment; filename='||fileName );
        HTP.p( 'Content-Length: ' || DBMS_LOB.GetLength(lobContent) );
        OWA_UTIL.http_header_close;
        --// now write the BLOB as a mime stream using the Web Procedural Gateway's
        --// doc load API
        WPG_DOCLOAD.download_file( lobContent );
exception when OTHERS then
        --// Decide what HTML to generate (if at all) if there is a failure
        HTP.prn( 'StreamFile() failed with '||SQLERRM(SQLCODE) );
end;
/