Insert a large file into a Database?

I have the following situation and am unsure of the best solution.
I have to do an update on a record within an MS SQL Server database, adding files that are potentially 50mb plus in size.
Basically I would like to read this file into the database bit by bit so as not to get OutOfMemoryExceptions thrown whilst attempting to load a large file into memory.
Is there a way I can open the file and read x number of bytes at a time into the database, say with the FileInputStream.read(byte[]) in a while loop.

My preference would be to save the path of the file to the Database, but the requirements state the file has to be stored within a Database!
My problem is I am not sure if the preparedStatement.setBinaryStream() will work the way I would like.
I think this:
preparedStatement.setBinaryStream( 1, in.read(), 50000);
will just read the whole file into memory and error with OutOfMemoryException. Or will it read in a byte at a time?
I would like to Do Something like:
byte[] data = new byte[1024];
preparedStatement.setBinaryStream( 1,
while((eof = in.read(data)) != -1), 50000);
which obviously will not work, but I hope you get what I would like to try and do.
Put in words "pass small portions of the file from the Disk through JDBC into the database till the End Of File is reached".
Thanks for the response in advance.

Similar Messages

How to insert a image file into oracle database

hi all
can anyone guide me how to insert a image file into oracle database now
i have created table using
create table imagestore(image blob);
but when inserting i totally lost don't know what to do how to write query to insert image file

Hi I don't have time to explain really, I did have to do this a while ago though so I will post a code snippet. This is using the commons file upload framework.
Firstly you need a multi part form data (if you are using a web page). If you are not using a web page ignore this bit.
out.println("<form name=\"imgFrm\" method=\"post\" enctype=\"multipart/form-data\" action=\"FileUploadServlet?thisPageAction=reloaded\" onSubmit=\"return submitForm();\"><input type=\"FILE\" name=\"imgSource\" size='60' class='smalltext' onKeyPress='return stopUserInput();' onKeyUp='stopUserInput();' onKeyDown='stopUserInput();' onMouseDown='noMouseDown(event);'>");
out.println("   <input type='submit' name='submit' value='Submit' class='smalltext'>");
out.println("</form>"); Import this once you have the jar file:
import org.apache.commons.fileupload.*;Now a method I wrote to upload the file. I am not saying that this is correct, or its the best way to do this. I am just saying it works for me.
private boolean uploadFile(HttpServletRequest request, HttpSession session) throws Exception {
        boolean result = true;
        String fileName = null;
        byte fileData[] = null;
        String fileUploadError = null;
        String imageType = "";
        String error = "";
        DiskFileUpload fb = new DiskFileUpload();
        List fileItems = fb.parseRequest(request);
        Iterator it = fileItems.iterator();
        while(it.hasNext()){
            FileItem fileItem = (FileItem)it.next();
            if (!fileItem.isFormField()) {
                fileName = fileItem.getName();
                fileData = fileItem.get();
                // Get the imageType from the filename extension
                if (fileName != null) {
                    int dotPos = fileName.indexOf('.');
                    if (dotPos >= 0 && dotPos != fileName.length()-1) {
                        imageType = fileName.substring(dotPos+1).toLowerCase();
                        if (imageType.equals("jpg")) {
                            imageType = "jpeg";
        String filePath = request.getParameter("FILE_PATH");
        session.setAttribute("filePath", filePath);
        session.setAttribute("fileData", fileData);
        session.setAttribute("fileName", fileName);
        session.setAttribute("imageType", imageType);
        return result;
     } And now finally the method to actually write the file to the database:
private int writeImageFile(byte[] fileData, String fileName, String imageType, String mode, Integer signatureIDIn, HttpServletRequest request) throws Exception {
        //If the previous code found a file that can be uploaded then
        //save it into the database via a pstmt
        String sql = "";
        UtilDBquery udbq = getUser(request).connectToDatabase();
        Connection con = null;
        int signatureID = 0;
        PreparedStatement pstmt = null;
        try {
            udbq.setUsePreparedStatements(true);
            con = udbq.getPooledConnection();
            con.setAutoCommit(false);
            if((!mode.equals("U")) || (mode.equals("U") && signatureIDIn == 0)) {
                sql = "SELECT SEQ_SIGNATURE_ID.nextval FROM DUAL";
                pstmt = con.prepareStatement(sql);
                ResultSet rs = pstmt.executeQuery();
                while(rs.next()) {
                   signatureID = rs.getInt(1);
                if (fileName != null && imageType != null) {
                    sql = "INSERT INTO T_SIGNATURE (SIGNATURE_ID, SIGNATURE) values (?,?)";
                    InputStream is2 = new ByteArrayInputStream(fileData);
                    pstmt = con.prepareStatement(sql);
                    pstmt.setInt(1, signatureID);
                    pstmt.setBinaryStream(2, is2, (int)(fileData.length));
                    pstmt.executeUpdate();
                    pstmt.close();
                    con.commit();
                    con = null;
            if(mode.equals("U") && signatureIDIn != 0) {
                signatureID = signatureIDIn.intValue();
                if (fileName != null && imageType != null) {
                    sql = "UPDATE T_SIGNATURE SET SIGNATURE = ? WHERE SIGNATURE_ID = ?";
                    InputStream is2 = new ByteArrayInputStream(fileData);
                    pstmt = con.prepareStatement(sql);
                    pstmt.setBinaryStream(1, is2, (int)(fileData.length));
                    pstmt.setInt(2, signatureID);
                    pstmt.executeUpdate();
                    pstmt.close();
                    con.commit();
                    con = null;
        } catch (Exception e) {
            con = null;
            throw new Exception(e.toString());
        return signatureID;
   }

How to insert an image file in Oracle database

hi
can you please tell me how to insert an image file into oracle database????
suppose there is one image file in c:\pictures\rose.jpg. how to insert that file into database? theoretically i know that will be BFILE type but i dont know how to insert that.
will be waiting for your reply........
thanks & regards,
Priyatosh

Hello,
The easiest way to load a blob is to use SQL loader.
This example comes from the utilities guide:
LOAD DATA
INFILE 'sample.dat'
INTO TABLE person_table
FIELDS TERMINATED BY ','
(name CHAR(20),
1 ext_fname FILLER CHAR(40),
2 "RESUME" LOBFILE(ext_fname) TERMINATED BY EOF)
Datafile (sample.dat)
Johny Quest,jqresume.txt,
Speed Racer,'/private/sracer/srresume.txt',
Secondary Datafile (jqresume.txt)
Johny Quest
500 Oracle Parkway
Secondary Datafile (srresume.txt)
Loading LOBs
10-18 Oracle Database Utilities
Speed Racer
400 Oracle Parkway
regards,
Ivo

How do you insert files into a database

Hi
I believe oracle can store files.
I am trying to create a web application that will include an option of uploading files to oracle.
What kind of data type do i create in the database, is it a file of a blob or what is it.
Also locally how do i insert the file into the database. Would it be something like
Insert into <tablename>(id,name,picture) values(1,Jennifer Aniston,c:\pictures\Jennifer Aniston.jpg)
Is there any problem inserting the file using a web site url.
Thanks
Garry

The easiest way to store files in the database is to use BLOB's. You can use a PreparedStatement and call the following API's:
setBinaryStream(int parameterIndex, ...)
setBlob(int i, Blob x)
SCJP, SCJD, SCEA, SCWCD, SCBCD
CISSP, PMP, PHR
OCPDBA 8i/9i
http://www.certgear.com

Performance problems loading an XML file into oracle database

Hello ODI Guru's,
I am trying to load and XML file into the database after doing simple business validations. But the interface takes hours to complete.
1. The XML files are large in size >200 Mb. We have an XSD file for the schema definition instead of a DTD.
2. We used the external database feature for loading these files in database.
The following configuration was used in the XML Data Server:
jdbc:snps:xml?f=D:\CustomerMasterData1\CustomerMasterInitialLoad1.xml&d=D:\CustomerMasterData1\CustomerMasterInitialLoad1.xsd&re=initialLoad&s=CM&db_props=oracle&ro=true
3. Now we reverse engineer the XML files and created models using ODI Designer
4. Similar thing was done for the target i.e. an Oracle database table as well.
5. Next we created a simple interface with one-to-one mapping from the XSD schema to the Oracle database table and executed the interface. This execution takes more than one hour to complete.
6. We are running ODI client on Windows XP Professional SP2.
7. The Oracle database server(Oracle 10g 10.2.0.3) for the target schema as well as the ODI master and work repositories are on the same machine.
8. I tried changing the following properties but it is not making much visible difference:
use_prepared_statements=Y
use_batch_update=Y
batch_update_size=510
commit_periodically=Y
num_inserts_before_commit=30000
I have another problem that when I set batch_update_size to value greater that 510 I get the following error:
java.sql.SQLException: class org.xml.sax.SAXException
class java.lang.ArrayIndexOutOfBoundsException said -32413
at com.sunopsis.jdbc.driver.xml.v.a(v.java)
The main concern is why should the interface taking so long to execute.
Please send suggestions to resolve the problem.
Thanks in advance,
Best Regards,
Nikunj

Approximately how many rows are you trying to insert?
One of the techniques which I found improved performance for this scenario was to extract from the xml to a flat file, then to use SQL*LOADER or external tables to load the data into Oracle.

How to load PDF files into oracle database and display them in APEX

Hi All,
We have a requirement for loading the Loading PDF files (lots of PDf files) to the oracle database and then display the PDF files in the Oracel APEX report.
So that User can view the PDF files. Our current APEX verison is 3.2..
Please let me know how to implement it and where to find the sample application!
Thanks in advanced!
Jane

Thanks Tony for your quick response!
We need to load a lot of PDfs (history + current).
I have a questions about the SQL loader. I am trying to insert a pdf file into a table by following the oracle loading sample:
http://download.oracle.com/docs/cd/B10501_01/text.920/a96518/aload.htm
Example Data File: loader2.dat
This file contains the data to be loaded into each row of the table, articles_formatted.
Each line contains a comma separated list of the fields to be loaded in articles_formatted. The last field of every line names the file to be loaded in to the text column:
Ben Kanobi, plaintext,Kawasaki news article,../sample_docs/kawasaki.txt,
But i don't know to where should I put my pdf file on the server.
for example:
,../sample_docs/kawasaki.txt,
Where is the file 'kawasaki.txt'??
I try my local path, it didn't work. like c:\temp.
then I loaded teh PDf file into our server(/findev20/olmtest/orafin/11.5.7/olmtestcomn/temp) , and In my data file. I put the path
1, pdf_emp1,../findev20/olmtest/orafin/11.5.7/olmtestcomn/temp
but I got the error: the system can not find the file specified.
Where should I put the PDf files on the server and how to specify the path in the data file?
Thanks!
Jane

How to insert a JPG file into a Table from a Form?

I create a Schema and a Table as follows:
SQL> create user myphoto identified by myphoto;
User created.
SQL> grant connect,resource to myphoto;
Grant succeeded.
SQL> create table myphoto.photo_table
2 (photo_id varchar2(10) primary key,
3 photo_content blob);
Table created.
I would like to build an Oracle Form with a Text Item to enter the full path and filename to be uploaded and inserted into the photo_table; and a Push Button to insert the jpg file into the photo_table.
Can any one give me details on how this could be done?

Hi,
have a look at webutil on otn.oracle.com/products/forms ---> Webutil
Webutil has the capability to load files into the database.
Frank

Procedure to Insert PDF from FIle System to Database

Hi ,
I have a requirement to put the pdf files from our Local Machine or File system into Database table ..
For that I have created a directory called "MY_PDF" with create and replace directory .
Also , I have created a database table to store the files from Local machine FIle System directory ( MY_PDF) .
I have created a procedure which takes 2 inputs ( ID and FIlename ) as parameters , this procedure when executes , it insert the file along with the ID in the database table .
Procedure is as follows :
CREATE OR REPLACE PROCEDURE proc_load_a_file( p_id IN NUMBER, p_filename IN VARCHAR2 )
AS
l_blob BLOB;
l_bfile BFILE;
x VARCHAR2(1000);
BEGIN
x:=p_filename;
INSERT INTO demo(id,theblob,filename) VALUES ( p_id, empty_blob() ,x)
RETURNING theBlob INTO l_blob;
UPDATE demo
SET locater =l_blob where id=p_id;
l_bfile := bfilename( 'MY_FILES', p_filename );
DBMS_LOB.FILEOPEN( l_bfile );
DBMS_LOB.LOADFROMFILE( l_blob, l_bfile,
DBMS_LOB.getlength( l_bfile ) );
DBMS_LOB.fileclose( l_bfile );
COMMIT;
END;
Now , my requirement is not to insert one one file each and every time when I execute the procedure .
My requirement is first to check the File system Directory (MY_PDF) . If
there is any pdf file in the directory then it will call that procedure and insert that file into the database untill no files found in the dorectory ( MY_PDF) .
I am not getting the idea how to do this ..
Please someone provide some valueable inputs .. so that I can finish it up .
Thanks
Prashant Dwivedi

Suggest using the FlowElement.setStyle method to attach additional information about the image to the InlineGraphicElement that gets created. Looking at the example code in the posted link you'll need to pass that data to the imageLoadComplete method. The InlineGraphicElement can be found by saving interactionManager.absoluteStart before the graphic is inserted and then calling textFlow.findLeaf(savedAbsoluteStart).
Hope that helps,
Richard

How to import an .csv file into the database?

and can we code the program in JSP to import the.csv file into the database.

It is better to use Java class to read the CSV file and store the contents in the database.
You can use JSP to upload the CSV file to the server if you want, but don't use it to perform database operations.
JSPs are good for displaying information on the front-end, and for displaying HTML forms, there are other technologies more suitable for the middle layer, back end and the database layer.
So break you application into
1) Front end - JSPs to display input html forms and to display data retrieved from the database.
2) Middle layer - Servlets and JavaBeans to interact with JSPs. The code that reads the CSV file to parse it's contents should be a Java Class in the middle layer. It makes use of Java File I/O
3) Database layer - Connects to the database using JDBC (Java Database Connectivity), and then writes to the database with SQL insert statements.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Keeping the above concepts in mind, first build a simple JSP and get it to work,
then research on Google , for Java File I/O , discover how to read a file,
Then search on how to readh a CSV file using Java.
After researching you should be able to read the CSV file line by line and store each line inside a Collection.
Then research on Google, on how to write to the database using JDBC
Write a simple program that inserts something to a dummy table in the database.
Then, read the data stored in the Collection, and write insert statements for each records in the collection.

Loading Large XML into Oracle Database

Hi,
I am fairly new to XML DB. I have been successful in registering a schema to a table in the Database. Now, I have to load the appropriate XML into that table. I am using the Simple Bulk Loader program found on this oracle site, however, when I load my XML file I get the following error: ORA-21700: object does not exist or is marked for delete.
So, I figured maybe simple bulk loader cannot handle large files? So I reduced my XML file and loaded it with the program and it worked. However, does anyone know how I can load large files into my registered schema table.
Thanks,
Prerna :o)

Did you specify genTables true or false when registering the XML Schema ?
Does you XML schema contain a recursive definition
Is it possible that after reducing the size of the document you no longer have nodes that contain recursive structures...

A query while importing an XML file into a Database Table

Hi,
I am Creating an ODI Project to import an XML file into a Database Table.With the help of the following link
http://www.oracle.com/webfolder/technetwork/tutorials/obe/fmw/odi/odi_11g/odi_project_xml-to-table/odi_project_xml-to-table.htm
I am facing a problem while creating Physical Schema for the XML Source Model.
For the
Schema(Schema)
and
Schema(Work Schema) field they have selected GEO_D.
What is GEO_D here??
or
What I have to select here?
1) schema of the xml file (NB:-I havn't created any .xsd or .dtd file for my .xml file)
or
2)my target servers schema
Please tell me what I'll do??
Thanks

and
Schema(Work Schema) field they have selected GEO_D.
What is GEO_D here??This is the schema name which is specified in the XML file .
What I have to select here?
1) schema of the xml file (NB:-I havn't created any .xsd or .dtd file for my .xml file)Yes
2)my target servers schema
Please tell me what I'll do??
Thanks

Inserting flash .swf file into template

I want to insert a flash file into the right sidebar to appear on all pages of the website. I know I need to do this via the template. I think I need to click in the insertion point, then copy the code (make sure the files are in the correct location). I've done this before with galleries ect.. but not in DW and not from a template. What's a little confusing is there are 3 templates. The first is in it's own folder (design template), the other two (theme 01 design & theme 01 design from template) are in the root folder, but the theme 01 design one doesn't seem to be changeale. Which template should I be working with?
www.wranglergirl.net
Thanks from a newbie! Amy

Are each of these 'templates' actually DW templates (i.e., are they named with a *.dwt extension)? If so, then they must all be located within the Templates folder at the root level of the site - it doesn't sound like that is the case. Also, why do you need three templates? Finally, which version of DW are you using? It's likely that you will have to make any links to your Flash element with ROOT relative links to be successful.

Can I insert the image file into PDF file in Adobe X Standard ?

can I insert the image file into PDF file in Adobe X Standard ?

http://matt.coneybeare.me/how-to-make-an-html-signature-in-apple-mail-for-maveri cks-os-x-10-dot-9/

How to insert a gif file into a table?

Hi,
I need to insert a gif file into a table. Can anyone tell me where I can find the information on how to create this kind of table, how to insert a gif file into a table and how to select?
Thanks,
Helen

Hi Helen,
You could read about that in the documentation which is available online.
For a starter: BLOB. And I bet there are many examples to be found on the web.
Good luck. :)
Regards,
Guido

How can I insert a completed file into another PDF file I am developing?

How can I insert a completed file into another PDF file I am developing?

Please refer : http://acrobatusers.com/tutorials/how-to-insert-a-pdf-into-a-pdf

Insert a large file into a Database?

Similar Messages

Maybe you are looking for