Importing a table with a BLOB column is taking too long

I am importing a user schema from 9i (9.2.0.6) database to 10g (10.2.1.0) database. One of the large tables (millions of records) with a BLOB column is taking too long to import (more that 24 hours). I have tried all the tricks I know to speed up the import. Here are some of the setting:
1 - set buffer to 500 Mb
2 - pre-created the table and turned off logging
3 - set indexes=N
4 - set constraints=N
5 - I have 10 online redo logs with 200 MB each
6 - Even turned off logging at the database level with disablelogging = true
It is still taking too long loading the table with the BLOB column. The BLOB field contains PDF files.
For your info:
Computer: Sun v490 with 16 CPUs, solaris 10
memory: 10 Gigabytes
SGA: 4 Gigabytes

Legatti,
I have feedback=10000. However by monitoring the import, I know that its loading average of 130 records per minute. Which is very slow considering that the table contains close to two millions records.
Thanks for your reply.

Similar Messages

Batch load images (tiff) into table with a BLOB column

Hi,
Does anyone know any third-party tool/software to bulk/batch load images into Oracle table? There is an ETL software to regularly pull the images (tiff) from a ftp server to a directory in the database server. There is no way I can hardcode the image filenames into the control file and use SQL*Loader to load the files. Is there any tool/software that can just grab whatever files in the directory and load them into Oracle. Then, I can write a program to extract the filename from the file already loaded into the table and update the filename column.
Thanks.
Andy

sound like simple scripting to me...
-- SQL loader example
http://www.orafaq.com/wiki/SQL%2ALoader_FAQ#How_does_one_use_SQL.2ALoader_to_load_images.2C_sound_clips_and_documents.3F
-- dynamically build control file
devuser1:/tmp>>touch image1.gif
devuser1:/tmp>>touch image2.jpg
devuser1:/tmp>>touch image3.jpg
devuser1:/tmp>>ls -l image*
-rw-rw-r--   1 devuser1   mygrp           0 Jul 10 11:19 image1.gif
-rw-rw-r--   1 devuser1   mygrp           0 Jul 10 11:19 image2.jpg
-rw-rw-r--   1 devuser1   mygrp           0 Jul 10 11:19 image3.jpg
devuser1:/tmp>>ls -l image* | awk '{ print NR "," $9}'
1,image1.gif
2,image2.jpg
3,image3.jpg
devuser1:/tmp>>echo "LOAD DATA" > t.ctl
devuser1:/tmp>>echo "INFILE *" >> t.ctl
devuser1:/tmp>>echo "..." >> t.ctl
devuser1:/tmp>>echo "BEGINDATA" >> t.ctl
devuser1:/tmp>>ls -l image* | awk '{ print NR "," $9}' >> t.ctl
devuser1:/tmp>>cat t.ctl
LOAD DATA
INFILE *
BEGINDATA
1,image1.gif
2,image2.jpg
3,image3.jpgEdited by: maceyah on Jul 10, 2009 12:42 PM

Accessing Tables using global index rowid is taking too long.

I am using Oracle 10.2.0.4.0. I am having tables which are ~100+GB in size which is range partitioned on a date column and hash subpartitioned on primary key column. And accessing these tables using global index row id using idex range scan( size of those indexes are ~7+ GB) taking ~5+ mins which results in ~1000 rows as out put. So is this means there is some structural problem with tables/indexes or its something issue at block level? How to dig more into this issue?

Hi,
930254 wrote:
I am using Oracle 10.2.0.4.0. I am having tables which are ~100+GB in size which is range partitioned on a date column and hash subpartitioned on primary key column. And accessing these tables using global index row id using idex range scan( size of those indexes are ~7+ GB) taking ~5+ mins which results in ~1000 rows as out put. So is this means there is some structural problem with tables/indexes or its something issue at block level? How to dig more into this issue?once you get a chance to run the query and collect some diagnostic info, here are the questions that you should ask yourself:
1) where is the 5 mins spent? doing index range scan itself, doing table index access by global rowid, waiting for something, doing something else?
2) how many reads are there and what kind (logical, physical, direct path)? per the entire query and per index range scan
3) is there any aggregation in the query? if there is, then 1000 rows output doesn't mean anything -- you could have to access millions or billions of rows to produce a 1000 row output in an aggregate query
The best tool to answer these questions is dbms_xplan.
Best regards,
Nikolay

Select query with primary key on RESB taking too long

Hello All,
The below Select query is taking about 1 Hour to run on production. I have done some analysis and the correct index is being selected.
The query is :
IF it_aufnr[] IS NOT INITIAL.
    SELECT rsnum
           rspos
           matnr
           werks
        lgort
           shkzg
           aufnr
           bdmng
           enmng
           FROM resb INTO TABLE it_rsnum FOR ALL ENTRIES IN it_aufnr
           WHERE rsnum EQ it_aufnr-rsnum
           AND   xloek NE 'X'
           AND   postp NE 'X'
           AND   resb~bdmng GE resb~enmng.
ENDIF.
Database Table RESB: 40,000,000 Records (40 Million)
Internal Table   it_aufnr: 20,000 Entries
Entries selected from RESB: 150,000.
Any ideas on how I could improve the performance ?
Thanks,
Shruthi

>     SELECT rsnum
>            rspos
>            matnr
>            werks
>         lgort
>            shkzg
>            aufnr
>            bdmng
>            enmng
>            FROM resb INTO TABLE it_rsnum FOR ALL ENTRIES IN it_aufnr
>            WHERE rsnum EQ it_aufnr-rsnum
>            AND   xloek NE 'X'
>            AND   postp NE 'X'
>            AND   resbbdmng GE resbenmng.
>   ENDIF.
>
> Database Table RESB: 40,000,000 Records (40 Million)
> Internal Table   it_aufnr: 20,000 Entries
> Entries selected from RESB: 150,000.
>
Hi,
the problem is the big for all entries table.
Your 20.000 records FAE will be split into SEVERAL sql statements depending on the size of the table.
Where do you get the it_auftrn from?
If it's another transparent table try to JOIN this table to your big table.
SELECT rsnum
            rspos
            matnr
            werks
        lgort
            shkzg
            aufnr
            bdmng
            enmng
            FROM resb JOIN tab_auftrn on resbrsnum = tab_auftrnauftrn
            AND   xloek NE 'X'
            AND   postp NE 'X'
            AND   resbbdmng GE resbenmng.
Make sure that your WHERE filter and the JOIN keys on both tables are supported by indexes.
150.000 records about of 40 Mio. can definitly be serviced by an appropriate index:
i.e. Index fields (check your PK or secondary indexes):
rsnum
enmng
aufnr
bye
yk

Select query based on joining of tables from different database is taking too long

Hi Team,
Select query on table with millions of records is taking very long time. It took 50mins, the below query is joining on multiple table from two databases DB1.dbo.Table1 contains 100 million records and also Table3 and Table4 (of different database) might
contain close to 1million records.
Select T1.*
From DB1.dbo.Table1 T1
Join DB1.dbo.Table2 T2 on RTRIM(T1.Col3) = RTRIM(T2.Col3)
Join Table4 CA on RTRIM(T1.Col1) + T2.Col2 = CA.Col1
Join Table3 U on CA.Col2 = U.Col2 AND RTRIM(T2.Col2) = U.Col3
Where U.Col4 NOT IN ('A1', 'A2', 'A3', 'A4', 'A5', 'A6','A7','A8','A9')
And (T1.flg IS NULL OR T1.flg = 'N')
And LTRIM(RTRIM(T2.Col2)) NOT IN ('B1','B2')
How can i improve the performance of this query. Actual thing is update the data in Db1.dbo.table1 based on the conditions but if the select is taking close to 1hr then update will take hours together. Indexes already implemented on all the tables.
Thanks,
Eshwar.
Please don't forget to Marked as Answer if my post solved your problem and use Vote As Helpful if a post was useful. It will helpful to other users.

Thanks.
Join Table4 CA on RTRIM(T1.Col1) + T2.Col2 = CA.Col1
This join is not working with spaces if Rtrim is not used.
Thanks! Eshwar.
Please don't forget to Marked as Answer if my post solved your problem and use Vote As Helpful if a post was useful. It will helpful to other users.

Problem importing a table with blob's

hi all, I'm facing the following situation.
Source DB : 10.2.0.3 (client's DB)
Destination DB (mine): 10.2.0.4
I've a dump file (traditional) of a particular schema.
I'm running import (imp) to import on my DB.
It runs fine until it reaches one particular table. This table has 6 colums, 3 of them are BLOB.
This table has 260000 rows (checked with export log).
When import reaches row 152352 it stops loading data, but import is still running.
what can I do to get more information from this situation in order to solve this problem?
Any suggestion will be appreciated!
Thanks in advance.

Pl identify the source and target OS versions. Are there any useful messages in the alert.log ? How long did the export take ? Rule of thumb states import will take twice as long. Have you tried expdp/impdp instead ? Also see the following -
How To Diagnose And Troubleshoot Import Or Datapump Import Hung Scenarios          (Doc ID 795034.1)
How To Find The Cause of a Hanging Import Session          (Doc ID 184842.1)
Import is Slow or Hangs          (Doc ID 1037231.6)
Export and Import of Table with LOB Columns (like CLOB and BLOB) has Slow Performance          (Doc ID 281461.1)
HTH
Srini

Using ESB + DBAdapter with a BLOB column

Hi
Are there any tips and tricks for using the database adapter with a BLOB column from a table?
I'd like to get a hex coded string out, if I put 012346789ABCDEF into the blob colum I get ASNFZ4mrze8= as the output in the file adapter.
Any tips welcome
Angus

Legatti,
I have feedback=10000. However by monitoring the import, I know that its loading average of 130 records per minute. Which is very slow considering that the table contains close to two millions records.
Thanks for your reply.

Import of Tables with CLOB problem.

Hi,
I am having problems with the import of tables with CLOB columns.
Tables with CLOB columns will force the use of the tablespace in which it (table) was created originally. It will not be created on the new schema's default tablespace. Is there a way to get around this problem?
More on this problem can be found here :
Re: Import error 1647, CLOB fields problem, need help!
I am using Oracle 9.2.0.6 standard edition, that is why the transportable tablespace option will not work.
Thanx
Rob

Sorry I have posted the wrong link, it should be:
Re: Import error 1647, CLOB fields problem, need help!
Regards
Rob

How to create a table with datatype blob and insert a pdf file (ravi)

how to create a table with datatype blob and insert a pdf file,
give me the explain asap
1.create the table?
2.insert the pdffiles into tables?
3.how to view the files?
Thanks & Regards
ravikumar.k
Edited by: 895044 on Dec 5, 2011 2:55 AM

895044 wrote:
how to create a table with datatype blob and insert a pdf file,
give me the explain asapPerhaps you should read...
{message:id=9360002}
especially point 2.
We're not just sitting here waiting to answer your question as quickly as possible for you.

What index is suitable for a table with no unique columns and no primary key

alpha
beta
gamma
col1
col2
col3
100
1
-1
a
b
c
100
1
-2
d
e
f
101
1
-2
t
t
y
102
2
1
j
k
l
Sample data above and below is the dataype for each one of them
alpha datatype- string
beta datatype-integer
gamma datatype-integer
col1,col2,col3 are all string datatypes.
Note:columns are not unique and we would be using alpha,beta,gamma to uniquely identify a record .Now as you see my sample data this is in a table which doesnt have index .I would like to have a index created covering these columns (alpha,beta,gamma) .I
beleive that creating clustered index having covering columns will be better.
What would you recommend the index type should be here in this case.Say data volume is 1 milion records and we always use the alpha,beta,gamma columns when we filiter or query records
what index is suitable for a table with no unique columns and primary key?
col1
col2
col3
Mudassar

Many thanks for your explanation .
When I tried querying using the below query on my heap table the sql server suggested to create NON CLUSTERED INDEX INCLUDING columns ,[beta],[gamma] ,[col1]
,[col2] ,[col3]
SELECT [alpha]
,[beta]
,[gamma]
,[col1]
,[col2]
,[col3]
FROM [TEST].[dbo].[Test]
where [alpha]='10100'
My question is why it didn't suggest Clustered INDEX and chose NON clustered index ?
Mudassar

Cartesian of data from two tables with no matching columns

Hello,
I was wondering – what’s the best way to create a Cartesian of data from two tables with no matching columns in such a way, so that there will be only a single SQL query generated?
I am thinking about something like:
for $COUNTRY in ns0: COUNTRY ()
for $PROD in ns1:PROD()
return <Results>
<COUNTRY> {fn:data($COUNTRY/COUNTRY_NAME)} </COUNTRY>
<PROD> {fn:data($PROD/PROD_NAME)} </PROD>
</Results>
And the expected result is combination of all COUNTRY_NAMEs with all PROD_NAMEs.
What I’ve noticed when checking query plan is that DSP will execute two queries to have the results – one for COUNTRY_NAME and another one for PROD_NAME. Which in general results in not the best performance ;-)
What I’ve noticed also is that when I add something like:
where COUNTRY_NAME != PROD_NAME
everything is ok and there is only one query created (it's red in the Query plan, but still it's ok from my pov). Still it looks to me more like a workaround, not a real best approach. I may be wrong though...
So the question is – what’s the suggested approach for such queries?
Thanks,
Leszek
Edited by xnts at 11/19/2007 10:54 AM

Which in general results in not the best performanceI disagree. Only for two tables with very few rows, would a single sql statement give better performance.
Suppose there are 10,000 rows in each table - the cross-product will result in 100 million rows. Sounds like a bad idea. For this reason, DSP will not push a cross-product to a database. It will get the rows from each table in separate sql statements (retrieving only 20,000 rows) and then produce the cross-product itself.
If you want to execute sql with cross-products, you can create a sql-statement based dataservice. I recommend against doing so.

How to create table with rows and columns in the layout mode?

One of my friends advised me to develop my whole site on the
layout mode as its better than the standard as he says
but I couldnot make an ordinary table with rows and columns
in th layout mode
is there any one who can tell me how to?
thanx alot

Your friend is obviously not a reliable source of HTML
information.
Murray --- ICQ 71997575
Adobe Community Expert
(If you *MUST* email me, don't LAUGH when you do so!)
==================
http://www.dreamweavermx-templates.com
- Template Triage!
http://www.projectseven.com/go
- DW FAQs, Tutorials & Resources
http://www.dwfaq.com - DW FAQs,
Tutorials & Resources
http://www.macromedia.com/support/search/
- Macromedia (MM) Technotes
==================
"Mr.Ghost" <[email protected]> wrote in
message
news:f060vi$npp$[email protected]..
> One of my friends advised me to develop my whole site on
the layout mode
> as its
> better than the standard as he says
> but I couldnot make an ordinary table with rows and
columns in th layout
> mode
> is there any one who can tell me how to?
> thanx alot
>

Problem import excel table with photos

When I import excel table with photos, the photos don't appear

Welcome to Project Siena!
If you're seeing an 'x' instead of your photo in the data source it's possible that Project Siena can't access the directory. Try saving your images in another folder such as C:\Users\Public\Pictures.
Others have also reported that if the Library isn't set up correctly in Windows 8 that they've had issues.
Here are some posts to look at while we wait for additional information from you:
Importing local images via an Excel file
Siena Gallery unable to load image
Images in Public Pictures directories showing
up as X in Img from URL
(fyi - this last post's screenshot will look different than yours as it was from Beta 1.)
Thor

How to import a table with renamed name

If I want to import a table from a DMP on UNIX towards my user where I already have a new version but I don't want to drop this new version 'cause I have only to move some records from one to the other, is there an import option for importing the table with another name? I mean I want to import on the same user, where there already exists the table MYTABLE, an old version of MYTABLE by impoting it with the new name MYTABLE_OLD, so afterwards I can update MYTABLE with some of old records of its old versione MYTABLE_OLD. Is it possible?

You cannot directly do this.
You can import the Table into a different schema and then rename the table.
export and import back into the required schema.
Import DataPump: How to Import Table Data into a Table that has Different Name ? [ID 342314.1]
Regards
Rajesh

Tabular Form based on table with lots of columns - how to avoid scrollbar?

Hi everybody,
I'm an old Forms and VERY new APEX user. My problem is the following: I have to migrate a form application to APEX.
The form is based on a table with lots of columns. In Forms you can spread the data over different tab pages.
How can I realize s.th similar in APEX? I definitely don't want to use a horizontal scroll bar...
Thanks in advance
Hilke

If the primary key is created by the user themselves (which is not recommended, you should have another ID which the user sees which would be the varchar2 and keep the primary key as is, the user really shouldn't ever edit the primary key) then all you need to do is make sure that the table is not populated with a primary key in the wizard and then make sure that you cannot insert a null into your varchar primary key text field.
IF you're doing it this way I would make a validation on the page which would run off a SQL Exists validation, something along the lines of
SELECT <primary key column>
FROM <your table>
WHERE upper(<primary key column>) = upper(<text field containing user input>);
and if it already exists, fire the validation claiming that it already exists and to come up with a new primary key.
Like I said if you really should have a primary key which the database refers to each individual record itself and then have an almost pseudo-primary key that the user can use. For example in the table it would look like this:
TABLE1
table_id (this is the primary key which you should NOT change)
user_table_id (this is the pretend primary key which the user can change)
other_columns
etc
etc
hope this helps in some way

Importing a table with a BLOB column is taking too long

Similar Messages

Maybe you are looking for