Max limit for bulk insert.
Hi friends,
We have 100 Million records in a table on production database. We are planning to move the data to test environment for performance tuning of the databse and queries.
We have created a DB Link between these DB's to transfer the data between the database.
We are planning to move the data in the following fashion:
insert into tab1(.....)
select * from tab1@prod_db_link;
do we have any limit for bulk insert?
> We are planning to move the data to test environment for performance tuning of the databse and
queries.
Flawed premise. What makes you think that the test environment will match the production with a 100% degree of accuracy!? - as that is what you need in order to
a) identify the performance bottlenecks
b) solve these
You are very much mistaken if you think you can identify actual performance issues happening on prod by mucking about on a test environment that just happens to have the same data volume.
Even data volume alone is meaningless as a copy of the data is logical - any potential issues with pctfree and pctused will not be reflected. Any potential hot spots on disk will not be reflected. The copy will not have the same number of segments and extents. Etc. Etc.
And this is just the data.. never mind numerous other issues ranging from actual hardware and operating system to Oracle instance configuration and production load and processing.
Similar Messages
-
Hi,
The client has varying levels of people reporting levels.Some go down just three levels, others go down 14 levels.
What is robust way to model and store that hierarchy and present its hierarchy values in OBI.
Is 11 the max limit for Hierarchy?
ThanksAFAIK, I dont think there is any thing limiting about OBIEE in terms of max. levels within a hierarchy other than the limits set by your data source (oracle has 1000 column limit AFAIK in a table).
Swapan. -
Max limit reached for tablespace in SAP BW
Hi Friends,
We are having one SAP BW system on DB6, AIX...in that the max limit for tablespabe is 512GB, which is going to be exceeded soon...
So it is planned to move some tables from tablespaces of aggregates i.e. FACTI, FACTD into new tablespaces...
How can this be achieved?
Thanks & Regards,
Kunal Patel.He Kunal,
As mentioned by Jens, you can use the DB2 9's Large RIDS (Larger Record Identifiers) support which significantly increases the size limit for table space. SAP has a tools to do the conversion for you - OSS Note 11058956 (ABAP tool for activating large RIDs).
Regards,
-Beck -
SQL Server 2012 Express bulk Insert flat file 1million rows with "" as delimeter
Hi,
I wanted to see if anyone can help me out. I am on SQL server 2012 express. I cannot use OPENROWSET because my system is x64 and my Microsoft office suit is x32 (Microsoft.Jet.OLEDB.4.0).
So I used Import wizard and is not working either.
The only thing that let me import this large file, is:
CREATE TABLE #LOADLARGEFLATFILE
Column1
varchar(100), Column2 varchar(100), Column3 varchar(100),
Column4 nvarchar(max)
BULK INSERT
#LOADLARGEFLATFILE
FROM 'C:\FolderBigFile\LARGEFLATFILE.txt'
WITH
FIRSTROW = 2,
FIELDTERMINATOR ='\t',
ROWTERMINATOR ='\n'
The problem with CREATE TABLE and BULK INSERT is that my flat file comes with text qualifiers - "". Is there a way to prevent the quotes "" from loading in the bulk insert? Below is the data.
Column1
Column2
Column3
Column4
"Socket Adapter"
8456AB
$4.25
"Item - Square Drive Socket Adapter | For "
"Butt Splice"
9586CB
$14.51
"Item - Butt Splice"
"Bleach"
6589TE
$27.30
"Item - Bleach | Size - 96 oz. | Container Type"
Ed,
Edwin LoperaHi lgnusLumen,
According to your description, you use BULK INSERT to import data from a data file to the SQL table. However, to be usable as a data file for bulk import, a CSV file must comply with the following restrictions:
1. Data fields never contain the field terminator.
2. Either none or all of the values in a data field are enclosed in quotation marks ("").
In your data file, the quotes aren't consistent, if you want to prevent the quotes "" from loading in the bulk insert, I recommend you use SQL Server Import and Export Wizard tools in SQL Server Express version. area, it will allow to strip the
double quote from columns, you can review the following screenshot.
In other SQL Server version, we can use SQL Server Integration Services (SSIS) to import data from a flat file (.csv) with removing the double quotes. For more information, you can review the following article.
http://www.mssqltips.com/sqlservertip/1316/strip-double-quotes-from-an-import-file-in-integration-services-ssis/
In addition, you can create a function to convert a CSV to a usable format for Bulk Insert. It will replace all field-delimiting commas with a new delimiter. You can then use the new field delimiter instead of a comma. For more information, see:
http://stackoverflow.com/questions/782353/sql-server-bulk-insert-of-csv-file-with-inconsistent-quotes
Regards,
Sofiya Li
Sofiya Li
TechNet Community Support -
Bulk Insert into a Table from CSV file
I have a CSV file with 1000 records and i have to insert those records into a table.
I tried for Bulk Insert command and Load data infile command but it throws error.
Am using Oracle 10g Express Edition.
I want to achieve it thru query command and not by plsql procedures
Please send me query syntax for this problem. . . .
Thanks in Advance,
Hariharan ST.Hi
If you create an external table that points to your csv file you will then be able populate your table from a query.
See: http://www.astral-consultancy.co.uk/cgi-bin/hunbug/doco.cgi?11210
Hope this helps -
Hello, I have done some tests to check the heap size of an application.
This is my test code:
public class Main {
public static void main(String[] args) {
Runtime runtime = Runtime.getRuntime();
// Max limit for heap allocation.
// It's in bytes.
long heapLimitBytes = runtime.maxMemory();
// Currently allocated heap.
// It's in bytes
long allocatedHeapBytes = runtime.totalMemory();
// Unused memory from the allocated heap.
// It's an approximation, it's in bytes.
long unusedAllocatedHeapBytes = runtime.freeMemory();
// Used memory from the allocated heap.
// It's an approximation, it's in bytes.
long usedAllocatedHeapBytes =
allocatedHeapBytes - unusedAllocatedHeapBytes;
System.out.println("Max limit for heap allocation: " +
getMBytes(heapLimitBytes) + "MB");
System.out.println("Currently allocated heap: " +
getMBytes(allocatedHeapBytes) + "MB");
System.out.println("Used allocated heap: " +
getMBytes(usedAllocatedHeapBytes) + "MB");
private static long getMBytes(long bytes) {
return (bytes / 1024L) / 1024L;
}Then I run this program with the option -Xmx1024m, and the result was:
On windows: Max limit for heap allocation: 1016MB
On HP-UX: Max limit for heap allocation: 983MB
Someone knows why the max limit is not 1024MB as I requested?
And why it shows a different value in windows than in HP-UX?
Thanks
Edited by: JoseLuis on Oct 5, 2008 11:29 AMThank you for the reply
I have checked and the page size in windows and HP-UX is 4KB.
Also, in the documentation for the -Xmx flag it says that the size must be multiple of 1024 bytes and bigger than 2 MB.
I may understand that the allocated size can be rounded to the nearest page (4 KB block), wich give you a difference less than 1 MB between the requested size and the real allocated size, but in windows the the difference is 8 MB and in HP-UX the difference is 41 MB. It's a big difference.
Am I missing something? -
Unique constraint violation error while bulk insert (TimesTen 7.0.1.0.0)
Hi,
I try to understand while I get error above when doing bulk inserts via TimesTen into Oracle database.
The single inserts after that works perfect (no record will dropped), when doing direct to oracle bulk insert works with same data perfect. We tried it with AWT, then with readonly tables and passtrough=2, nothing works. The error comes even the table is empty.
Any suggestions? is this a bug in timesten itself? Have I connect for bulk inserts direct to oracle database?
Thanks in advanced
Rajko AlbrechtIt may be a TT bug but if so it is not an obvious one since bulk inserts definitely work in TimesTen...
Can you please provide:
1. The schema of the table in question (including any indices)
2. Details on how you are doing the bulk inserts (C/ODBC program, Java/JDBC program or ...). Actual program source code would be helpful.
3. A (small) example of the data that you know would give this error.
Thanks,
Chris -
Bulk insertion of CTI Route points in CUCM
Hi team,
Is there any way we can insert bulk number of CTI ROUTE POINT's in Cisco Call Manager. I have seen the option for bulk insertion of phones.But i didn't get any option for creating bulk number of CTI Route Points.
Can someone please help me on this issue.
Thanks,
Sasikumar.Hi Gergely,
Thankyou for responding.
--> The CUCM is in 10.0 version.
--> Actually i am pretty new to CUCM so i didn't understand what you mean by look at AXL .Can you please elaborate more.
Thanks in advance,
Sasikumar. -
Hi all,
I have a performance issue in the below code,where i am trying to insert the data from table_stg into target_tab and in parent_tab tables and then to child tables via cursor with bulk collect .the target_tab and parent_tab are huge tables and have a row wise trigger enabled on it .the trigger is mandatory . This timetaken for this block to execute is 5000 seconds.Now my requirement is to reduce it to 5 to 10 mins.
can someone please guide me here.Its bit urgent .Awaiting for your response.
declare
vmax_Value NUMBER(5);
vcnt number(10);
id_val number(20);
pc_id number(15);
vtable_nm VARCHAR2(100);
vstep_no VARCHAR2(10);
vsql_code VARCHAR2(10);
vsql_errm varchar2(200);
vtarget_starttime timestamp;
limit_in number :=10000;
idx number(10);
cursor stg_cursor is
select
DESCRIPTION,
SORT_CODE,
ACCOUNT_NUMBER,
to_number(to_char(CORRESPONDENCE_DATE,'DD')) crr_day,
to_char(CORRESPONDENCE_DATE,'MONTH') crr_month,
to_number(substr(to_char(CORRESPONDENCE_DATE,'DD-MON-YYYY'),8,4)) crr_year,
PARTY_ID,
GUID,
PAPERLESS_REF_IND,
PRODUCT_TYPE,
PRODUCT_BRAND,
PRODUCT_HELD_ID,
NOTIFICATION_PREF,
UNREAD_CORRES_PERIOD,
EMAIL_ID,
MOBILE_NUMBER,
TITLE,
SURNAME,
POSTCODE,
EVENT_TYPE,
PRIORITY_IND,
SUBJECT,
EXT_PRD_ID_TX,
EXT_PRD_HLD_ID_TX,
EXT_SYS_ID,
EXT_PTY_ID_TX,
ACCOUNT_TYPE_CD,
COM_PFR_TYP_TX,
COM_PFR_OPT_TX,
COM_PFR_RSN_CD
from table_stg;
type rec_type is table of stg_rec_type index by pls_integer;
v_rt_all_cols rec_type;
BEGIN
vstep_no := '0';
vmax_value := 0;
vtarget_starttime := systimestamp;
id_val := 0;
pc_id := 0;
success_flag := 0;
vstep_no := '1';
vtable_nm := 'before cursor';
OPEN stg_cursor;
vstep_no := '2';
vtable_nm := 'After cursor';
LOOP
vstep_no := '3';
vtable_nm := 'before fetch';
--loop
FETCH stg_cursor BULK COLLECT INTO v_rt_all_cols LIMIT limit_in;
vstep_no := '4';
vtable_nm := 'after fetch';
--EXIT WHEN v_rt_all_cols.COUNT = 0;
EXIT WHEN stg_cursor%NOTFOUND;
FOR i IN 1 .. v_rt_all_cols.COUNT
LOOP
dbms_output.put_line(upper(v_rt_all_cols(i).event_type));
if (upper(v_rt_all_cols(i).event_type) = upper('System_enforced')) then
vstep_no := '4.1';
vtable_nm := 'before seq sel';
select PC_SEQ.nextval into pc_id from dual;
vstep_no := '4.2';
vtable_nm := 'before insert corres';
INSERT INTO target1_tab
(ID,
PARTY_ID,
PRODUCT_BRAND,
SORT_CODE,
ACCOUNT_NUMBER,
EXT_PRD_ID_TX,
EXT_PRD_HLD_ID_TX,
EXT_SYS_ID,
EXT_PTY_ID_TX,
ACCOUNT_TYPE_CD,
COM_PFR_TYP_TX,
COM_PFR_OPT_TX,
COM_PFR_RSN_CD,
status)
VALUES
(pc_id,
v_rt_all_cols(i).party_id,
decode(v_rt_all_cols(i).product_brand,'LTB',2,'HLX',1,'HAL',1,'BOS',3,'VER',4,0),
v_rt_all_cols(i).sort_code,
'XXXX'||substr(trim(v_rt_all_cols(i).ACCOUNT_NUMBER),length(trim(v_rt_all_cols(i).ACCOUNT_NUMBER))-3,4),
v_rt_all_cols(i).EXT_PRD_ID_TX,
v_rt_all_cols(i).EXT_PRD_HLD_ID_TX,
v_rt_all_cols(i).EXT_SYS_ID,
v_rt_all_cols(i).EXT_PTY_ID_TX,
v_rt_all_cols(i).ACCOUNT_TYPE_CD,
v_rt_all_cols(i).COM_PFR_TYP_TX,
v_rt_all_cols(i).COM_PFR_OPT_TX,
v_rt_all_cols(i).COM_PFR_RSN_CD,
NULL);
vstep_no := '4.3';
vtable_nm := 'after insert corres';
else
select COM_SEQ.nextval into id_val from dual;
vstep_no := '6';
vtable_nm := 'before insertcomm';
if (upper(v_rt_all_cols(i).event_type) = upper('REMINDER')) then
vstep_no := '6.01';
vtable_nm := 'after if insertcomm';
insert into parent_tab
(ID ,
CTEM_CODE,
CHA_CODE,
CT_CODE,
CONTACT_POINT_ID,
SOURCE,
RECEIVED_DATE,
SEND_DATE,
RETRY_COUNT)
values
(id_val,
lower(v_rt_all_cols(i).event_type),
decode(v_rt_all_cols(i).product_brand,'LTB',2,'HLX',1,'HAL',1,'BOS',3,'VER',4,0),
'Email',
v_rt_all_cols(i).email_id,
'IADAREMINDER',
systimestamp,
systimestamp,
0);
else
vstep_no := '6.02';
vtable_nm := 'after else insertcomm';
insert into parent_tab
(ID ,
CTEM_CODE,
CHA_CODE,
CT_CODE,
CONTACT_POINT_ID,
SOURCE,
RECEIVED_DATE,
SEND_DATE,
RETRY_COUNT)
values
(id_val,
lower(v_rt_all_cols(i).event_type),
decode(v_rt_all_cols(i).product_brand,'LTB',2,'HLX',1,'HAL',1,'BOS',3,'VER',4,0),
'Email',
v_rt_all_cols(i).email_id,
'CORRESPONDENCE',
systimestamp,
systimestamp,
0);
END if;
vstep_no := '6.11';
vtable_nm := 'before chop';
if (v_rt_all_cols(i).ACCOUNT_NUMBER is not null) then
v_rt_all_cols(i).ACCOUNT_NUMBER := 'XXXX'||substr(trim(v_rt_all_cols(i).ACCOUNT_NUMBER),length(trim(v_rt_all_cols(i).ACCOUNT_NUMBER))-3,4);
insert into child_tab
(COM_ID,
KEY,
VALUE)
values
(id_val,
'IB.Correspondence.AccountNumberMasked',
v_rt_all_cols(i).ACCOUNT_NUMBER);
end if;
vstep_no := '6.1';
vtable_nm := 'before stateday';
if (v_rt_all_cols(i).crr_day is not null) then
insert into child_tab
(COM_ID,
KEY,
VALUE)
values
(id_val,
--'IB.Correspondence.Date.Day',
'IB.Crsp.Date.Day',
v_rt_all_cols(i).crr_day);
end if;
vstep_no := '6.2';
vtable_nm := 'before statemth';
if (v_rt_all_cols(i).crr_month is not null) then
insert into child_tab
(COM_ID,
KEY,
VALUE)
values
(id_val,
--'IB.Correspondence.Date.Month',
'IB.Crsp.Date.Month',
v_rt_all_cols(i).crr_month);
end if;
vstep_no := '6.3';
vtable_nm := 'before stateyear';
if (v_rt_all_cols(i).crr_year is not null) then
insert into child_tab
(COM_ID,
KEY,
VALUE)
values
(id_val,
--'IB.Correspondence.Date.Year',
'IB.Crsp.Date.Year',
v_rt_all_cols(i).crr_year);
end if;
vstep_no := '7';
vtable_nm := 'before type';
if (v_rt_all_cols(i).product_type is not null) then
insert into child_tab
(COM_ID,
KEY,
VALUE)
values
(id_val,
'IB.Product.ProductName',
v_rt_all_cols(i).product_type);
end if;
vstep_no := '9';
vtable_nm := 'before title';
if (trim(v_rt_all_cols(i).title) is not null) then
insert into child_tab
(COM_ID,
KEY,
VALUE )
values
(id_val,
'IB.Customer.Title',
trim(v_rt_all_cols(i).title));
end if;
vstep_no := '10';
vtable_nm := 'before surname';
if (v_rt_all_cols(i).surname is not null) then
insert into child_tab
(COM_ID,
KEY,
VALUE)
values
(id_val,
'IB.Customer.LastName',
v_rt_all_cols(i).surname);
end if;
vstep_no := '12';
vtable_nm := 'before postcd';
if (trim(v_rt_all_cols(i).POSTCODE) is not null) then
insert into child_tab
(COM_ID,
KEY,
VALUE)
values
(id_val,
'IB.Customer.Addr.PostCodeMasked',
substr(replace(v_rt_all_cols(i).POSTCODE,' ',''),length(replace(v_rt_all_cols(i).POSTCODE,' ',''))-2,3));
end if;
vstep_no := '13';
vtable_nm := 'before subject';
if (trim(v_rt_all_cols(i).SUBJECT) is not null) then
insert into child_tab
(COM_ID,
KEY,
VALUE)
values
(id_val,
'IB.Correspondence.Subject',
v_rt_all_cols(i).subject);
end if;
vstep_no := '14';
vtable_nm := 'before inactivity';
if (trim(v_rt_all_cols(i).UNREAD_CORRES_PERIOD) is null or
trim(v_rt_all_cols(i).UNREAD_CORRES_PERIOD) = '3' or
trim(v_rt_all_cols(i).UNREAD_CORRES_PERIOD) = '6' or
trim(v_rt_all_cols(i).UNREAD_CORRES_PERIOD) = '9') then
insert into child_tab
(COM_ID,
KEY,
VALUE)
values
(id_val,
'IB.Correspondence.Inactivity',
v_rt_all_cols(i).UNREAD_CORRES_PERIOD);
end if;
vstep_no := '14.1';
vtable_nm := 'after notfound';
end if;
vstep_no := '15';
vtable_nm := 'after notfound';
END LOOP;
end loop;
vstep_no := '16';
vtable_nm := 'before closecur';
CLOSE stg_cursor;
vstep_no := '17';
vtable_nm := 'before commit';
DELETE FROM table_stg;
COMMIT;
vstep_no := '18';
vtable_nm := 'after commit';
EXCEPTION
WHEN OTHERS THEN
ROLLBACK;
success_flag := 1;
vsql_code := SQLCODE;
vsql_errm := SUBSTR(sqlerrm,1,200);
error_logging_pkg.inserterrorlog('samp',vsql_code,vsql_errm, vtable_nm,vstep_no);
RAISE_APPLICATION_ERROR (-20011, 'samp '||vstep_no||' SQLERRM:'||SQLERRM);
end;
ThanksIts bit urgent
NO - it is NOT urgent. Not to us.
If you have an urgent problem you need to hire a consultant.
I have a performance issue in the below code,
Maybe you do and maybe you don't. How are we to really know? You haven't posted ANYTHING indicating that a performance issue exists. Please read the FAQ for how to post a tuning request and the info you need to provide. First and foremost you have to post SOMETHING that actually shows that a performance issue exists. Troubleshooting requires FACTS not just a subjective opinion.
where i am trying to insert the data from table_stg into target_tab and in parent_tab tables and then to child tables via cursor with bulk collect .the target_tab and parent_tab are huge tables and have a row wise trigger enabled on it .the trigger is mandatory . This timetaken for this block to execute is 5000 seconds.Now my requirement is to reduce it to 5 to 10 mins.
Personally I think 5000 seconds (about 1 hr 20 minutes) is very fast for processing 800 trillion rows of data into parent and child tables. Why do you think that is slow?
Your code has several major flaws that need to be corrected before you can even determine what, if anything, needs to be tuned.
This code has the EXIT statement at the beginning of the loop instead of at the end
FETCH stg_cursor BULK COLLECT INTO v_rt_all_cols LIMIT limit_in;
vstep_no := '4';
vtable_nm := 'after fetch';
--EXIT WHEN v_rt_all_cols.COUNT = 0;
EXIT WHEN stg_cursor%NOTFOUND;
The correct place for the %NOTFOUND test when using BULK COLLECT is at the END of the loop; that is, the last statement in the loop.
You can use a COUNT test at the start of the loop but ironically you have commented it out and have now done it wrong. Either move the NOTFOUND test to the end of the loop or remove it and uncomment the COUNT test.
WHEN OTHERS THEN
ROLLBACK;
That basically says you don't even care what problem occurs or whether the problem is for a single record of your 10,000 in the collection. You pretty much just throw away any stack trace and substitute your own message.
Your code also has NO exception handling for any of the individual steps or blocks of code.
The code you posted also begs the question of why you are using NAME=VALUE pairs for child data rows? Why aren't you using a standard relational table for this data?
As others have noted you are using slow-by-slow (row by row processing). Let's assume that PL/SQL, the bulk collect and row-by-row is actually necessary.
Then you should be constructing the parent and child records into collections and then inserting them in BULK using FORALL.
1. Create a collection for the new parent rows
2. Create a collection for the new child rows
3. For each set of LIMIT source row data
a. empty the parent and child collections
b. populate those collections with new parent/child data
c. bulk insert the parent collection into the parent table
d. bulk insert the child collection into the child table
And unless you really want to either load EVERYTHING or abandon everything you should use bulk exception handling so that the clean data gets processed and only the dirty data gets rejected. -
How to use BULK INSERT for a data from a cursor?
Oracle 10G enterprise edition.
I tried to Bulk insert datas returning from a cursor, its returning error.
PLS-00302: component 'LAST' must be declared
I need some help to use the Bulk INSERT here.Can any one help me to specify what error i have made?
CREATE OR REPLACE PROCEDURE HOT_ADMIN.get_search_keyword_stats_prc
IS
CURSOR c_get_scenarios
IS
SELECT a.*,ROWNUM rnum
FROM (
SELECT TRUNC(r.search_date) sdate,
r.search_hits hits,
r.search_type stype,
r.search_qualification qual,
r.search_location loc,
r.search_town stown,
r.search_postcode pcode,
r.search_college college,
r.search_colname colname,
r.search_text text,
r.affiliate_id affiliate,
r.search_study_mode smode,
r.location_hint hint,
r.search_posttown ptown,
COUNT(1) cnt
FROM w_search_headers r
WHERE search_text IS NOT NULL
AND NVL(search_type,' ') <> 'C'
AND TRUNC(search_date)= TO_DATE(TO_CHAR(SYSDATE-1,'DD-MON-RRRR'))
GROUP BY TRUNC(r.search_date),
r.search_hits,
r.search_type,
r.search_qualification,
r.search_location,
r.search_town,
r.search_postcode,
r.search_college,
r.search_colname,
r.search_text,
r.affiliate_id,
r.search_study_mode,
r.location_hint,
r.search_posttown
ORDER BY cnt desc
) a
WHERE ROWNUM <=1000;
lc_get_data c_get_scenarios%ROWTYPE;
BEGIN
OPEN c_get_scenarios;
FETCH c_get_scenarios into lc_get_data;
CLOSE c_get_scenarios;
FORALL i IN 1..lc_get_data.last
INSERT INTO W_SEARCH_SCENARIO_STATS VALUES ( i.sdate,
i.hits,
i.stype,
i.qual,
i.loc,
i.stown,
i.pcode,
i.college,
i.colname,
i.text,
i.affiliate,
i.smode,
i.hint,
i.ptown,
i.cnt
COMMIT;
END;This isn't what you asked, but I've generally found it helpful to list the columns in an INSERT statement before the values. It is of course optional, but useful for reference when looking at the statement later
-
Bulk Load question for an insert statement.
I'm looking to put the following statement into a FORALL statement using BULK COLLLECT and I need some guidance.
Am I going to be putting the SELECT statement into a cursor and then load the cursor values into a defined Nested Table type defined variable?
INSERT INTO TEMP_ASSOC_CURRENT_WEEK_IDS
SELECT aor.associate_office_record_id ,
sched.get_assoc_sched_rotation_week(aor.associate_office_record_id, v_weekType.start_date) week_id
FROM ASSOCIATE_OFFICE_RECORDS aor
WHERE aor.OFFICE_ID = v_office_id
AND (
(aor.lt_assoc_stage_result_id in (4,8)
AND v_officeWeekType.start_date >= trunc(aor.schedule_start_date)
OR aor.lt_assoc_stage_result_id in (1, 2)
));I see people are reading this so for the insanely curious here's how I did it.
Type AOR_REC is RECORD(
associate_office_record_id dbms_sql.number_table,
week_id dbms_sql.number_table); --RJS.***Setting up Type for use with Bulk Collect FORALL statements.
v_a_rec AOR_REC; -- RJS. *** defining variable of defined Type to use with Bulk Collect FORALL statements.
CURSOR cur_aor_ids -- RJS *** Cursor for BULK COLLECT.
IS
SELECT aor.associate_office_record_id associate_office_record_id,
sched.get_assoc_sched_rotation_week(aor.associate_office_record_id, v_weekType.start_date) week_id
FROM ASSOCIATE_OFFICE_RECORDS aor
WHERE aor.OFFICE_ID = v_office_id
AND (
(aor.lt_assoc_stage_result_id in (4,8)
AND v_officeWeekType.start_date >= trunc(aor.schedule_start_date)
OR aor.lt_assoc_stage_result_id in (1, 2)
FOR UPDATE NOWAIT;
BEGIN
BEGIN
OPEN cur_aor_ids;
LOOP
FETCH cur_aor_ids BULK COLLECT into
v_a_rec.associate_office_record_id, v_a_rec.week_id; --RJS. *** Bulk Load your cursor data into a buffer to do the Delete all at once.
FORALL i IN 1..v_a_rec.associate_office_record_id.COUNT SAVE EXCEPTIONS
INSERT INTO TEMP_ASSOC_CURRENT_WEEK_IDS
(associate_office_record_id,week_id)
VALUES
(v_a_rec.associate_office_record_id(i), v_a_rec.week_id(i)); --RJS. *** Single FORALL BULK DELETE statement.
EXIT WHEN cur_aor_ids%NOTFOUND;
END LOOP;
CLOSE cur_aor_ids;
EXCEPTION
WHEN OTHERS THEN
dbms_output.put_line('ERROR ENCOUNTERED IS SQLCODE = '|| SQLCODE ||' AND SQLERRM = ' || SQLERRM);
dbms_output.put_line('Number of INSERT statements that
failed: ' || SQL%BULK_EXCEPTIONS.COUNT);
End;
Easy right? -
Hi All,
I have a cursor in PL/SQL block and it selects the data for about 100 columns, my requirement is to insert the data of about 60 columns into one table and rest of the columns into another. Please let me know how do I implement the same by using BULK INSERT.
Thanks for your help in advance.Why not dispense with the CURSOR and instead use a multi-table INSERT...SELECT?
INSERT ALL
INTO table1 (col1, col2, ..., col60)
VALUES (col1, col2, ..., col60)
INTO table2 (col61,...col100)
VALUES (col61,...col100)
SELECT col1,..., col100
FROM etc.where the SELECT is doing the same query as the CURSOR.
Edited by: user142857 on Nov 12, 2009 10:06 AM -
Governer limit for inserting a Custom section in detail page of SOD
HI,
I want to insert multiple custom section in detail page of Siebel on demand objects.
What is the governer limit for insertion of the Custom sections?
Thanks
YashHi Sean,
I am sorry for your confusion. Yes, your understanding is correct.
I tried to add manually and i got the result as 'There can be at most 25 custom web applets in a detail page'. Is this correct?
Thanks
Yash -
How to get current month from filename and bulk insert from text file into table?
I set up some dynamic SQL to help my bulk copy data from a text file to a table. This works fine for files that come in every day; I get the previous day’s data, based on the file name that’s placed
in the folder. That’s why I’m using the ‘-1’. The dates will look like this: '20140131', so I'm using type 112.
declare @fullpath1 varchar(1000)
select @fullpath1 = '''\\system.local\ms\london\FTP\' + convert(varchar, getdate()-1, 112) + '_INDEXPRICES_EOM.SPC'''
declare @cmd1 nvarchar(1000)
print (@cmd1)
select @cmd1 = 'bulk insert [dbo].[SB_Monthly] from ' + @fullpath1 + ' with (FIELDTERMINATOR = ''\t'', FIRSTROW = 5, LASTROW = 675, ROWTERMINATOR=''0x0a'')'
print(@cmd1)
exec (@cmd1)
I think the syntax will be somewhat similar to this:
YEAR(date_column)=YEAR(getdate()) AND MONTH(date_column)=MONTH(getdate())
I’m not totally sure how to incorporate that into my current syntax.
Knowledge is the only thing that I can give you, and still retain, and we are both better off for it.I tried a couple versions of this.
Declare @StartDate Date, @EndDate Date
Select @StartDate = convert(varchar, getdate()-28, 112), @EndDate = convert(varchar, getdate()-1, 112)
BEGIN
declare @fullpath1 varchar(1000)
select @fullpath1 = '''\\ms\london\FTP\' + ''' between ''' + Convert(Varchar(10), @StartDate, 101) + ''' and ''' + Convert(Varchar(10), @EndDate, 101) + '''_SP.SPC'''
declare @cmd1 nvarchar(1000)
print (@cmd1)
select @cmd1 = 'bulk insert [dbo].[SPBMI_Monthly] from ' + @fullpath1 + ' with (FIELDTERMINATOR = ''\t'', FIRSTROW = 5, LASTROW = 675, ROWTERMINATOR=''0x0a'')'
print(@cmd1)
exec (@cmd1)
END
Here’s the string:
bulk insert [dbo].[SPBMI_Monthly] from '\\ms\london\FTP\' between '02/03/2014' and '03/02/2014'_SP.SPC' with (FIELDTERMINATOR = '\t', FIRSTROW = 5, LASTROW = 675, ROWTERMINATOR='0x0a')
The error message I keep getting is:
Msg 156, Level 15, State 1, Line 1
Incorrect syntax near the keyword 'between'.
Msg 319, Level 15, State 1, Line 1
Incorrect syntax near the keyword 'with'. If this statement is a common table expression, an xmlnamespaces clause or a change tracking context clause, the previous statement must be terminated with a semicolon.
I feel like I’m already pushing this thing to the limit.
Maybe this last part isn’t possible.
Knowledge is the only thing that I can give you, and still retain, and we are both better off for it. -
I have this code which doesn't cause any error, and actually gives message 'query executed successfully', but it doesn't load any data.
bulk insert [dbo].[SPGT]
from '\\sys.local\london-sql\FTP\20140210_SPGT.SPL'
WITH (
KEEPNULLS,
FIRSTROW=5,
FIELDTERMINATOR='\t',
ROWTERMINATOR='\n'
How can I debug the issue, or see what the script is REALLY doing? It's not doing what I think it's doing.
All permissions, rights, etc are setup correctly. I just run the code successfully with a .txt file. Maybe it has something to do with the extension...
Knowledge is the only thing that I can give you, and still retain, and we are both better off for it.Yes, here is the final solution (for the benefit of others who find this anytime in the future).
CREATE
TABLE [dbo].[ICM]
Date DATETIME,
Type VARCHAR(MAX),
Change VARCHAR(MAX),
SP_ID VARCHAR(MAX),
Sedol VARCHAR(MAX),
Cusip VARCHAR(MAX),
Issue_Name VARCHAR(MAX),
Cty VARCHAR(MAX),
PE VARCHAR(MAX),
Cap_Range VARCHAR(MAX),
GICS VARCHAR(MAX),
Curr VARCHAR(MAX),
Local_Price DECIMAL(19,8),
Index_Total_Shares DECIMAL(19,8),
IWF DECIMAL(19,8),
Index_Curr VARCHAR(MAX),
Float_MCAP DECIMAL(19,8),
Total_MCAP DECIMAL(19,8),
Daily_Price_Rtn DECIMAL(19,8),
Daily_Total_Rtn DECIMAL(19,8),
FX_Rate DECIMAL(19,8),
Growth_Weight DECIMAL(19,8),
Value_Weight DECIMAL(19,8),
Bloomberg_ID VARCHAR(MAX),
RIC VARCHAR(MAX),
Exchange_Ticker VARCHAR(MAX),
ISIN VARCHAR(MAX),
SSB_ID VARCHAR(MAX),
REIT_Flag VARCHAR(MAX),
Weight
DECIMAL(19,8),
Shares DECIMAL(19,8)
bulk
insert dbo.ICM
from
'C:\Documents and Settings\london\Desktop\ICM.txt'
WITH
FIRSTROW
= 2,
FIELDTERMINATOR
= ',',
ROWTERMINATOR
= '\n'
GO
This was a bit confusing at first, because I've never done it before, and also, I was getting all kinds of errors, which turned out to be numbers in string fields and strings in number fields. Basically, the data that was given to me was totally screwed
up. That compounded the problem exponentially. I finally got the correct data, and I'm all set now.
Thanks everyone!
Knowledge is the only thing that I can give you, and still retain, and we are both better off for it.
Maybe you are looking for
-
Cr XIR1 to create a custom function to change the timestamp of a field
Post Author: palm CA Forum: Crystal Reports Hi, I am trying to create a custom function to change the timezone of a field in the report depending on the timezone selected by the user in the prompt I wrote a SQL Expression to do this , but i ended up
-
Crystal Reports 2008 SP3 &VS Studio 2008
Hi, i'm trying to use Crystal Reports 2008 with SP3, Visual Studio 2008 and OS Windows Server 2003 (I use this Server OS because i'm developing for Microsoft SharePoint). Before upgrading to Crystal Reports SP3 i could not set a basic Visual Studio C
-
Can SCOM 2012 be placed into Windows Azure to monitor on premise services ?
We are wanting to relocate our SCOM management group into the cloud. The SCOM management group is used to monitor multiple forests in multiple locations using SCOM gateways. Firstly is this supported ? has anyone in the community experience of run
-
Function Module to send output list to mail.
Hi, Could anyone let me know the function module to send output list to mail. Regards, Ramesh
-
Hi, The export log of my database seems to be weird. Following is the excerpt from the export log: . . exporting table FLOATING_INTEREST 4 rows exported . . exporting table FLOATING_RATE 0 rows exported . . exporting table FORM_SETTINGS 0 rows export