Loading data with data workshop

Hello,
We load lots of .csv files into apex using the Dataworkshop tool in Apex. Most .csv files are 60-75K rows of data. It used to take just minutes and was quick and easy way to load data. However, recently it has started taking much much longer (as in several hours vs. 5 or 6 minutes normally) to load data this way into production.
Any ideas of why all of a sudden it takes much longer to load data this way? Is there anything that can be tuned?
We are running Application Express 4.1.1.00.23 on RH 5 linux with 11.2.0.2 database.
Thank you for any help you can provide.
Thank you,
Mark

Mehabub,
Thank you for responding. In a sense that is kind of what we are doing. We truncate the table and upload our .csv to this table and then process into our other tables.
The table does not have a particularly large number of rows (usually 60,000 - 70,000 rows), but does have 112 columns.
It all seems to work fine in our test environment, but there are not the number of users there as in production. We have been looking at the SGA parameters and seem to make headway, but then the customer tries to upload and it slows way way down in production.
Our customers really like this functionality we have given them with apex, but I am running out of ideas of what to look at.
Thank you,
Mark

Similar Messages

Performance question when compare date with date or char with char

Hello from Germany (Frankfurt) !
I am working on Oracle 9.2.0.8.0
and have to solve following problem:
Comparison of a date and char fields.
Now two ways to do it.
Either I compare char with char and convert date to char,
or I compare date with date and convert char to date.
Now the performace question. Which operation takes more effort for the database?
So why not to try and to see the results?
First create table:
CREATE TABLE TEST (
char_date VARCHAR2(8),
real_date DATE
NOLOGGING;
Then insert 1.000.000 rows
BEGIN
for x in 1..1000000
loop
insert into test (char_date, real_date) VALUES('19990101', TO_DATE('2006.01.01', 'YYYY.MM.DD'));
end loop;
COMMIT;
END;
Collect statistics
EXEC dbms_stats.gather_table_stats('TESTER', 'TEST');
Now run some selects for date to char conversion:
Elapsed: 00:00:00.00
SQL> select * from test t where TO_DATE(char_date, 'YYYYMMDD') > real_date;
no rows selected
Elapsed: 00:00:03.02
SQL> select * from test t where TO_DATE(char_date, 'YYYYMMDD') > real_date;
no rows selected
And some selects for char to date conversion:
Elapsed: 00:00:03.02
SQL> select * from test t where char_date > TO_CHAR(real_date, 'YYYYMMDD');
no rows selected
Elapsed: 00:00:02.05
SQL> select * from test t where char_date > TO_CHAR(real_date, 'YYYYMMDD');
no rows selected
Elapsed: 00:00:02.05
SQL>
As you see when I compare char with char and convert date to char it seems to be faster (almost 1 second)
Is the test correct?
I still not sure, what gets better performance...
Any idea?
Thanks!

Depends on whether you want the right results or not.
Why don't you run the following two queries and see if the difference in results tells you anything?:
with t as (select to_date('01/02/2007', 'dd/mm/yyyy') date_col from dual
           union all
           select to_date('02/02/2007', 'dd/mm/yyyy') from dual
           union all
           select to_date('03/02/2007', 'dd/mm/yyyy') from dual
           union all
           select to_date('03/03/2006', 'dd/mm/yyyy') from dual)
select *
from   t
where date_col < to_date('04/03/2006', 'dd/mm/yyyy');
with t as (select to_date('01/02/2007', 'dd/mm/yyyy') date_col from dual
           union all
           select to_date('02/02/2007', 'dd/mm/yyyy') from dual
           union all
           select to_date('03/02/2007', 'dd/mm/yyyy') from dual
           union all
           select to_date('03/03/2006', 'dd/mm/yyyy') from dual)
select *
from   t
where to_char(date_col) < '04/03/2006';

Sql loader error with date format

Hi everyone,
I have table and have a data in one coulmn RECORDED_DATE like '20090224' and my client is asking me to load this coulmn data in 'yyyymmdd' format.I am strucked up with my ideas.I used to_date('20090124','yyyymmdd') in control file.but it is also not working Here it is my control file
LOAD DATA
INFILE 'C:\xxxx\SQLLDR\HE data\HE_data_Feb.txt'
BADFILE 'C:\xxxx\SQLLDR\HE data.bad'
DISCARDFILE 'C:\xxxx\SQLLDR\HE data.dsc'
INTO TABLE LSCCMGR.FASTPAY_HE_DATA
REPLACE
fields terminated by X'09'
TRAILING NULLCOLS
(RECORDED_DATE "TO_DATE(:RECORDED_DATE,'mm/dd/yyyy')"
AGENT_ID
MEASURE
TRANSACTIONS
FEES
If i excute like this i am getting the error like
Record 1: Rejected - Error on table LSCCMGR.FASTPAY_HE_DATA, column RECORDED_DATE.
ORA-01843: not a valid month
Getting for all records,what i need o change to get the RECORDED_DATE as dateformat.Ple any one help me in this issue to resolve
How can we perform this using sql loader pls let me know,Thanks in advance.
Sravan

Hi,
>>(RECORDED_DATE "TO_DATE(:RECORDED_DATE,'mm/dd/yyyy')"
*Change this line to*
(RECORDED_DATE "TO_DATE(:RECORDED_DATE,'yyyymmdd')"Regards,

SQL Loader Problem with Date Format

Dear all,
I am dealing with a problem in loading data with SQL Loader. The problem is in the date format.
More specifically, I created the following Control File:
file.ctl
LOAD DATA
INFILE 'D:\gbal\chatium.log'
APPEND INTO TABLE CHAT_SL
FIELDS TERMINATED BY WHITESPACE
TRAILING NULLCOLS
(SL1 DATE "Mon DD, YYYY HH:MI:SS FF3AM",
SL2 char,
SL3 DATE "Mon DD, YYYY HH:MI:SS FF3AM",
SL4 char,
SL5 char,
SL6 char,
SL7 char,
SL8 char,
SL9 char,
SL10 char,
SL11 char,
SL12 char,
SL13 char,
SL14 char,
SL15 char)
The data we want to load are in the following file:
Apr 29, 2007 12:05:49 AM 1060615 Apr 29, 2007 12:05:35 AM 306978537730 24026384 chatium.user.userinfo WAP 0
Apr 29, 2007 12:12:51 AM 1061251 Apr 29, 2007 12:12:27 AM 306978537730 24026384 chatium.channel.list WAP 0
Apr 29, 2007 12:12:51 AM 1061264 Apr 29, 2007 12:12:32 AM 306978537730 24026384 chatium.channel.listdetail WAP 0
Apr 29, 2007 12:13:51 AM 1061321 Apr 29, 2007 12:13:31 AM 306978537730 24026384 chatium.user.search WAP 0
Apr 29, 2007 12:13:51 AM 1061330 Apr 29, 2007 12:13:37 AM 306978537730 24026384 chatium.user.userinfo WAP 0
The error log file is the following:
SQL*Loader: Release 9.2.0.1.0 - Production on Mon Apr 30 11:29:16 2007
Copyright (c) 1982, 2002, Oracle Corporation. All rights reserved.
Control File: file.ctl
Data File: D:\gbal\chatium.log
Bad File: chatium.bad
Discard File: none specified
(Allow all discards)
Number to load: ALL
Number to skip: 0
Errors allowed: 50
Bind array: 64 rows, maximum of 256000 bytes
Continuation: none specified
Path used: Conventional
Table CHAT_SL, loaded from every logical record.
Insert option in effect for this table: APPEND
TRAILING NULLCOLS option in effect
Column Name Position Len Term Encl Datatype
SL1 FIRST * WHT DATE MonDD,YYYYHH:MI:SS
SL2 NEXT * WHT CHARACTER
SL3 NEXT * WHT CHARACTER
SL4 NEXT * WHT CHARACTER
SL5 NEXT * WHT CHARACTER
SL6 NEXT * WHT CHARACTER
SL7 NEXT * WHT CHARACTER
SL8 NEXT * WHT CHARACTER
SL9 NEXT * WHT CHARACTER
SL10 NEXT * WHT CHARACTER
SL11 NEXT * WHT CHARACTER
SL12 NEXT * WHT CHARACTER
SL13 NEXT * WHT CHARACTER
SL14 NEXT * WHT CHARACTER
SL15 NEXT * WHT CHARACTER
Record 1: Rejected - Error on table CHAT_SL, column SL1.
ORA-01840: input value not long enough for date format
Record 2: Rejected - Error on table CHAT_SL, column SL1.
ORA-01840: input value not long enough for date format
Record 3: Rejected - Error on table CHAT_SL, column SL1.
ORA-01840: input value not long enough for date format
Record 4: Rejected - Error on table CHAT_SL, column SL1.
ORA-01840: input value not long enough for date format
I wonder if you could help me.
Thank you very much in advance.
Giorgos Baliotis

SQL> select to_date('Apr 29, 2007 12:05:49 AM','Mon DD, YYYY HH:MI:SS FF3AM') from dual;
select to_date('Apr 29, 2007 12:05:49 AM','Mon DD, YYYY HH:MI:SS FF3AM') from dual
ERROR at line 1:
ORA-01821: date format not recognized
SQL> ed
Wrote file afiedt.buf
1* select to_date('Apr 29, 2007 12:05:49 AM','Mon DD, YYYY HH:MI:SS AM') from dual
SQL> /
TO_DATE(
29/04/07
SQL> Then, you defined blank space as separator, but there is spaces in your date inside your file. So, you should add double-quotes around the date field like below, and add optionally enclosed by '"' into your ctlfile.
"Apr 29, 2007 12:05:49 AM" 1060615 "Apr 29, 2007 12:05:35 AM" 306978537730 24026384 chatium.user.userinfo WAP 0
"Apr 29, 2007 12:12:51 AM" 1061251 "Apr 29, 2007 12:12:27 AM" 306978537730 24026384 chatium.channel.list WAP 0
"Apr 29, 2007 12:12:51 AM" 1061264 "Apr 29, 2007 12:12:32 AM" 306978537730 24026384 chatium.channel.listdetail WAP 0
"Apr 29, 2007 12:13:51 AM" 1061321 "Apr 29, 2007 12:13:31 AM" 306978537730 24026384 chatium.user.search WAP 0
"Apr 29, 2007 12:13:51 AM" 1061330 "Apr 29, 2007 12:13:37 AM" 306978537730 24026384 chatium.user.userinfo WAP 0Example :
http://download-uk.oracle.com/docs/cd/B19306_01/server.102/b14215/ldr_concepts.htm#sthref477
Nicolas.

Load Grid with data from System.Data.DataTable

Hi all.
A have a System.Data.DataTable e need load a grid with data from this DataTable.
How Load a Grid with data from System.Data.DataTable?

Hi Francisco,
actually i don't think that you can use a .NET datatable to fill a business one grid.
the business one has it's own datatable.
in c# it would look like this:
oForm.DataSources.DataTables.Add("MyDataTable");
((SAPbouiCOM.Grid)(oForm.Items.Item("grid").Specific)).DataTable = oForm.DataSources.DataTables.Item("MyDataTable");
oForm.DataSources.DataTables.Item(0).ExecuteQuery("SELECT Field FROM [dbname].dbo.TABLE");
((SAPbouiCOM.Grid)(oForm.Items.Item("grid").Specific)).CollapseLevel = 1;
((SAPbouiCOM.Grid)(oForm.Items.Item("grid").Specific)).AutoResizeColumns();
((SAPbouiCOM.Grid)(oForm.Items.Item("grid").Specific)).SelectionMode = SAPbouiCOM.BoMatrixSelect.ms_Single;
good night
lg David

Load rows with date fileds include time

Hi, I need to load rows with SQLLoader. The data has am field with date and time but using mask don4t load the time, only date. How do I load date and time?
Thanks...

The following should work to load both the date and time (as long as it is not rejected for some reason)date_cancel date "YYYY/MM/DD HH24:MI:SS"To display the date and time after it has been loaded, you will have to use on of the following:ALTER SESSION SET NLS_DATE_FORMAT = 'DD-MON-YYYY HH:MI:SS AM';
or
SELECT to_char(<column_name,'DD-MON-YYYY HH:MI:SS AM') from <table_name>;

Loading data with dates using SQL*Loader

Dear everyone
I am currently trying to load some data containing dates using SQL*Loader.
For termination of fields I have been using ^ because I have some book titles which contain " and ' as part of their title. I found that the TO_DATE function did not seem to work using ^ instead of ". Would I be correct? I think the Oracle manual says that " must be used.
After some Web research I eventually amended my control file to as follows:
load data
infile 'h:\insert_statements\22_insert_into_SCAN_FILE_INFO.txt'
REPLACE
into table SCAN_FILE_INFO
fields terminated by "," optionally enclosed by '^'
TRAILING NULLCOLS
(scan_id, scan_filename
file_format_id
orig_scanning_resolution_dpi
scanner_id, scanner_operator_id
scanning_date "TO_DATE (:scanning_date, 'YYYY-MM-DD')"
original_map_publication_id
reprint_publication_id)
A simple line of data is as follow:
280001, ^1910 - London^, 270001, 400, 250001, 260001, "TO_DATE('2007-06-06', 'YYYY-MM-DD')", 200019,
The final column being null.
However when I attempt that I get the following error message:
Record 1: Rejected - Error on table SCAN_FILE_INFO, column SCANNING_DATE.
ORA-01841: (full) year must be between -4713 and +9999, and not be 0
If I change the scanning_date part to:
scanning_date "EXPRESSION TO_DATE (:scanning_date, 'YYYY-MM-DD')",
or
scanning_date "CONSTANT TO_DATE (:scanning_date, 'YYYY-MM-DD')",
I get the error message:
Record 1: Rejected - Error on table SCAN_FILE_INFO, column SCANNING_DATE.
ORA-00917: missing comma
As soon as I do the following:
scanning_date "EXPRESSION, TO_DATE (:scanning_date, 'YYYY-MM-DD')",
or
scanning_date "CONSTANT, TO_DATE (:scanning_date, 'YYYY-MM-DD')",
I get too many values error message:
Record 1: Rejected - Error on table SCAN_FILE_INFO.
ORA-00913: too many values
I also tested out scanning_date DATE "YYYY-MM-DD", but that just gave the same ORA-01841 error message as above.
I must be doing something very simple which is wrong but I cannot figure it out.
Kind regards
Tim

And why do you have scanning date as "TO_DATE('2007-06-06', 'YYYY-MM-DD')" in your infile? All you need is 2007-06-06. If you can not change infile generation code, use:
load data
infile 'h:\insert_statements\22_insert_into_SCAN_FILE_INFO.txt'
REPLACE
into table SCAN_FILE_INFO
fields terminated by "," optionally enclosed by '^'
TRAILING NULLCOLS
(scan_id, scan_filename
file_format_id
orig_scanning_resolution_dpi
scanner_id, scanner_operator_id
scanning_date "TO_DATE(REPLACE(REPLACE(:scanning_date,'TO_DATE('),'''YYYY-MM-DD'')'), 'YYYY-MM-DD')"
original_map_publication_id
reprint_publication_id)SY.

Replacement of FORECAST DELIVERY DATE with DATE OF FIRST RECEIPT

Please help me in my scenario below and post your valuable comments-
My Scenario is like -
For example, In REPORT, For the item 10 of the order 4700035361 our FIRST RECEIPT DATE is in march (14/03/2011) and FORECAST DELIVERY DATE is in december (30/12/2011). This purchase order item was completely received.
Requirement is like---
Associated cube is ZCUIMP01.
(1) The evolution will consist of replace the FORECAST DELIVERY DATE with FIRST RECEIPT DATE of the item.
(2) The period (Fiscper and Fiscyear) will also be modified.
(3) Hence the calculation of the YTD will take into account the order and its receipt date.
(4) The adjustment of the history will be necessary for the year 2011.
Can anyone suggest, from where i should start my analysis (Should it be Infocube level??).
What will be the approach to fulfill all the above mentioned requirement (Should we think about Remodeling concept for creating new data model, if yes then how to do?).
Please reply asap, it will be great help for me.
My advance thanks to you.
Thanks & Regards
Rupali Singh

Hi Rupali,
As I understand there is a business change that you need to justify in our report. Unfortunately your explanation of the design has a considerable gap to visualize. In any case, if you need help on to how to approach such a change -
1. Understand the business change, its purpose and its imapct on your current design and the report.
2. Understanding the purpose is very important since you will then be able to find workarounds, redesign as per the change required.
3. Start from the query and check if your replacement of characterisitics with new characteristics can be obtained using functions like Replacement path, attributes, customer exits, etc. Changing a query/creating a new report on the existing model is the most feasible option.
4. If not, try exploring option to keep the design untouched, and creating a new InfoObject/DSO like structure to capture the date modifications, etc which then can be used on a MultiProvider.
Hope the above helps!
Regards,
Kunal Gandhi

How i map the caf data with data of UWL in web dynpro

Hi Experts,
i created a bpm project with nwce 7.1.1 and it have a independent caf to keep it's data and status.
now i have to create a view(use web dynpro) such like UWL,
my question is
how i map the caf data with UWL, and let the view can open the task(popup a window, just like UWL do) for user to finish their job?
thanks.

Hi Vic
Your requirement is exect functionality of UWL based on WEBDYNPRO, CAF and BPM for automation? ,and question is How to use CAF with WD or CAF with UWL API's ?. please clarify it.
Please fo through from given doc misght give u some idea
1. [How to develop Web Dynpro UI for your CAF project |http://www.sdn.sap.com/irj/scn/weblogs;jsessionid=(J2EE3417300)ID0722080650DB02400261803144436507End?blog=/pub/wlg/5414]
Best Regards
Satish Kumar

Filtering data with date birth

Hi ,
i added a field(date of birth) in Candidate search page in irecruitment through Additional Search criteria DFF..
This search filtering the data whatever date of birth matching record .But i need candidates having date of birth on or before my entered value .
How do i get this ?
Is it possible througd personalization ?
pl guide me ..
regards,
kumar
Message was edited by:
kumarduraipandi

Can you check if the page uses a query region, if yes then you can enable advanced search programmatically by setting a property on the queryBean. If it is a manual search region then you can add the items through personalization and write code in the controller and AM by extending them. You can do the same with query region simple search panel as well.

Query For Retrieving Data With Date

Hi Guys
I am having a hard time to figure out the sql query.
I have a table with data. Below is the illustration.
date location
item
1 jan 14 A
apple
2 jan 14 A
apple
3 jan 14 B
apple
4 jan 14 B
apple
1 jan 14 A
orange
2 jan 14 A
orange
3 jan 14 B
orange
4 jan 14 B
orange
My question it how to get the latest date on each location and item
This is the result I want
date location
item
2 jan 14 A
apple
4 jan 14 B
apple
2 jan 14 A
orange
4 jan 14 B
orange
Thanks.

Try the below:
create table #temp (sdate datetime,location char(1),Item varchar(20))
insert into #temp values('1 jan 14','A','Apple')
insert into #temp values('2 jan 14','A','Apple')
insert into #temp values('3 jan 14','B','Apple')
insert into #temp values('4 jan 14','B','Apple')
insert into #temp values('1 jan 14','A','Orange')
insert into #temp values('2 jan 14','A','Orange')
insert into #temp values('3 jan 14','B','Orange')
insert into #temp values('4 jan 14','B','Orange')
;With cte as
Select * , Row_Number()Over(partition by location, item order by sdate desc) Rn
From #Temp
Select * From cte Where rn=1 Order by Item asc,location asc
drop table #temp

Query For Retrieving Data With Date and grouping

Hi Guys
I am having a hard time to figure out the sql query.
I have a table with data. Below is the illustration.
date location item description
1 jan 14 A apple desc1
2 jan 14 A apple desc2
3 jan 14 B apple desc1
4 jan 14 B apple desc2
1 jan 14 A orange desc1
2 jan 14 A orange desc2
3 jan 14 B orange desc1
4 jan 14 B orange desc2
My question it how to get the latest date on each location and item along with the description field
This is the result I want
date location item description
2 jan 14 A apple desc2
4 jan 14 B apple desc2
2 jan 14 A orange desc2
4 jan 14 B orange desc2
Thanks.

provided it a datetime/date field you've for date you can do this
SELECT [date],location,itemdescription
FROM
SELECT [date],location,itemdescription,
ROW_NUMBER() OVER (PARTITION BY location ORDER BY [date] DESC) AS Rn
FROM table
)t
WHERE Rn = 1
Please Mark This As Answer if it helps to solve the issue Visakh ---------------------------- http://visakhm.blogspot.com/ https://www.facebook.com/VmBlogs

How to repeat data with Data Merge?

How can I repeat data in the same text block with Data Merge? I have an Excel file with a class schedule that I need to import but when I use Data Merge, it wants to create a new page for each line. I want to keep the data in one text block per page and use Data Merge so when I get the new Excel file, it will be easy to update. This seems like it would be a simple thing to do, but I can't find the answer. Can anyone help?

Time for me to sound like a broken record. Data merge could be used for this but I don't think it was ever intended for such recurring publications like you are making. There will always be a lot of repetitive work that always will need to be performed after each update.
Instead, I think you should read up on using an XML workflow. Now, the XML out of excel needs transformed to be viable inside ID. But that is reasonably easy and can be done in an XML editor or upon importing into ID. Though it has been set aside for a time, I am making an example of a college course catalog for a customer. I'll see if I can finish it off by the weekend. If so, I'll post it to this thread.
Take care, Mike

Incorrect date with date from Windows ics Files since installing OS X Mountain

since installing OS X mounting the introducing fo dates with windows ics files is 5 month earlier. With iPhone no problem.

I'm having the same problem on my late 2008 MBP 15". Never had a problem before installing Mountain Lion.   Now I have to reboot about 4-5 times a week.
My computer doesn't lock up completely.    But Wi-Fi stops (you see only one black line on the WiFi icon), the clock stops (I was wondering why it seemed so early earlier tonight - the clock had been stopped for two hours) and many, but not all applications are "not responding".    However, if I try to quit the applications that appear to be okay, then they also become "not responding".   I have to force shut each application and then I can reboot using the Apple>Restart... command.
One of the things I've noticed is that many times it's iTunes that has the "not responding" message when the other apps don't, even if I haven't used it.   I'm going to Quit iTunes and see if it still happens.
I don't think it's heat related. My computer feels no hotter than it ever did.
If 10.8.1 fixes this, I'm okay with it (although it was still sloppy for Apple to release with such a bug.)    But the question is whether Apple even knows about it, since they provide no mechanism for non-developers to report bugs.

Loading master data with date-fields

Hello,
we are loading with the extractor 0crm_mktelm5_attr data to the data target 0crm_mktelm (infoobject).
The fields "from" and "to" are initial in psa. But the Update in the master data abort with the failure:
"Invalid "to" date '00000000 ' " and "Invalid "to" date '00000000 ' ".
The data-fields in psa are initial, and in other BW-Systems, we have not problems with the extractor 0crm_mktelm5_attr and initial date-fields.
Have any body an idea??
Thanks
Murat

Hello Shashank,
thanks,
the Infosource.mapping was false.
regards
Murat

Loading data with data workshop

Similar Messages

Maybe you are looking for