Performance problem during commit
hi,
i register 5000 objects in a UnitOfWork.
only a small number of objects changed.
when commit the UnitOfWork i'm waiting
over 10 seconds before the commit end.
what can i do to increase the commit performance
thanks
Hello,
A rule of thumb (so to speak) when using TopLink's UOW is to only register those objects that are needed in the UOW. In your case, only register those objects that will change.
When the UOW commits, it has to determine for all 5000 objects what has changed. 10 seconds is not unreasonable for this large set of data.
Darren
Similar Messages
-
Database migration to MAXDB and Performance problem during R3load import
Hi All Experts,
We want to migrate our SAP landscape from oracle to MAXDB(SAPDB). we have exported database of size 1.2 TB by using package and table level splitting method in 16 hrs.
Now I am importing into MAXDB. But the import is running very slow (more than 72 hrs).
Details of import process as per below.
We have been using distribution monitor to import in to target system with maxdb database 7.7 release. We are using three parallel application servers to import and with distributed R3load processes on each application servers with 8 CPU.
Database System is configured with 8CPU(single core) and 32 GB physical RAM. MAXDB Cache size for DB instance is allocated with 24GB. As per SAP recommendation We are running R3load process with parallel 16 CPU processes. Still import is going too slow with more that 72 hrs. (Not acceptable).
We have split 12 big tables in to small units using table splitting , also we have split packages in small to run in parallel. We maintained load order in descending order of table and package size. still we are not able to improve import performance.
MAXDB parameters are set as per below.
CACHE_SIZE 3407872
MAXUSERTASKS 60
MAXCPU 8
MAXLOCKS 300000
CAT_CACHE_SUPPLY 262144
MaxTempFilesPerIndexCreation 131072
We are using all required SAP kernel utilities with recent release during this process. i.e. R3load ,etc
So Now I request all SAP as well as MAXDB experts to suggest all possible inputs to improve the R3load import performance on MAXDB database.
Every input will be highly appreciated.
Please let me know if I need to provide more details about import.
Regards
SantoshHello,
description of parameter:
MaxTempFilesPerIndexCreation(from version 7.7.0.3)
Number of temporary result files in the case of parallel indexing
The database system indexes large tables using multiple server tasks. These server tasks write their results to temporary files. When the number of these files reaches the value of this parameter, the database system has to merge the files before it can generate the actual index. This results in a decline in performance.
as for max value, I wouldn't exceed the max valuem for 26G value 131072 should be sufficient. I used same value for 36G CACHE SIZE
On the other side, do you know which task is time consuming? is it table import? index creation?
maybe you can run migtime on import directory to find out
Stanislav -
Performance problem while CPU is 80% Idel ?
Hi,
My end users are claim for performance problem during execution of batch process.
As you can see there are 1,745 statement executing each second.
Awr report shows 98.1% of the time , waits on CPU .
Also Awr report shows that Host CPU is :79.9% Idel.
The second wait event shows only 212 seconds waits on db file sequential read.
Yet , 4 minute in 1 hour period is seems not an issue.
Please advise
DB Name DB Id Instance Inst Num Startup Time Release RAC
QERP xxx erp 1 21-Jan-13 15:40 11.2.0.2.0 ; NO
Host Name Platform CPUs Cores Sockets Memory(GB)
erptst HP-UX IA (64-bit) 16 16 4 127.83
Snap Id Snap Time Sessions Curs/Sess
Begin Snap: 40066 22-Jan-13 20:00:52 207 9.6
End Snap: 40067 22-Jan-13 21:00:05 210 9.6
Elapsed: 59.21 (mins)
DB Time: 189.24 (mins)
Cache Sizes Begin End
~~~~~~~~~~~ ---------- ----------
Buffer Cache: 8,800M 8,800M Std Block Size: 8K
Shared Pool Size: 1,056M 1,056M Log Buffer: 49,344K
Load Profile Per Second Per Transaction Per Exec Per Call
~~~~~~~~~~~~ --------------- --------------- ---------- ----------
DB Time(s): 3.2 ; 0.1 ; 0.00 ; 0.05
DB CPU(s): 3.1 ; 0.1 ; 0.00 ; 0.05
Redo size: 604,285.1 ; 27,271.3
Logical reads: 364,792.3 ; 16,463.0
Block changes: 3,629.5 ; 163.8
Physical reads: 21.5 ; 1.0
Physical writes: 95.3 ; 4.3
User calls: 68.7 ; 3.1
Parses: 212.9 ; 9.6
Hard parses: 0.3 ; 0.0
W/A MB processed: 1.2 ; 0.1
Logons: 0.3 ; 0.0
Executes: 1,745.2 ; 78.8
Rollbacks: 1.2 ; 0.1
Transactions: 22.2
Instance Efficiency Percentages (Target 100%)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Buffer Nowait %: 100.00 ; Redo NoWait %: 100.00
Buffer Hit %: 99.99 ; In-memory Sort %: 100.00
Library Hit %: 99.95 ; Soft Parse %: 99.85
Execute to Parse %: 87.80 ; Latch Hit %: 99.99
Parse CPU to Parse Elapsd %: 74.76 ; % Non-Parse CPU: 99.89
Shared Pool Statistics Begin End
Memory Usage %: 75.37 ; 76.85
% SQL with executions>1: 95.31 ; 85.98
% Memory for SQL w/exec>1: 90.33 ; 82.84
Top 5 Timed Foreground Events
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Avg
wait % DB
Event Waits Time(s) (ms) time Wait Class
DB CPU 11,144 98.1
db file sequential read 52,714 214 4 1.9 User I/O
SQL*Net break/reset to client 29,050 6 0 .1 Applicatio
log file sync 2,536 6 2 .0 Commit
buffer busy waits 4,338 2 1 .0 Concurrenc
Host CPU (CPUs: 16 Cores: 16 Sockets: 4)
~~~~~~~~ Load Average
Begin End %User %System %WIO %Idle
0.34 ; 0.33 ; 19.7 ; 0.4 ; 1.8 ; 79.9Nikolay Savvinov wrote:
if the users are complaining about performance of the batch process, then that's what you should be looking at, not the entire system.I find it strange to see "end users" and "the batch process" in the same sentence (as it was in the first post). "End users" gives me the feeling of a significant number of concurrent sessions with people waiting for results in real time at the far end, while "batch process" carries the image a small number of large scale processes running overnight to prepare the data for the following morning.
I mention this because my first view of the AWR output was: you've got 16 CPUs, only three in use, virtually no users, and doing very little work, how can the users complain. (One answer, of course, is that the 13 CPUs could be locked out of use as far as Oracle is concerned). On the second read I decided that the "users" had gone home, and the complaint was simply that the batch process wasn't completing in time.
In this case I think "the entire system" IS "the batch process"
Determine which stored procedures and/or SQL statements took longer than usual and then find out why. Most likely you'll be able to find
everything you need in AWR views (DBA_HIST_SQL%) and ASH archive (DBA_HIST_ACTIVE_SESS_HISTORY).
If the batch process has changed dramatically and recently, then a simple first step might be to look at the current AWR report, find the few most time-consuming SQL statements, and use the awrsqrpt.sql script to find their history of execution plans.
But I'd also just look at the expensive SQL - bearing in mind, particularly, that there are very few user calls per second, yet many hundred executions per second: it strikes me that there could be quite a lot of PL/SQL going on doing something a little bit expensive many times or some PL/SQL function that calls some SQL that used to be called rarely from an SQL statement but is now (due, perhaps to a change in plan) being called much more frequently - so check SQL Ordered by Executions.
Regards
Jonathan Lewis -
Critical performance problem upon bulk load of groups
All (including product development),
I think there are missing indexes in wwsec_flat$ and wwsec_sys_priv$. Anyway, I'd like assistance on fixing the critical performance problems I see, properly. Read on...
During and after bulk load of a few (about 500) users and groups from an external database, it becomes evident that there's a performance problem somewhere. Many of the calls to wwsec_api.addGroupToList took several minutes to finish. Afterwards the machine went 100% CPU just from logging in with the portal30 user (which happens to be group owner for all the groups).
Running SQL trace points in the directions of the following SQL statement:
SELECT ID,PARENT_ID,NAME,TITLE_ID,TITLEIMAGE_ID,ROLLOVERIMAGE_ID,
DESCRIPTION_ID,LAYOUT_ID,STYLE_ID,PAGE_TYPE,CREATED_BY,CREATED_ON,
LAST_MODIFIED_BY,LAST_MODIFIED_ON,PUBLISHED_ON,HAS_BANNER,HAS_FOOTER,
EXPOSURE,SHOW_CHILDREN,IS_PUBLIC,INHERIT_PRIV,IS_READY,EXECUTE_MODE,
CACHE_MODE,CACHE_EXPIRES,TEMPLATE FROM
WWPOB_PAGE$ WHERE ID = :b1
I checked the existing indexes, and see that the following ones are missing (I'm about to test with these, but have not yet done so):
CREATE UNIQUE INDEX "PORTAL30"."WWSEC_FLAT_IX_GROUP_ID"
ON "PORTAL30"."WWSEC_FLAT$"("GROUP_ID")
TABLESPACE "PORTAL" PCTFREE 10 INITRANS 2 MAXTRANS 255
STORAGE ( INITIAL 32K NEXT 160K MINEXTENTS 1 MAXEXTENTS 4096
PCTINCREASE 1 FREELISTS 1)
LOGGING
CREATE UNIQUE INDEX "PORTAL30"."WWSEC_FLAT_IX_PERSON_ID"
ON "PORTAL30"."WWSEC_FLAT$"("PERSON_ID")
TABLESPACE "PORTAL" PCTFREE 10 INITRANS 2 MAXTRANS 255
STORAGE ( INITIAL 32K NEXT 160K MINEXTENTS 1 MAXEXTENTS 4096
PCTINCREASE 1 FREELISTS 1)
LOGGING
CREATE UNIQUE INDEX "PORTAL30"."WWSEC_SYS_PRIV_IX_PATCH1"
ON "PORTAL30"."WWSEC_SYS_PRIV$"("OWNER", "GRANTEE_GROUP_ID",
"GRANTEE_TYPE", "OWNER", "NAME", "OBJECT_TYPE_NAME")
TABLESPACE "PORTAL" PCTFREE 10 INITRANS 2 MAXTRANS 255
STORAGE ( INITIAL 32K NEXT 80K MINEXTENTS 1 MAXEXTENTS 4096
PCTINCREASE 1 FREELISTS 1)
LOGGING
Note that when I deleted the newly inserted groups, the CPU consumption immediately went down from 100% to some 2-3%.
This behaviour has been observed on a Sun Solaris system, but I think it's the same on NT (I have observed it during the bulk load on my NT laptop, but so far have not had the time to test further.).
Also note: In the call to addGroupToList, I set owner to true for all groups.
Also note: During loading of the groups, I logged a few errors, all of the same type ("PORTAL30.WWSEC_API", line 2075), as follows:
Error: Problem calling addGroupToList for child group'Marketing' (8030), list 'NO_OSL_Usenet'(8017). Reason: java.sql.SQLException: ORA-06510: PL/SQL: unhandled user-defined exception ORA-06512: at "PORTAL30.WWSEC_API", line 2075
Please help. If you like, I may supply the tables and the java program that I use. It's fully reproducable.
Thanks,
Erik Hagen (you may call me on +47 90631013)
nullYES!
I have now tested with insertion of the missing indexes. It seems the call to addGroupToList takes just as long time as before, but the result is much better: WITH THE INDEXES DEFINED, THERE IS NO LONGER A PERFORMANCE PROBLEM!! The index definitions that I used are listed below (I added these to the ones that are there in Portal 3.0.8, but I guess some of those could have been deleted).
About the info at http://technet.oracle.com:89/ubb/Forum70/HTML/000894.html: Yes! Thanks! Very interesting, and I guess you found the cause for the error messages and maybe also for the performance problem during bulk load (I'll look into it as soon as possible anbd report what I find.).
Note: I have made a pretty foolproof and automated installation script (or actually, it's part of my Java program), that will let anybody interested recreate the problem. Mail your interest to [email protected].
============================================
CREATE INDEX "PORTAL30"."LDAP_WWSEC_PERS_IX1"
ON "PORTAL30"."WWSEC_PERSON$"("MANAGER")
TABLESPACE "PORTAL" PCTFREE 10 INITRANS 2 MAXTRANS 255
STORAGE ( INITIAL 32K NEXT 32K MINEXTENTS 1 MAXEXTENTS 4096
PCTINCREASE 1 FREELISTS 1)
LOGGING;
CREATE INDEX PORTAL30.LDAP_WWSEC_PERS_IX2
ON PORTAL30.WWSEC_PERSON$('ORGANIZATION')
TABLESPACE PORTAL PCTFREE 10 INITRANS 2 MAXTRANS 255
STORAGE ( INITIAL 32K NEXT 32K MINEXTENTS 1 MAXEXTENTS 4096
PCTINCREASE 1 FREELISTS 1)
LOGGING;
CREATE INDEX PORTAL30.LDAP_WWSEC_PERS_PK
ON PORTAL30.WWSEC_PERSON$('ID')
TABLESPACE PORTAL PCTFREE 10 INITRANS 2 MAXTRANS 255
STORAGE ( INITIAL 32K NEXT 32K MINEXTENTS 1 MAXEXTENTS 4096
PCTINCREASE 1 FREELISTS 1)
LOGGING;
CREATE INDEX PORTAL30.LDAP_WWSEC_PERS_UK
ON PORTAL30.WWSEC_PERSON$('USER_NAME')
TABLESPACE PORTAL PCTFREE 10 INITRANS 2 MAXTRANS 255
STORAGE ( INITIAL 32K NEXT 32K MINEXTENTS 1 MAXEXTENTS 4096
PCTINCREASE 1 FREELISTS 1)
LOGGING;
CREATE INDEX PORTAL30.LDAP_WWWSEC_FLAT_UK
ON PORTAL30.WWSEC_FLAT$("GROUP_ID", "PERSON_ID",
"SPONSORING_MEMBER_ID")
TABLESPACE PORTAL PCTFREE 10 INITRANS 2 MAXTRANS 255
STORAGE ( INITIAL 32K NEXT 256K MINEXTENTS 1 MAXEXTENTS 4096
PCTINCREASE 0 FREELISTS 1);
CREATE INDEX PORTAL30.LDAP_WWWSEC_FLAT_PK
ON PORTAL30.WWSEC_FLAT$("ID")
TABLESPACE PORTAL PCTFREE 10 INITRANS 2 MAXTRANS 255
STORAGE ( INITIAL 32K NEXT 256K MINEXTENTS 1 MAXEXTENTS 4096
PCTINCREASE 0 FREELISTS 1);
CREATE INDEX PORTAL30.LDAP_WWWSEC_FLAT_IX5
ON PORTAL30.WWSEC_FLAT$("GROUP_ID", "PERSON_ID")
TABLESPACE PORTAL PCTFREE 10 INITRANS 2 MAXTRANS 255
STORAGE ( INITIAL 32K NEXT 256K MINEXTENTS 1 MAXEXTENTS 4096
PCTINCREASE 0 FREELISTS 1);
CREATE INDEX PORTAL30.LDAP_WWWSEC_FLAT_IX4
ON PORTAL30.WWSEC_FLAT$("SPONSORING_MEMBER_ID")
TABLESPACE PORTAL PCTFREE 10 INITRANS 2 MAXTRANS 255
STORAGE ( INITIAL 32K NEXT 256K MINEXTENTS 1 MAXEXTENTS 4096
PCTINCREASE 0 FREELISTS 1);
CREATE INDEX PORTAL30.LDAP_WWWSEC_FLAT_IX3
ON PORTAL30.WWSEC_FLAT$("GROUP_ID")
TABLESPACE PORTAL PCTFREE 10 INITRANS 2 MAXTRANS 255
STORAGE ( INITIAL 32K NEXT 256K MINEXTENTS 1 MAXEXTENTS 4096
PCTINCREASE 0 FREELISTS 1);
CREATE INDEX PORTAL30.LDAP_WWWSEC_FLAT_IX2
ON PORTAL30.WWSEC_FLAT$("PERSON_ID")
TABLESPACE PORTAL PCTFREE 10 INITRANS 2 MAXTRANS 255
STORAGE ( INITIAL 32K NEXT 256K MINEXTENTS 1 MAXEXTENTS 4096
PCTINCREASE 0 FREELISTS 1);
CREATE INDEX "PORTAL30"."LDAP_WWSEC_SYSP_IX1"
ON "PORTAL30"."WWSEC_SYS_PRIV$"("GRANTEE_GROUP_ID")
TABLESPACE "PORTAL" PCTFREE 10 INITRANS 2 MAXTRANS 255
STORAGE ( INITIAL 32K NEXT 56K MINEXTENTS 1 MAXEXTENTS 4096
PCTINCREASE 1 FREELISTS 1)
LOGGING;
CREATE INDEX "PORTAL30"."LDAP_WWSEC_SYSP_IX2"
ON "PORTAL30"."WWSEC_SYS_PRIV$"("GRANTEE_USER_ID")
TABLESPACE "PORTAL" PCTFREE 10 INITRANS 2 MAXTRANS 255
STORAGE ( INITIAL 32K NEXT 56K MINEXTENTS 1 MAXEXTENTS 4096
PCTINCREASE 1 FREELISTS 1)
LOGGING;
CREATE INDEX "PORTAL30"."LDAP_WWSEC_SYSP_IX3"
ON "PORTAL30"."WWSEC_SYS_PRIV$"("OBJECT_TYPE_NAME", "NAME")
TABLESPACE "PORTAL" PCTFREE 10 INITRANS 2 MAXTRANS 255
STORAGE ( INITIAL 32K NEXT 56K MINEXTENTS 1 MAXEXTENTS 4096
PCTINCREASE 1 FREELISTS 1)
LOGGING;
CREATE INDEX "PORTAL30"."LDAP_WWSEC_SYSP_PK"
ON "PORTAL30"."WWSEC_SYS_PRIV$"("ID")
TABLESPACE "PORTAL" PCTFREE 10 INITRANS 2 MAXTRANS 255
STORAGE ( INITIAL 32K NEXT 56K MINEXTENTS 1 MAXEXTENTS 4096
PCTINCREASE 1 FREELISTS 1)
LOGGING;
CREATE INDEX "PORTAL30"."LDAP_WWSEC_SYSP_UK"
ON "PORTAL30"."WWSEC_SYS_PRIV$"("OBJECT_TYPE_NAME",
"NAME", "OWNER", "GRANTEE_TYPE", "GRANTEE_GROUP_ID",
"GRANTEE_USER_ID")
TABLESPACE "PORTAL" PCTFREE 10 INITRANS 2 MAXTRANS 255
STORAGE ( INITIAL 32K NEXT 88K MINEXTENTS 1 MAXEXTENTS 4096
PCTINCREASE 1 FREELISTS 1)
LOGGING;
==================================
Thanks,
Erik Hagen
null -
Performance problems with abap report SAPLPRGN_STRUCTURE during users login
Hello,
after patchimport of SAP BASIS level 21 and SAPABAP level 21 our SAP Production System (ECC6-UNICODE KERNEL 240) denote a performance problems. A lot of ours users during logon phase must wait until 10-12 minutes before that the navigation menu is displayed. The problem is generated by the slow performance of the standard SAP REPORT SAPLPRGN_STRUCTURE that require a lot time. Any idea?
ThanksHi Pat,
I would stop SAP and use DLTR3PKG, If this doesn't help, you need to investigate, if it is a CPU or DB-Time issue in ST03. Then you have to handle accordingly ...
Regards
Volker Gueldenpfennig, consolut international ag
http://www.consolut.net - http://www.4soi.de - http://www.easymarketplace.de -
COMMIT WORK - performance problem
Dear Fellow SDNers,
I seek your help on the following problem:
Scenario : Inbound idoc which updates an Outbound delivery with Picked quantity, posts the goods issue and then creates billing document
Approach : I am using the function module SD_DELIVERY_UPDATE_PICKING to update the delivery from the idoc data and to post goods issue. Thereafter, i use BAPI_BILLINGDOC_CREATEMULTIPLE to create the billing document. Before calling this BAPI, I use a COMMIT WORK statement to update the relevant tables so as to enable invoice creation properly.
Problem: The COMMIT WORK statement takes a lot of time to execute (I have no update tasks that could lead to this), so much that the idoc (probably) has a timeout and ends up in status 64. As a result, the succeeding part of the code (after COMMIT WORK) is not executed and the billing document is not created.
When I debug this, the COMMIT WORK statement leads to a strange screen (which looks like a blank report output screen, with its title as "UPDATE CONTROL". However (of course), there is no timeout while debugging and the billing document is successfully created.
Could anyone provide some pointers to solve this problem?
regards,
Priyanki have a custom function module Y_IDOC_INPUT_WMSPICK001 which is responsible for idoc inbound processing. SAP PI sends the inbound data to ECC and once this is done, this function module is executed.
This FM has the following code sequence inside it...
1) Call the FM SD_DELIVERY_UPDATE_PICKING
2) COMMIT WORK AND WAIT.
3) Call the BAPI_BILLINGDOC_CREATEMULTIPLE
Step1 is successfully executed, the step 2 takes a long time, and after that, the step 3 is not executed at all and the idoc ends up with a yellow light (status 64).
hope it clarifies what I am doing
regards,
Priyank -
Performance problems with XMLTABLE and XMLQUERY involving relational data
Hello-
Is anyone out there using XMLTABLE or XMLQUERY with more than a toy set of data? I am running into serious performance problems tyring to do basic things such as:
* Combine records in 10 relational tables into a single table of XMLTYPE records using XMLTABLE. This hangs indefinitely for any more than 800 records. Oracle has confirmed that this is a problem and is working on a fix.
* Combine a single XMLTYPE record with several relational code tables into a single XMLTYPE record using XMLQUERY and ora:view() to insert code descriptions after each code. Performance is 10 seconds for 10 records (terrible) passing a batch of records , or 160 seconds for one record (unacceptable!). How can it take 10 times longer to process 1/10th the number of records? Ironically, the query plan says it will do a full table scan of records for the batch, but an index access for the one record passed to the XMLQUERY.
I am rapidly losing faith in XML DB, and desparately need some hints on how to work around these performance problems, or at least some assurance that others have been able to get this thing to perform.<Note>Long post, sorry.</Note>
First, thanks for the responses above. I'm impressed with the quality of thought put into them. (Do the forum rules allow me to offer rewards? :) One suggestion in particular made a big performance improvement, and I’m encouraged to hear of good performance in pure XML situations. Unfortunately, I think there is a real performance challenge in two use cases that are pertinent to the XML+relational subject of this post and probably increasingly common as XML DB usage increases:
• Converting legacy tabular data into XML records; and
• Performing code table lookups for coded values in XML records.
There are three things I want to accomplish with this post:
• Clarify what we are trying to accomplish, which might expose completely different approaches than I have tried
• Let you know what I tried so far and the rationale for my approach to help expose flaws in my thinking and share what I have learned
• Highlight remaining performance issues in hopes that we can solve them
What we are trying to accomplish:
• Receive a monthly feed of 10,000 XML records (batched together in text files), each containing information about an employee, including elements that repeat for every year of service. We may need to process an annual feed of 1,000,000 XML records in the future.
• Receive a one-time feed of 500,000 employee records stored in about 10 relational tables, with a maximum join depth of 2 or 3. This is inherently a relational-to-XML process. One record/second is minimally acceptable, but 10 records/sec would be better.
• Consolidate a few records (from different providers) for each employee into a single record. Given the data volume, we need to achieve a minimum rate of 10 records per second. This may be an XML-only process, or XML+relational if code lookups are done during consolidation.
• Allow the records to be viewed and edited, with codes resolved into user-friendly descriptions. Since a user is sitting there, code lookups done when a record is viewed (vs. during consolidation) should not take more than 3 seconds total. We have about 20 code tables averaging a few hundred rows each, though one has 450,000 rows.
As requested earlier, I have included code at the end of this post for example tables and queries that accurately (but simply) replicate our real system.
Why we did and why:
• Stored the source XML records as CLOBS: We did this to preserve the records exactly as they were certified and sent from providers. In addition, we always access the entire XML record as a whole (e.g., when viewing a record or consolidating employee records), so this storage model seemed like a good fit. We can copy them into another format if necessary.
• Stored the consolidated XML employee records as “binary XML”. We did this because we almost always access a single, entire record as a whole (for view/edit), but might want to create some summary statistics at some point. Binary XML seemed the best fit.
• Used ora:view() for both tabular source records and lookup tables. We are not aware of any alternatives at this time. If it made sense, most code tables could be pre-converted into XML documents, but this seemed risky from a performance standpoint because the lookups use both code and date range constraints (the meaning of codes changes over time).
• Stored records as XMLTYPE columns in a table with other key columns on the table, plus an XMLTYPE metadata column. We thought this would facilitate pulling a single record (or a few records for a given employee) quickly. We knew this might be unnecessary given XML indexes and virtual columns, but were not experienced with those and wanted the comfort of traditional keys. We did not used XMLTYPE tables or the XML Repository for documents.
• Used XMLTABLE to consolidate XML records by looping over each distinct employee ID in the source batch. We also tried XMLQUERY and it seems to perform about the same. We can achieve 10 to 20 records/second if we do not do any code lookups during consolidation, just meeting our performance requirement, but still much slower than expected.
• Used PL/SQL with XMLFOREST to convert tabular source records to XML by looping over distinct employee IDs. We tried this outside PL/SQL both with XMLFOREST and XMLTABLE+ora:view(), but it hangs in both cases for more than 800 records (a known/open issue). We were able to get it to work by using an explicit cursor to loop over distinct employee IDs (rather than processing all records at once within the query). The performance is one record/second, which is minimally acceptable and interferes with other database activity.
• Used XMLQUERY plus ora:view() plus XPATH constraints to perform code lookups. When passing a single employee record, the response time ranges from 1 sec to 160 sec depending on the length of the record (i.e., number of years of service). We achieved a 5-fold speedup using an XMLINDEX (thank you Marco!!). The result may be minimally acceptable, but I’m baffled why the index would be needed when processing a single XML record. Other things we tried: joining code tables in the FOR...WHERE clauses, joining code tables using LET with XPATH constraints and LET with WHERE clause constraints, and looking up codes individually via JDBC from the application code at presentation time. All those approaches were slower. Note: the difference I mentioned above in equality/inequality constraint performance was due to data record variations not query plan variations.
What issues remain?
We have a minimally acceptable solution from a performance standpoint with one very awkward PL/SQL workaround. The performance of a mixed XML+relational data query is still marginal IMHO, until we properly utilize available optimizations, fix known problems, and perhaps get some new query optimizations. On the last point, I think the query plan for tabular lookups of codes in XML records is falling short right now. I’m reminded of data warehousing in the days before hash joins and star join optimization. I would be happy to be wrong, and just as happy for viable workarounds if I am right!
Here are the details on our code lookup challenge. Additional suggestions would be greatly appreciated. I’ll try to post more detail on the legacy table conversion challenge later.
-- The main record table:
create table RECORDS (
SSN varchar2(20),
XMLREC sys.xmltype
xmltype column XMLREC store as binary xml;
create index records_ssn on records(ssn);
-- A dozen code tables represented by one like this:
create table CODES (
CODE varchar2(4),
DESCRIPTION varchar2(500)
create index codes_code on codes(code);
-- Some XML records with coded values (the real records are much more complex of course):
-- I think this took about a minute or two
DECLARE
ssn varchar2(20);
xmlrec xmltype;
i integer;
BEGIN
xmlrec := xmltype('<?xml version="1.0"?>
<Root>
<Id>123456789</Id>
<Element>
<Subelement1><Code>11</Code></Subelement1>
<Subelement2><Code>21</Code></Subelement2>
<Subelement3><Code>31</Code></Subelement3>
</Element>
<Element>
<Subelement1><Code>11</Code></Subelement1>
<Subelement2><Code>21</Code></Subelement2>
<Subelement3><Code>31</Code></Subelement3>
</Element>
<Element>
<Subelement1><Code>11</Code></Subelement1>
<Subelement2><Code>21</Code></Subelement2>
<Subelement3><Code>31</Code></Subelement3>
</Element>
</Root>
for i IN 1..100000 loop
insert into records(ssn, xmlrec) values (i, xmlrec);
end loop;
commit;
END;
-- Some code data like this (ignoring date ranges on codes):
DECLARE
description varchar2(100);
i integer;
BEGIN
description := 'This is the code description ';
for i IN 1..3000 loop
insert into codes(code, description) values (to_char(i), description);
end loop;
commit;
end;
-- Retrieve one record while performing code lookups. Takes about 5-6 seconds...pretty slow.
-- Each additional lookup (times 3 repeating elements in the data) adds about 1 second.
-- A typical real record has 5 Elements and 20 Subelements, meaning more than 20 seconds to display the record
-- Note we are accessing a single XML record based on SSN
-- Note also we are reusing the one test code table multiple times for convenience of this test
select xmlquery('
for $r in Root
return
<Root>
<Id>123456789</Id>
{for $e in $r/Element
return
<Element>
<Subelement1>
{$e/Subelement1/Code}
<Description>
{ora:view("disaac","codes")/ROW[CODE=$e/Subelement1/Code]/DESCRIPTION/text() }
</Description>
</Subelement1>
<Subelement2>
{$e/Subelement2/Code}
<Description>
{ora:view("disaac","codes")/ROW[CODE=$e/Subelement2/Code]/DESCRIPTION/text()}
</Description>
</Subelement2>
<Subelement3>
{$e/Subelement3/Code}
<Description>
{ora:view("disaac","codes")/ROW[CODE=$e/Subelement3/Code]/DESCRIPTION/text() }
</Description>
</Subelement3>
</Element>
</Root>
' passing xmlrec returning content)
from records
where ssn = '10000';
The plan shows the nested loop access that slows things down.
By contrast, a functionally-similar SQL query on relational data will use a hash join and perform 10x to 100x faster, even for a single record. There seems to be no way for the optimizer to see the regularity in the XML structure and perform a corresponding optimization in joining the code tables. Not sure if registering a schema would help. Using structured storage probably would. But should that be necessary given we’re working with a single record?
Operation Object
|SELECT STATEMENT ()
| SORT (AGGREGATE)
| NESTED LOOPS (SEMI)
| TABLE ACCESS (FULL) CODES
| XPATH EVALUATION ()
| SORT (AGGREGATE)
| NESTED LOOPS (SEMI)
| TABLE ACCESS (FULL) CODES
| XPATH EVALUATION ()
| SORT (AGGREGATE)
| NESTED LOOPS (SEMI)
| TABLE ACCESS (FULL) CODES
| XPATH EVALUATION ()
| SORT (AGGREGATE)
| XPATH EVALUATION ()
| SORT (AGGREGATE)
| XPATH EVALUATION ()
| TABLE ACCESS (BY INDEX ROWID) RECORDS
| INDEX (RANGE SCAN) RECORDS_SSN
With an xmlindex, the same query above runs in about 1 second, so is about 5x faster (0.2 sec/lookup), which is almost good enough. Is this the answer? Or is there a better way? I’m not sure why the optimizer wants to scan the code tables and index into the (one) XML record, rather than the other way around, but maybe that makes sense if the optimizer wants to use the same general plan as when the WHERE clause constraint is relaxed to multiple records.
-- Add an xmlindex. Takes about 2.5 minutes
create index records_record_xml ON records(xmlrec)
indextype IS xdb.xmlindex;
Operation Object
|SELECT STATEMENT ()
| SORT (GROUP BY)
| FILTER ()
| NESTED LOOPS ()
| FAST DUAL ()
| TABLE ACCESS (BY INDEX ROWID) SYS113473_RECORDS_R_PATH_TABLE
| INDEX (RANGE SCAN) SYS113473_RECORDS_R_PATHID_IX
| SORT (AGGREGATE)
| FILTER ()
| TABLE ACCESS (FULL) CODES
| FILTER ()
| NESTED LOOPS ()
| FAST DUAL ()
| TABLE ACCESS (BY INDEX ROWID) SYS113473_RECORDS_R_PATH_TABLE
| INDEX (RANGE SCAN) SYS113473_RECORDS_R_PATHID_IX
| SORT (GROUP BY)
| FILTER ()
| NESTED LOOPS ()
| FAST DUAL ()
| TABLE ACCESS (BY INDEX ROWID) SYS113473_RECORDS_R_PATH_TABLE
| INDEX (RANGE SCAN) SYS113473_RECORDS_R_PATHID_IX
| SORT (AGGREGATE)
| FILTER ()
| TABLE ACCESS (FULL) CODES
| FILTER ()
| NESTED LOOPS ()
| FAST DUAL ()
| TABLE ACCESS (BY INDEX ROWID) SYS113473_RECORDS_R_PATH_TABLE
| INDEX (RANGE SCAN) SYS113473_RECORDS_R_PATHID_IX
| SORT (GROUP BY)
| FILTER ()
| NESTED LOOPS ()
| FAST DUAL ()
| TABLE ACCESS (BY INDEX ROWID) SYS113473_RECORDS_R_PATH_TABLE
| INDEX (RANGE SCAN) SYS113473_RECORDS_R_PATHID_IX
| SORT (AGGREGATE)
| FILTER ()
| TABLE ACCESS (FULL) CODES
| FILTER ()
| NESTED LOOPS ()
| FAST DUAL ()
| TABLE ACCESS (BY INDEX ROWID) SYS113473_RECORDS_R_PATH_TABLE
| INDEX (RANGE SCAN) SYS113473_RECORDS_R_PATHID_IX
| SORT (AGGREGATE)
| FILTER ()
| NESTED LOOPS ()
| FAST DUAL ()
| TABLE ACCESS (BY INDEX ROWID) SYS113473_RECORDS_R_PATH_TABLE
| INDEX (RANGE SCAN) SYS113473_RECORDS_R_PATHID_IX
| SORT (AGGREGATE)
| TABLE ACCESS (BY INDEX ROWID) SYS113473_RECORDS_R_PATH_TABLE
| INDEX (RANGE SCAN) SYS113473_RECORDS_R_PATHID_IX
| TABLE ACCESS (BY INDEX ROWID) RECORDS
| INDEX (RANGE SCAN) RECORDS_SSN
Am I on the right path, or am I totally using the wrong approach? I thought about using XSLT but was unsure how to reference the code tables.
I’ve done the best I can constraining the main record to a single row passed to the XMLQUERY. Given Mark’s post (thanks!) should I be joining and constraining the code tables in the SQL WHERE clause too? That’s going to make the query much more complicated, but right now we’re more concerned about performance than complexity. -
URGENT------MB5B : PERFORMANCE PROBLEM
Hi,
We are getting the time out error while running the transaction MB5B. We have posted the same to SAP global support for further analysis, and SAP revrted with note 1005901 to review.
The note consists of creating the Z table and some Z programs to execute the MB5B without time out error, and SAP has not provided what type logic has to be written and how we can be addressed this.
Could any one suggest us how can we proceed further.
Note as been attached for reference.
Note 1005901 - MB5B: Performance problems
Note Language: English Version: 3 Validity: Valid from 05.12.2006
Summary
Symptom
o The user starts transaction MB5B, or the respective report
RM07MLBD, for a very large number of materials or for all materials
in a plant.
o The transaction terminates with the ABAP runtime error
DBIF_RSQL_INVALID_RSQL.
o The transaction runtime is very long and it terminates with the
ABAP runtime error TIME_OUT.
o During the runtime of transaction MB5B, goods movements are posted
in parallel:
- The results of transaction MB5B are incorrect.
- Each run of transaction MB5B returns different results for the
same combination of "material + plant".
More Terms
MB5B, RM07MLBD, runtime, performance, short dump
Cause and Prerequisites
The DBIF_RSQL_INVALID_RSQL runtime error may occur if you enter too many
individual material numbers in the selection screen for the database
selection.
The runtime is long because of the way report RM07MLBD works. It reads the
stocks and values from the material masters first, then the MM documents
and, in "Valuated Stock" mode, it then reads the respective FI documents.
If there are many MM and FI documents in the system, the runtimes can be
very long.
If goods movements are posted during the runtime of transaction MB5B for
materials that should also be processed by transaction MB5B, transaction
MB5B may return incorrect results.
Example: Transaction MB5B should process 100 materials with 10,000 MM
documents each. The system takes approximately 1 second to read the
material master data and it takes approximately 1 hour to read the MM and
FI documents. A goods movement for a material to be processed is posted
approximately 10 minutes after you start transaction MB5B. The stock for
this material before this posting has already been determined. The new MM
document is also read, however. The stock read before the posting is used
as the basis for calculating the stocks for the start and end date.
If you execute transaction MB5B during a time when no goods movements are
posted, these incorrect results do not occur.
Solution
The SAP standard release does not include a solution that allows you to
process mass data using transaction MB5B. The requirements for transaction
MB5B are very customer-specific. To allow for these customer-specific
requirements, we provide the following proposed implementation:
Implementation proposal:
o You should call transaction MB5B for only one "material + plant"
combination at a time.
o The list outputs for each of these runs are collected and at the
end of the processing they are prepared for a large list output.
You need three reports and one database table for this function. You can
store the lists in the INDX cluster table.
o Define work database table ZZ_MB5B with the following fields:
- Material number
- Plant
- Valuation area
- Key field for INDX cluster table
o The size category of the table should be based on the number of
entries in material valuation table MBEW.
Report ZZ_MB5B_PREPARE
In the first step, this report deletes all existing entries from the
ZZ_MB5B work table and the INDX cluster table from the last mass data
processing run of transaction MB5B.
o The ZZ_MB5B work table is filled in accordance with the selected
mode of transaction MB5B:
- Stock type mode = Valuated stock
- Include one entry in work table ZZ_MB5B for every "material +
valuation area" combination from table MBEW.
o Other modes:
- Include one entry in work table ZZ_MB5B for every "material +
plant" combination from table MARC
Furthermore, the new entries in work table ZZ_MB5B are assigned a unique
22-character string that later serves as a key term for cluster table INDX.
Report ZZ_MB5B_MONITOR
This report reads the entries sequentially in work table ZZ_MB5B. Depending
on the mode of transaction MB5B, a lock is executed as follows:
o Stock type mode = Valuated stock
For every "material + valuation area" combination, the system
determines all "material + plant" combinations. All determined
"material + plant" combinations are locked.
o Other modes:
- Every "material + plant" combination is locked.
- The entries from the ZZ_MB5B work table can be processed as
follows only if they have been locked successfully.
- Start report RM07MLBD for the current "Material + plant"
combination, or "material + valuation area" combination,
depending on the required mode.
- The list created is stored with the generated key term in the
INDX cluster table.
- The current entry is deleted from the ZZ_MB5B work table.
- Database updates are executed with COMMIT WORK AND WAIT.
- The lock is released.
- The system reads the next entry in the ZZ_MB5B work table.
Application
- The lock ensures that no goods movements can be posted during
the runtime of the RM07MLBD report for the "material + Plant"
combination to be processed.
- You can start several instances of this report at the same
time. This method ensures that all "material + plant"
combinations can be processed at the same time.
- The system takes just a few seconds to process a "material +
Plant" combination so there is just minimum disruption to
production operation.
- This report is started until there are no more entries in the
ZZ_MB5B work table.
- If the report terminates or is interrupted, it can be started
again at any time.
Report ZZ_MB5B_PRINT
You can use this report when all combinations of "material + plant", or
"material + valuation area" from the ZZ_MB5B work table have been
processed. The report reads the saved lists from the INDX cluster table and
adds these individual lists to a complete list output.
Estimated implementation effort
An experienced ABAP programmer requires an estimated three to five days to
create the ZZ_MB5B work table and these three reports. You can find a
similar program as an example in Note 32236: MBMSSQUA.
If you need support during the implementation, contact your SAP consultant.
Header Data
Release Status: Released for Customer
Released on: 05.12.2006 16:14:11
Priority: Recommendations/additional info
Category: Consulting
Main Component MM-IM-GF-REP IM Reporting (no LIS)
The note is not release-dependent.
Thanks in advance.
Edited by: Neliea on Jan 9, 2008 10:38 AM
Edited by: Neliea on Jan 9, 2008 10:39 AMbefore you try any of this try working with database-hints as described in note 921165, 902157, 918992
-
Performance problem with transaction log
We are having some performance problem in SAP BW 3.5 system running on MS SQL server 2000.The box is sized 63,574 MB. The transaction logs gets filled up after loading data in to a transactional cube or after doing selective deletion. The size of the transaction log is 7,587MB currently.
Basis team feels that when performing either loading or selective deletion, SQL server views it as a single transaction and doesn't commit until every record is written. And so as a result, transaction logs fills up ultimately bringing the system down.
The system log shows a DBIF error during the transaction log fill up as follows:
Database error 9002 at COM
> [9002] the log file for database 'BWP' is full. Back up the
> Transaction log for the database to free up some log space.
Function COMMIT on connection R/3 failed
Perform rollback
Can we make changes to Database to make commit action frequently? Is there any parameters we could change to reduce the packet size? Is there some setting to be changed in SQL server?
Any Help will be appreciated.if you have disk space avialable you can allocate more space to the transaction log.
-
Performance problem PRD environment
Hi Everybody,
Some moments during the day we are having a performance problem in our PRD environment. The system get slow.
The CPU get 100% of his usage.
The most used process are:
oracle.exe - 35%
disp+work - 30%
disp+work - 11%
disp+work - 05%
disp+work - 05%
In SM50 we have the programs running.
In ST03N we can see the most sequential reads and DB time problems.
If we do a performance tunning of the programs identified we can reduce the performance problem?
Anybody know another solution or where can i get more information to see where exactly the problem is?More transactions
maybe?
Best Regards,
Fábio Karnik TchobnianHi,
In SM50 we have the programs running.
At what staus they are like commit , read etc; you also need to check for active JOBs get the details.
In ST03N we can see the most sequential reads and DB time problems.
Check for poorest SQL statement from ST04 => SQL cache => remove * => check for total execution time (descending order) => check for latest 3-4 SQL statement with your'e developer to fine tune the programe
This is just for analysis if this is happening every time then above solution for upgrading CPU is worth than tuning
Regards; -
Hello,
I've been having performance problems with one query using XMLAGG. Explain plan shows that all tables are joined using correct indexes - there is nothing wrong with it. After series of experiments with the query I found out that the problem is in use of XMLAGG function. It aggregates the data before submitting it to the result XML document.
I there any way to generate something like
<Main attr1="...">
<el1>...</el1>
<el2>...</el2>
<SubElement>
<row>row1 from the database</row>
<row>row2 from the database</row>
<row>rowN from the database</row>
</SubElement>
<Main>
without using XMLAGG in the subquery or somehow switch the aggregation off? I dont really need the records to be aggregated, all I need is to be able to generate multiple record based XML in the subquery which would have normally failed on (single-row query returned more than one record exception) without XMLAGG.
Thanks in advance
AlexeyTKPROF: Release 9.2.0.1.0 - Production on Wed Jan 31 15:21:34 2007
Copyright (c) 1982, 2002, Oracle Corporation. All rights reserved.
Trace file: lnsqd1_ora_7350_perfoamance_test_01.trc
Sort options: default
count = number of times OCI procedure was executed
cpu = cpu time in seconds executing
elapsed = elapsed time in seconds executing
disk = number of physical reads of buffers from disk
query = number of buffers gotten for consistent read
current = number of buffers gotten in current mode (usually for update)
rows = number of rows processed by the fetch or execute call
ALTER SESSION SET EVENTS '10046 trace name context forever, level 1'
call count cpu elapsed disk query current rows
Parse 0 0.00 0.00 0 0 0 0
Execute 1 0.00 0.00 0 0 0 0
Fetch 0 0.00 0.00 0 0 0 0
total 1 0.00 0.00 0 0 0 0
Misses in library cache during parse: 0
Optimizer goal: ALL_ROWS
Parsing user id: 113
create table tmp_003
as
select a.*, 'NEW ' as eventtype
from v_deal_xml a
where a.deal_id in
select a.pk_value
from da_history a, gx_type b
where a.evtime <= to_date('1/22/2007 12:59:26', 'MM/DD/RRRR HH24:MI:SS')
and a.type_id = b.type_id
and b.xmltype = 'GDP/DEAL'
and a.eventtype = 'NEW'
call count cpu elapsed disk query current rows
Parse 1 0.12 0.12 0 60 1 0
Execute 1 108.30 105.87 0 6343 1387 172
Fetch 0 0.00 0.00 0 0 0 0
total 2 108.42 106.00 0 6403 1388 172
Misses in library cache during parse: 1
Optimizer goal: ALL_ROWS
Parsing user id: 113
insert into tmp_003
select a.*, 'UPDATE' as eventtype
from v_deal_xml a
where a.deal_id in
select a.pk_value --b.xmltype, a.*
from da_history a, gx_type b
where a.evtime <= to_date('1/22/2007 12:59:26', 'MM/DD/RRRR HH24:MI:SS')
and a.type_id = b.type_id
and b.xmltype = 'GDP/DEAL'
and a.eventtype = 'UPDATE'
call count cpu elapsed disk query current rows
Parse 1 0.13 0.13 0 60 0 0
Execute 1 256.74 250.96 0 18752 3815 381
Fetch 0 0.00 0.00 0 0 0 0
total 2 256.87 251.09 0 18812 3815 381
Misses in library cache during parse: 1
Optimizer goal: ALL_ROWS
Parsing user id: 113
Rows Row Source Operation
381 NESTED LOOPS (cr=1194 pr=0 pw=0 time=26458 us)
381 VIEW VW_NSO_1 (cr=430 pr=0 pw=0 time=6665 us)
381 SORT UNIQUE (cr=430 pr=0 pw=0 time=6285 us)
381 MERGE JOIN (cr=430 pr=0 pw=0 time=3761 us)
1 TABLE ACCESS BY INDEX ROWID GX_TYPE (cr=2 pr=0 pw=0 time=67 us)
5 INDEX FULL SCAN XPKGX_TYPE (cr=1 pr=0 pw=0 time=24 us)(object id 71282)
381 SORT JOIN (cr=428 pr=0 pw=0 time=3698 us)
603 TABLE ACCESS FULL DA_HISTORY (cr=428 pr=0 pw=0 time=1339 us)
381 TABLE ACCESS BY INDEX ROWID DEAL (cr=764 pr=0 pw=0 time=16692 us)
381 INDEX UNIQUE SCAN DEAL_PK (cr=383 pr=0 pw=0 time=8926 us)(object id 71236)
commit
call count cpu elapsed disk query current rows
Parse 1 0.00 0.00 0 0 0 0
Execute 1 0.00 0.00 0 0 1 0
Fetch 0 0.00 0.00 0 0 0 0
total 2 0.00 0.00 0 0 1 0
Misses in library cache during parse: 0
Parsing user id: 113
OVERALL TOTALS FOR ALL NON-RECURSIVE STATEMENTS
call count cpu elapsed disk query current rows
Parse 3 0.25 0.25 0 120 1 0
Execute 4 365.04 356.84 0 25095 5203 553
Fetch 0 0.00 0.00 0 0 0 0
total 7 365.29 357.10 0 25215 5204 553
Misses in library cache during parse: 2
OVERALL TOTALS FOR ALL RECURSIVE STATEMENTS
call count cpu elapsed disk query current rows
Parse 83 0.01 0.01 0 0 0 0
Execute 125 0.03 0.03 0 108 201 46
Fetch 114 0.01 0.00 0 221 0 72
total 322 0.05 0.04 0 329 201 118
Misses in library cache during parse: 3
Misses in library cache during execute: 3
4 user SQL statements in session.
125 internal SQL statements in session.
129 SQL statements in session.
Trace file: lnsqd1_ora_7350_perfoamance_test_01.trc
Trace file compatibility: 9.00.01
Sort options: default
1 session in tracefile.
4 user SQL statements in trace file.
125 internal SQL statements in trace file.
129 SQL statements in trace file.
45 unique SQL statements in trace file.
11196 lines in trace file. -
Performance optimization during database selection.
hi gurus,
pls any explain about this...
Strong knowledge of performance optimization during database selection.
regards,
praveenHi Praveen,
Performance Notes
1.Keep the Result Set Small
You should aim to keep the result set small. This reduces both the amount of memory used in the database system and the network load when transferring data to the application server. To reduce the size of your result sets, use the WHERE and HAVING clauses.
Using the WHERE Clause
Whenever you access a database table, you should use a WHERE clause in the corresponding Open SQL statement. Even if a program containing a SELECT statement with no WHERE clause performs well in tests, it may slow down rapidly in your production system, where the data volume increases daily. You should only dispense with the WHERE clause in exceptional cases where you really need the entire contents of the database table every time the statement is executed.
When you use the WHERE clause, the database system optimizes the access and only transfers the required data. You should never transfer unwanted data to the application server and then filter it using ABAP statements.
Using the HAVING Clause
After selecting the required lines in the WHERE clause, the system then processes the GROUP BY clause, if one exists, and summarizes the database lines selected. The HAVING clause allows you to restrict the grouped lines, and in particular, the aggregate expressions, by applying further conditions.
Effect
If you use the WHERE and HAVING clauses correctly:
There are no more physical I/Os in the database than necessary
No unwanted data is stored in the database cache (it could otherwise displace data that is actually required)
The CPU usage of the database host is minimize
The network load is reduced, since only the data that is required by the application is transferred to the application server.
Minimize the Amount of Data Transferred
Data is transferred between the database system and the application server in blocks. Each block is up to 32 KB in size (the precise size depends on your network communication hardware). Administration information is transported in the blocks as well as the data.
To minimize the network load, you should transfer as few blocks as possible. Open SQL allows you to do this as follows:
Restrict the Number of Lines
If you only want to read a certain number of lines in a SELECT statement, use the UP TO <n> ROWS addition in the FROM clause. This tells the database system only to transfer <n> lines back to the application server. This is more efficient than transferring more lines than necessary back to the application server and then discarding them in your ABAP program.
If you expect your WHERE clause to return a large number of duplicate entries, you can use the DISTINCT addition in the SELECT clause.
Restrict the Number of Columns
You should only read the columns from a database table that you actually need in the program. To do this, list the columns in the SELECT clause. Note here that the INTO CORRESPONDING FIELDS addition in the INTO clause is only efficient with large volumes of data, otherwise the runtime required to compare the names is too great. For small amounts of data, use a list of variables in the INTO clause.
Do not use * to select all columns unless you really need them. However, if you list individual columns, you may have to adjust the program if the structure of the database table is changed in the ABAP Dictionary. If you specify the database table dynamically, you must always read all of its columns.
Use Aggregate Functions
If you only want to use data for calculations, it is often more efficient to use the aggregate functions of the SELECT clause than to read the individual entries from the database and perform the calculations in the ABAP program.
Aggregate functions allow you to find out the number of values and find the sum, average, minimum, and maximum values.
Following an aggregate expression, only its result is transferred from the database.
Data Transfer when Changing Table Lines
When you use the UPDATE statement to change lines in the table, you should use the WHERE clause to specify the relevant lines, and then SET statements to change only the required columns.
When you use a work area to overwrite table lines, too much data is often transferred. Furthermore, this method requires an extra SELECT statement to fill the work area. Minimize the Number of Data Transfers
In every Open SQL statement, data is transferred between the application server and the database system. Furthermore, the database system has to construct or reopen the appropriate administration data for each database access. You can therefore minimize the load on the network and the database system by minimizing the number of times you access the database.
Multiple Operations Instead of Single Operations
When you change data using INSERT, UPDATE, and DELETE, use internal tables instead of single entries. If you read data using SELECT, it is worth using multiple operations if you want to process the data more than once, other wise, a simple select loop is more efficient.
Avoid Repeated Access
As a rule you should read a given set of data once only in your program, and using a single access. Avoid accessing the same data more than once (for example, SELECT before an UPDATE).
Avoid Nested SELECT Loops
A simple SELECT loop is a single database access whose result is passed to the ABAP program line by line. Nested SELECT loops mean that the number of accesses in the inner loop is multiplied by the number of accesses in the outer loop. You should therefore only use nested SELECT loops if the selection in the outer loop contains very few lines.
However, using combinations of data from different database tables is more the rule than the exception in the relational data model. You can use the following techniques to avoid nested SELECT statements:
ABAP Dictionary Views
You can define joins between database tables statically and systemwide as views in the ABAP Dictionary. ABAP Dictionary views can be used by all ABAP programs. One of their advantages is that fields that are common to both tables (join fields) are only transferred once from the database to the application server.
Views in the ABAP Dictionary are implemented as inner joins. If the inner table contains no lines that correspond to lines in the outer table, no data is transferred. This is not always the desired result. For example, when you read data from a text table, you want to include lines in the selection even if the corresponding text does not exist in the required language. If you want to include all of the data from the outer table, you can program a left outer join in ABAP.
The links between the tables in the view are created and optimized by the database system. Like database tables, you can buffer views on the application server. The same buffering rules apply to views as to tables. In other words, it is most appropriate for views that you use mostly to read data. This reduces the network load and the amount of physical I/O in the database.
Joins in the FROM Clause
You can read data from more than one database table in a single SELECT statement by using inner or left outer joins in the FROM clause.
The disadvantage of using joins is that redundant data is read from the hierarchically-superior table if there is a 1:N relationship between the outer and inner tables. This can considerably increase the amount of data transferred from the database to the application server. Therefore, when you program a join, you should ensure that the SELECT clause contains a list of only the columns that you really need. Furthermore, joins bypass the table buffer and read directly from the database. For this reason, you should use an ABAP Dictionary view instead of a join if you only want to read the data.
The runtime of a join statement is heavily dependent on the database optimizer, especially when it contains more than two database tables. However, joins are nearly always quicker than using nested SELECT statements.
Subqueries in the WHERE and HAVING Clauses
Another way of accessing more than one database table in the same Open SQL statement is to use subqueries in the WHERE or HAVING clause. The data from a subquery is not transferred to the application server. Instead, it is used to evaluate conditions in the database system. This is a simple and effective way of programming complex database operations.
Using Internal Tables
It is also possible to avoid nested SELECT loops by placing the selection from the outer loop in an internal table and then running the inner selection once only using the FOR ALL ENTRIES addition. This technique stems from the time before joins were allowed in the FROM clause. On the other hand, it does prevent redundant data from being transferred from the database.
Using a Cursor to Read Data
A further method is to decouple the INTO clause from the SELECT statement by opening a cursor using OPEN CURSOR and reading data line by line using FETCH NEXT CURSOR. You must open a new cursor for each nested loop. In this case, you must ensure yourself that the correct lines are read from the database tables in the correct order. This usually requires a foreign key relationship between the database tables, and that they are sorted by the foreign key. Minimize the Search Overhead
You minimize the size of the result set by using the WHERE and HAVING clauses. To increase the efficiency of these clauses, you should formulate them to fit with the database table indexes.
Database Indexes
Indexes speed up data selection from the database. They consist of selected fields of a table, of which a copy is then made in sorted order. If you specify the index fields correctly in a condition in the WHERE or HAVING clause, the system only searches part of the index (index range scan).
The primary index is always created automatically in the R/3 System. It consists of the primary key fields of the database table. This means that for each combination of fields in the index, there is a maximum of one line in the table. This kind of index is also known as UNIQUE.
If you cannot use the primary index to determine the result set because, for example, none of the primary index fields occur in the WHERE or HAVING clause, the system searches through the entire table (full table scan). For this case, you can create secondary indexes, which can restrict the number of table entries searched to form the result set.
You specify the fields of secondary indexes using the ABAP Dictionary. You can also determine whether the index is unique or not. However, you should not create secondary indexes to cover all possible combinations of fields.
Only create one if you select data by fields that are not contained in another index, and the performance is very poor. Furthermore, you should only create secondary indexes for database tables from which you mainly read, since indexes have to be updated each time the database table is changed. As a rule, secondary indexes should not contain more than four fields, and you should not have more than five indexes for a single database table.
If a table has more than five indexes, you run the risk of the optimizer choosing the wrong one for a particular operation. For this reason, you should avoid indexes with overlapping contents.
Secondary indexes should contain columns that you use frequently in a selection, and that are as highly selective as possible. The fewer table entries that can be selected by a certain column, the higher that columns selectivity. Place the most selective fields at the beginning of the index. Your secondary index should be so selective that each index entry corresponds to at most five percent of the table entries. If this is not the case, it is not worth creating the index. You should also avoid creating indexes for fields that are not always filled, where their value is initial for most entries in the table.
If all of the columns in the SELECT clause are contained in the index, the system does not have to search the actual table data after reading from the index. If you have a SELECT clause with very few columns, you can improve performance dramatically by including these columns in a secondary index.
Formulating Conditions for Indexes
You should bear in mind the following when formulating conditions for the WHERE and HAVING clauses so that the system can use a database index and does not have to use a full table scan.
Check for Equality and Link Using AND
The database index search is particularly efficient if you check all index fields for equality (= or EQ) and link the expressions using AND.
Use Positive Conditions
The database system only supports queries that describe the result in positive terms, for example, EQ or LIKE. It does not support negative expressions like NE or NOT LIKE.
If possible, avoid using the NOT operator in the WHERE clause, because it is not supported by database indexes; invert the logical expression instead.
Using OR
The optimizer usually stops working when an OR expression occurs in the condition. This means that the columns checked using OR are not included in the index search. An exception to this are OR expressions at the outside of conditions. You should try to reformulate conditions that apply OR expressions to columns relevant to the index, for example, into an IN condition.
Using Part of the Index
If you construct an index from several columns, the system can still use it even if you only specify a few of the columns in a condition. However, in this case, the sequence of the columns in the index is important. A column can only be used in the index search if all of the columns before it in the index definition have also been specified in the condition.
Checking for Null Values
The IS NULL condition can cause problems with indexes. Some database systems do not store null values in the index structure. Consequently, this field cannot be used in the index.
Avoid Complex Conditions
Avoid complex conditions, since the statements have to be broken down into their individual components by the database system.
Reduce the Database Load
Unlike application servers and presentation servers, there is only one database server in your system. You should therefore aim to reduce the database load as much as possible. You can use the following methods:
Buffer Tables on the Application Server
You can considerably reduce the time required to access data by buffering it in the application server table buffer. Reading a single entry from table T001 can take between 8 and 600 milliseconds, while reading it from the table buffer takes 0.2 - 1 milliseconds.
Whether a table can be buffered or not depends its technical attributes in the ABAP Dictionary. There are three buffering types:
Resident buffering (100%) The first time the table is accessed, its entire contents are loaded in the table buffer.
Generic buffering In this case, you need to specify a generic key (some of the key fields) in the technical settings of the table in the ABAP Dictionary. The table contents are then divided into generic areas. When you access data with one of the generic keys, the whole generic area is loaded into the table buffer. Client-specific tables are often buffered generically by client.
Partial buffering (single entry) Only single entries are read from the database and stored in the table buffer.
When you read from buffered tables, the following happens:
1. An ABAP program requests data from a buffered table.
2. The ABAP processor interprets the Open SQL statement. If the table is defined as a buffered table in the ABAP Dictionary, the ABAP processor checks in the local buffer on the application server to see if the table (or part of it) has already been buffered.
3. If the table has not yet been buffered, the request is passed on to the database. If the data exists in the buffer, it is sent to the program.
4. The database server passes the data to the application server, which places it in the table buffer.
5. The data is passed to the program.
When you change a buffered table, the following happens:
1. The database table is changed and the buffer on the application server is updated. The database interface logs the update statement in the table DDLOG. If the system has more than one application server, the buffer on the other servers is not updated at once.
2. All application servers periodically read the contents of table DDLOG, and delete the corresponding contents from their buffers where necessary. The granularity depends on the buffering type. The table buffers in a distributed system are generally synchronized every 60 seconds (parameter: rsdisp/bufreftime).
3. Within this period, users on non-synchronized application servers will read old data. The data is not recognized as obsolete until the next buffer synchronization. The next time it is accessed, it is re-read from the database.
You should buffer the following types of tables:
Tables that are read very frequently
Tables that are changed very infrequently
Relatively small tables (few lines, few columns, or short columns)
Tables where delayed update is acceptable.
Once you have buffered a table, take care not to use any Open SQL statements that bypass the buffer.
The SELECT statement bypasses the buffer when you use any of the following:
The BYPASSING BUFFER addition in the FROM clause
The DISTINCT addition in the SELECT clause
Aggregate expressions in the SELECT clause
Joins in the FROM clause
The IS NULL condition in the WHERE clause
Subqueries in the WHERE clause
The ORDER BY clause
The GROUP BY clause
The FOR UPDATE addition
Furthermore, all Native SQL statements bypass the buffer.
Avoid Reading Data Repeatedly
If you avoid reading the same data repeatedly, you both reduce the number of database accesses and reduce the load on the database. Furthermore, a "dirty read" may occur with database tables other than Oracle. This means that the second time you read data from a database table, it may be different from the data read the first time. To ensure that the data in your program is consistent, you should read it once only and then store it in an internal table.
Sort Data in Your ABAP Programs
The ORDER BY clause in the SELECT statement is not necessarily optimized by the database system or executed with the correct index. This can result in increased runtime costs. You should only use ORDER BY if the database sort uses the same index with which the table is read. To find out which index the system uses, use SQL Trace in the ABAP Workbench Performance Trace. If the indexes are not the same, it is more efficient to read the data into an internal table or extract and sort it in the ABAP program using the SORT statement.
Use Logical Databases
SAP supplies logical databases for all applications. A logical database is an ABAP program that decouples Open SQL statements from application programs. They are optimized for the best possible database performance. However, it is important that you use the right logical database. The hierarchy of the data you want to read must reflect the structure of the logical database, otherwise, they can have a negative effect on performance. For example, if you want to read data from a table right at the bottom of the hierarchy of the logical database, it has to read at least the key fields of all tables above it in the hierarchy. In this case, it is more efficient to use a SELECT statement.
Work Processes
Work processes execute the individual dialog steps in R/3 applications. The next two sections describe firstly the structure of a work process, and secondly the different types of work process in the R/3 System.
Structure of a Work Process
Work processes execute the dialog steps of application programs. They are components of an application server. The following diagram shows the components of a work process:
Each work process contains two software processors and a database interface.
Screen Processor
In R/3 application programming, there is a difference between user interaction and processing logic. From a programming point of view, user interaction is controlled by screens. As well as the actual input mask, a screen also consists of flow logic. The screen flow logic controls a large part of the user interaction. The R/3 Basis system contains a special language for programming screen flow logic. The screen processor executes the screen flow logic. Via the dispatcher, it takes over the responsibility for communication between the work process and the SAPgui, calls modules in the flow logic, and ensures that the field contents are transferred from the screen to the flow logic.
ABAP Processor
The actual processing logic of an application program is written in ABAP - SAPs own programming language. The ABAP processor executes the processing logic of the application program, and communicates with the database interface. The screen processor tells the ABAP processor which module of the screen flow logic should be processed next. The following screen illustrates the interaction between the screen and the ABAP processors when an application program is running.
Database Interface
The database interface provides the following services:
Establishing and terminating connections between the work process and the database.
Access to database tables
Access to R/3 Repository objects (ABAP programs, screens and so on)
Access to catalog information (ABAP Dictionary)
Controlling transactions (commit and rollback handling)
Table buffer administration on the application server.
The following diagram shows the individual components of the database interface:
The diagram shows that there are two different ways of accessing databases: Open SQL and Native SQL.
Open SQL statements are a subset of Standard SQL that is fully integrated in ABAP. They allow you to access data irrespective of the database system that the R/3 installation is using. Open SQL consists of the Data Manipulation Language (DML) part of Standard SQL; in other words, it allows you to read (SELECT) and change (INSERT, UPDATE, DELETE) data. The tasks of the Data Definition Language (DDL) and Data Control Language (DCL) parts of Standard SQL are performed in the R/3 System by the ABAP Dictionary and the authorization system. These provide a unified range of functions, irrespective of database, and also contain functions beyond those offered by the various database systems.
Open SQL also goes beyond Standard SQL to provide statements that, in conjunction with other ABAP constructions, can simplify or speed up database access. It also allows you to buffer certain tables on the application server, saving excessive database access. In this case, the database interface is responsible for comparing the buffer with the database. Buffers are partly stored in the working memory of the current work process, and partly in the shared memory for all work processes on an application server. Where an R/3 System is distributed across more than one application server, the data in the various buffers is synchronized at set intervals by the buffer management. When buffering the database, you must remember that data in the buffer is not always up to date. For this reason, you should only use the buffer for data which does not often change.
Native SQL is only loosely integrated into ABAP, and allows access to all of the functions contained in the programming interface of the respective database system. Unlike Open SQL statements, Native SQL statements are not checked and converted, but instead are sent directly to the database system. Programs that use Native SQL are specific to the database system for which they were written. R/3 applications contain as little Native SQL as possible. In fact, it is only used in a few Basis components (for example, to create or change table definitions in the ABAP Dictionary).
The database-dependent layer in the diagram serves to hide the differences between database systems from the rest of the database interface. You choose the appropriate layer when you install the Basis system. Thanks to the standardization of SQL, the differences in the syntax of statements are very slight. However, the semantics and behavior of the statements have not been fully standardized, and the differences in these areas can be greater. When you use Native SQL, the function of the database-dependent layer is minimal.
Types of Work Process
Although all work processes contain the components described above, they can still be divided into different types. The type of a work process determines the kind of task for which it is responsible in the application server. It does not specify a particular set of technical attributes. The individual tasks are distributed to the work processes by the dispatcher.
Before you start your R/3 System, you determine how many work processes it will have, and what their types will be. The dispatcher starts the work processes and only assigns them tasks that correspond to their type. This means that you can distribute work process types to optimize the use of the resources on your application servers.
The following diagram shows again the structure of an application server, but this time, includes the various possible work process types:
The various work processes are described briefly below. Other parts of this documentation describe the individual components of the application server and the R/3 System in more detail.
Dialog Work Process
Dialog work processes deal with requests from an active user to execute dialog steps.
Update Work Process
Update work processes execute database update requests. Update requests are part of an SAP LUW that bundle the database operations resulting from the dialog in a database LUW for processing in the background.
Background Work Process
Background work processes process programs that can be executed without user interaction (background jobs).
Enqueue Work Process
The enqueue work process administers a lock table in the shared memory area. The lock table contains the logical database locks for the R/3 System and is an important part of the SAP LUW concept. In an R/3 System, you may only have one lock table. You may therefore also only have one application server with enqueue work processes.
Spool Work Process
The spool work process passes sequential datasets to a printer or to optical archiving. Each application server may contain several spool work process.
The services offered by an application server are determined by the types of its work processes. One application server may, of course, have more than one function. For example, it may be both a dialog server and the enqueue server, if it has several dialog work processes and an enqueue work process.
You can use the system administration functions to switch a work process between dialog and background modes while the system is still running. This allows you, for example, to switch an R/3 System between day and night operation, where you have more dialog than background work processes during the day, and the other way around during the night.
ABAP Application Server
R/3 programs run on application servers. They are an important component of the R/3 System. The following sections describe application servers in more detail.
Structure of an ABAP Application Server
The application layer of an R/3 System is made up of the application servers and the message server. Application programs in an R/3 System are run on application servers. The application servers communicate with the presentation components, the database, and also with each other, using the message server.
The following diagram shows the structure of an application server:
The individual components are:
Work Processes
An application server contains work processes, which are components that can run an application. Work processes are components that are able to execute an application (that is, one dialog step each). Each work process is linked to a memory area containing the context of the application being run. The context contains the current data for the application program. This needs to be available in each dialog step. Further information about the different types of work process is contained later on in this documentation.
Dispatcher
Each application server contains a dispatcher. The dispatcher is the link between the work processes and the users logged onto the application server. Its task is to receive requests for dialog steps from the SAP GUI and direct them to a free work process. In the same way, it directs screen output resulting from the dialog step back to the appropriate user.
Gateway
Each application server contains a gateway. This is the interface for the R/3 communication protocols (RFC, CPI/C). It can communicate with other application servers in the same R/3 System, with other R/3 Systems, with R/2 Systems, or with non-SAP systems.
The application server structure as described here aids the performance and scalability of the entire R/3 System. The fixed number of work processes and dispatching of dialog steps leads to optimal memory use, since it means that certain components and the memory areas of a work process are application-independent and reusable. The fact that the individual work processes work independently makes them suitable for a multi-processor architecture. The methods used in the dispatcher to distribute tasks to work processes are discussed more closely in the section Dispatching Dialog Steps.
Shared Memory
All of the work processes on an application server use a common main memory area called shared memory to save contexts or to buffer constant data locally.
The resources that all work processes use (such as programs and table contents) are contained in shared memory. Memory management in the R/3 System ensures that the work processes always address the correct context, that is the data relevant to the current state of the program that is running. A mapping process projects the required context for a dialog step from shared memory into the address of the relevant work process. This reduces the actual copying to a minimum.
Local buffering of data in the shared memory of the application server reduces the number of database reads required. This reduces access times for application programs considerably. For optimal use of the buffer, you can concentrate individual applications (financial accounting, logistics, human resources) into separate application server groups.
Database Connection
When you start up an R/3 System, each application server registers its work processes with the database layer, and receives a single dedicated channel for each. While the system is running, each work process is a user (client) of the database system (server). You cannot change the work process registration while the system is running. Neither can you reassign a database channel from one work process to another. For this reason, a work process can only make database changes within a single database logical unit of work (LUW). A database LUW is an inseparable sequence of database operations. This has important consequences for the programming model explained below.
Dispatching Dialog Steps
The number of users logged onto an application server is often many times greater than the number of available work processes. Furthermore, it is not restricted by the R/3 system architecture. Furthermore, each user can run several applications at once. The dispatcher has the important task of distributing all dialog steps among the work processes on the application server.
The following diagram is an example of how this might happen:
1. The dispatcher receives the request to execute a dialog step from user 1 and directs it to work process 1, which happens to be free. The work process addresses the context of the application program (in shared memory) and executes the dialog step. It then becomes free again.
2. The dispatcher receives the request to execute a dialog step from user 2 and directs it to work process 1, which is now free again. The work process executes the dialog step as in step 1.
3. While work process 1 is still working, the dispatcher receives a further request from user 1 and directs it to work process 2, which is free.
4. After work processes 1 and 2 have finished processing their dialog steps, the dispatcher receives another request from user 1 and directs it to work process 1, which is free again.
5. While work process 1 is still working, the dispatcher receives a further request from user 2 and directs it to work process 2, which is free.
From this example, we can see that:
A dialog step from a program is assigned to a single work process for execution.
The individual dialog steps of a program can be executed on different work processes, and the program context must be addressed for each new work process.
A work process can execute dialog steps of different programs from different users.
The example does not show that the dispatcher tries to distribute the requests to the work processes such that the same work process is used as often as possible for the successive dialog steps in an application. This is useful, since it saves the program context having to be addressed each time a dialog step is executed.
Dispatching and the Programming Model
The separation of application and presentation layer made it necessary to split up application programs into dialog steps. This, and the fact that dialog steps are dispatched to individual work processes, has had important consequences for the programming model.
As mentioned above, a work process can only make database changes within a single database logical unit of work (LUW). A database LUW is an inseparable sequence of database operations. The contents of the database must be consistent at its beginning and end. The beginning and end of a database LUW are defined by a commit command to the database system (database commit). During a database LUW, that is, between two database commits, the database system itself ensures consistency within the database. In other words, it takes over tasks such as locking database entries while they are being edited, or restoring the old data (rollback) if a step terminates in an error.
A typical SAP application program extends over several screens and the corresponding dialog steps. The user requests database changes on the individual screens that should lead to the database being consistent once the screens have all been processed. However, the individual dialog steps run on different work processes, and a single work process can process dialog steps from other applications. It is clear that two or more independent applications whose dialog steps happen to be processed on the same work process cannot be allowed to work with the same database LUW.
Consequently, a work process must open a separate database LUW for each dialog step. The work process sends a commit command (database commit) to the database at the end of each dialog step in which it makes database changes. These commit commands are called implicit database commits, since they are not explicitly written into the application program.
These implicit database commits mean that a database LUW can be kept open for a maximum of one dialog step. This leads to a considerable reduction in database load, serialization, and deadlocks, and enables a large number of users to use the same system.
However, the question now arises of how this method (1 dialog step = 1 database LUW) can be reconciled with the demand to make commits and rollbacks dependent on the logical flow of the application program instead of the technical distribution of dialog steps. Database update requests that depend on one another form logical units in the program that extend over more than one dialog step. The database changes associated with these logical units must be executed together and must also be able to be undone together.
The SAP programming model contains a series of bundling techniques that allow you to group database updates together in logical units. The section of an R/3 application program that bundles a set of logically-associated database operations is called an SAP LUW. Unlike a database LUW, a SAP LUW includes all of the dialog steps in a logical unit, including the database update.
Happy Reading...
shibu -
I am facing a performance problem with my current application (PL/SQL packaged procedure)
My application takes data from 4 temporary tables, does a lot of validation and
puts them into permanent tables.(updates if present else inserts)
One of the temporary tables is parent table and can have 0 or more rows in
the other tables.
I have analyzed all my tables and indexes and checked all my SQLs
They all seem to be using the indexes correctly.
There are 1.6 million records combined in all 4 tables.
I am using Oracle 8i.
How do I determine what is causing the problem and which part is taking time.
Please help.
The skeleton of the code which we have written looks like this
MAIN LOOP ( 255308 records)-- Parent temporary table
-----lots of validation-----
update permanent_table1
if sql%rowcount = 0 then
insert into permanent_table1
Loop2 (0-5 records)-- child temporary table1
-----lots of validation-----
update permanent_table2
if sql%rowcount = 0 then
insert into permanent_table2
end loop2
Loop3 (0-5 records)-- child temporary table2
-----lots of validation-----
update permanent_table3
if sql%rowcount = 0 then
insert into permanent_table3
end loop3
Loop4 (0-5 records)-- child temporary table3
-----lots of validation-----
update permanent_table4
if sql%rowcount = 0 then
insert into permanent_table4
end loop4
-- COMMIT after every 3000 records
END MAIN LOOP
Thanks
Ashwin N.Do this intead of ditching the PL/SQL.
DECLARE
TYPE NumTab IS TABLE OF NUMBER(4) INDEX BY BINARY_INTEGER;
TYPE NameTab IS TABLE OF CHAR(15) INDEX BY BINARY_INTEGER;
pnums NumTab;
pnames NameTab;
t1 NUMBER(5);
t2 NUMBER(5);
t3 NUMBER(5);
BEGIN
FOR j IN 1..5000 LOOP -- load index-by tables
pnums(j) := j;
pnames(j) := 'Part No. ' || TO_CHAR(j);
END LOOP;
t1 := dbms_utility.get_time;
FOR i IN 1..5000 LOOP -- use FOR loop
INSERT INTO parts VALUES (pnums(i), pnames(i));
END LOOP;
t2 := dbms_utility.get_time;
FORALL i IN 1..5000 -- use FORALL statement
INSERT INTO parts VALUES (pnums(i), pnames(i));
get_time(t3);
dbms_output.put_line('Execution Time (secs)');
dbms_output.put_line('---------------------');
dbms_output.put_line('FOR loop: ' || TO_CHAR(t2 - t1));
dbms_output.put_line('FORALL: ' || TO_CHAR(t3 - t2));
END;
Try this link, http://download-west.oracle.com/docs/cd/B10501_01/appdev.920/a96624/05_colls.htm#23723 -
Performance Problems on Faces Navigation Diagram and Hyperthreading query
Am I the only one having performance problems when dealing with Faces-Config Diagrams of about 35 JSPs displayed on the sheet. using Jdev 10.1.3 It's taking my workstation about a full minute and a half to update the name of an arrow. The most stressed component during this task seems to be the CPU.
And just another question has anybody investigated how is the performance of Jdev affected by either enabling or disabling hyperthreading? In my case my CPU usage manages to reach only 50%. I'm tempted to switch HT off to let JDev use all the cpu power. if that would be the case.Hello Diego,
you mentioned that you compared a BEx Query with the Web INtelligence report. Could you provide more details here ?
- what are the elements in the rows, columns and free characterisitcs in the BEx Query ?
- was the query execute as designed in the BEx Query Designer with BEx Web Reporting ?
- what are the elements in the WebIntelligence Query panel ?
thanks
Ingo -
Performance problems with DFSN, ABE and SMB
Hello,
We have identified a problem with DFS-Namespace (DFSN), Access Based Enumeration (ABE) and SMB File Service.
Currently we have two Windows Server 2008 R2 servers providing the domain-based DFSN in functional level Windows Server 2008 R2 with activated ABE.
The DFSN servers have the most current hotfixes for DFSN and SMB installed, according to http://support.microsoft.com/kb/968429/en-us and http://support.microsoft.com/kb/2473205/en-us
We have only one AD-site and don't use DFS-Replication.
Servers have 2 Intel X5550 4 Core CPUs and 32 GB Ram.
Network is a LAN.
Our DFSN looks like this:
\\contoso.com\home
Contains 10.000 Links
Drive mapping on clients to subfolder \\contoso.com\home\username
\\contoso.com\group
Contains 2500 Links
Drive mapping on clients directly to \\contoso.com\group
On \\contoso.com\group we serve different folders for teams, projects and other groups with different access permissions based on AD groups.
We have to use ABE, so that users see only accessible Links (folders)
We encounter sometimes multiple times a day enterprise-wide performance problems for 30 seconds when accessing our Namespaces.
After six weeks of researching and analyzing we were able to identify the exact problem.
Administrators create a new DFS-Link in our Namespace \\contoso.com\group with correct permissions using the following command line:
dfsutil.exe link \\contoso.com\group\project123 \\fileserver1\share\project123
dfsutil.exe property sd grant \\contoso.com\group\project123 CONTOSO\group-project123:RX protect replace
This is done a few times a day.
There is no possibility to create the folder and set the permissions in one step.
DFSN process on our DFSN-servers create the new link and the corresponding folder in C:\DFSRoots.
At this time, we have for example 2000+ clients having an active session to the root of the namespace \\contoso.com\group.
Active session means a Windows Explorer opened to the mapped drive or to any subfolder.
The file server process (Lanmanserver) sends a change notification (SMB-Protocol) to each client with an active session \\contoso.com\group.
All the clients which were getting the notification now start to refresh the folder listing of \\contoso.com\group
This was identified by an network trace on our DFSN-servers and different clients.
Due to ABE the servers have to compute the folder listing for each request.
DFS-Service on the servers doen't respond for propably 30 seconds to any additional requests. CPU usage increases significantly over this period and went back to normal afterwards. On our hardware from about 5% to 50%.
Users can't access all DFS-Namespaces during this time and applications using data from DFS-Namespace stop responding.
Side effect: Windows reports on clients a slow-link detection for \\contoso.com\home, which can be offline available for users (described here for WAN-connections: http://blogs.technet.com/b/askds/archive/2011/12/14/slow-link-with-windows-7-and-dfs-namespaces.aspx)
Problem doesn't occure when creating a link in \\contoso.com\home, because users have only a mapping to subfolders.
Currently, the problem doesn't occure also for \\contoso.com\app, because users usually don't use Windows Explorer accessing this mapping.
Disabling ABE reduces the DFSN freeze time, but doesn't solve the problem.
Problem also occurs with Windows Server 2012 R2 as DFSN-server.
There is a registry key available for clients to avoid the reponse to the change notification (NoRemoteChangeNotify, see http://support.microsoft.com/kb/812669/en-us)
This might fix the problem with DFSN, but results in other problems for the users. For example, they have to press F5 for refreshing every remote directory on change.
Is there a possibility to disable the SMB change notification on server side ?
TIA and regards,
Ralf GaudesHi,
Thanks for posting in Microsoft Technet Forums.
I am trying to involve someone familiar with this topic to further look at this issue. There might be some time delay. Appreciate your patience.
Thank you for your understanding and support.
Regards.
We
are trying to better understand customer views on social support experience, so your participation in this
interview project would be greatly appreciated if you have time.
Thanks for helping make community forums a great place.
Maybe you are looking for
-
Making Main Images Links in Photo Gallery
Hello. I've made an image gallery and it works great. What I want to do is make the main image a link to a certain part of the site. The first link works great, but then I get: TypeError: Error #1009: Cannot access a property or method of a null obje
-
Dear Experts, When I ma posting vendor invoice or vendor payment, system is not giving message u201CDown payment exist for vendoru201D when there is down payment exist for the same vendor, I have selected u201Ccommitment warningsu201D check box in OB
-
IDVD 4 Pauses During Project Opening
OMG! Someone needs to help me ASAP!!!! I have a nearly completed iDVD project with slideshows, videos, many menus, etc. However, I recently userd the iDVD_Egg to create a disk image and burn it as a test; the test work fine. However, immediately upon
-
Hi anybody I am waiting for the functionality to download a hole folder (with files/a project) at once from my cloud space (not only single files!) without syncronizing with desktop cloud - like I can do e.g. via a ZIP-file with dropbox. Anyone an id
-
Call a dynamic action automatically on a regular basis
Hi, We have a classic report on a page which we would like to refresh automaticallly every X minutes. We have done this using setInterval and $a_report however as I understand, this is unsupported functionality... So I would like to go down the dynam