Report Performance degradation

hi,
We are using around 16 entities in crm on demand R 16which includes both default as well as custom entites.
Since custom entities are not visible in the historical subject area , we decided to stick to the real time reporting.
Now the issue is , we have total 45 lakh record in these entites as a whole.We have reports where we need to retrieve the data across all the enties in one report.Intially we tested the reports with lesser no of records...the report performance was not that bad....but gradually it has degraded as we loaded more n more data over a period of time.The reports now takes approx 5-10 min and then finally diaplays an error msg.Infact after creating a report structure in Step 1 - Define Criteria......n moving to Step 2 - Create Layout it takes abnormal amount of time to display.As far as reports are concerned, we have built them using the best practice except the "Historical Subject Area Issue".
Ideally for best performance how many records should be there one entity?
What cud be the other reasons for such a performance?
We are working in a multi tenant enviroment
Edited by: Rita Negi on Dec 13, 2009 5:50 AM

Rita,
Any report built over the real-time subject areas will timeout after a period of 10 minutes. Real-time subject areas are really not suited for large reports and you'll find running them also degrades the application performance.
Things that will degrade performance are:
* Joins to other dimensions
* Custom calculations
* Number of records
* Number of fields returned
There are some things that just can't be done in real-time. I would look to remove joins from other dimensions e.g. Accounts/Contacts/Opportunities all in the same report. Apply more restrictive filters, e.g. current week/month to reduce the number of records required. Alternatively have very simple report, extract to excel and modify from there. Hopefully in R17 this will be added as a feature but it seems like you're stuck till then
Thanks
Oli @ Innoveer

Similar Messages

SCOM reports "A significant portion of the database buffer cache has been written out to the system paging file. This may result in severe performance degradation"

This was discussed here, with no resolution
http://social.technet.microsoft.com/Forums/en-US/exchange2010/thread/bb073c59-b88f-471b-a209-d7b5d9e5aa28?prof=required
I have the same issue. This is a single-purpose physical mailbox server with 320 users and 72GB of RAM. That should be plenty. I've checked and there are no manual settings for the database cache. There are no other problems with
the server, nothing reported in the logs, except for the aforementioned error (see below).
The server is sluggish. A reboot will clear up the problem temporarily. The only processes using any significant amount of memory are store.exe (using 53GB), regsvc (using 5) and W3 and Monitoringhost.exe using 1 GB each. Does anyone have
any ideas on this?
Warning ESE Event ID 906.
Information Store (1497076) A significant portion of the database buffer cache has been written out to the system paging file. This may result in severe performance degradation. See help link for complete details of possible causes. Resident cache
has fallen by 213107 buffers (or 11%) in the last 207168 seconds. Current Total Percent Resident: 79% (1574197 of 1969409 buffers)

Brian,
We had this event log entry as well which SCOM picked up on, and 10 seconds before it the Forefront Protection 2010 for Exchange updated all of its engines.
We are running Exchange 2010 SP2 RU3 with no file system antivirus (the boxes are restricted and have UAC turned on as mitigations). We are running the servers primarily as Hub Transport servers with 16GB of RAM, but they do have the mailbox role installed
for the sole purpose of serving as our public folder servers.
So we theroized the STORE process was just grabbing a ton of RAM, and occasionally it was told to dump the memory so the other processes could grab some - thus generating the alert. Up until last night we thought nothing of it, but ~25 seconds after the
cache flush to paging file, we got the following alert:
Log Name:      Application
Source:        MSExchangeTransport
Date:          8/2/2012 2:08:14 AM
Event ID:      17012
Task Category: Storage
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      HTS1.company.com
Description:
Transport Mail Database: The database could not allocate memory. Please close some applications to make sure you have enough memory for Exchange Server. The exception is Microsoft.Exchange.Isam.IsamOutOfMemoryException: Out of Memory (-1011)
   at Microsoft.Exchange.Isam.JetInterop.CallW(Int32 errFn)
   at Microsoft.Exchange.Isam.JetInterop.MJetOpenDatabase(MJET_SESID sesid, String file, String connect, MJET_GRBIT grbit, MJET_WRN& wrn)
   at Microsoft.Exchange.Isam.JetInterop.MJetOpenDatabase(MJET_SESID sesid, String file, MJET_GRBIT grbit)
   at Microsoft.Exchange.Isam.JetInterop.MJetOpenDatabase(MJET_SESID sesid, String file)
   at Microsoft.Exchange.Isam.Interop.MJetOpenDatabase(MJET_SESID sesid, String file)
   at Microsoft.Exchange.Transport.Storage.DataConnection..ctor(MJET_INSTANCE instance, DataSource source).
Followed by:
Log Name:      Application
Source:        MSExchangeTransport
Date:          8/2/2012 2:08:15 AM
Event ID:      17106
Task Category: Storage
Level:         Information
Keywords:      Classic
User:          N/A
Computer:      HTS1.company.com
Description:
Transport Mail Database: MSExchangeTransport has detected a critical storage error, updated the registry key (SOFTWARE\Microsoft\ExchangeServer\v14\Transport\QueueDatabase) and as a result, will attempt self-healing after process restart.
Log Name:      Application
Source:        MSExchangeTransport
Date:          8/2/2012 2:13:50 AM
Event ID:      17102
Task Category: Storage
Level:         Warning
Keywords:      Classic
User:          N/A
Computer:      HTS1.company.com
Description:
Transport Mail Database: MSExchangeTransport has detected a critical storage error and has taken an automated recovery action. This recovery action will not be repeated until the target folders are renamed or deleted. Directory path:E:\EXCHSRVR\TransportRoles\Data\Queue
is moved to directory path:E:\EXCHSRVR\TransportRoles\Data\Queue\Queue.old.
So it seems as if the Forefront Protection 2010 for Exchange inadvertently trigger the cache flush which didn't appear to happen quick or thuroughly enough for the transport service to do what it needed to do, so it freaked out and performed the subsequent
actions.
Do you have any ideas on how to prevent this 906 warning, which cascaded into a transport service outage?
Thanks!

Performance degradation: unfetched field [PublishingPageContent] caused extra roundtrip

Hi All,
   I am facing some serious application pool crash in one of my customer's Production site SharePoint servers. The Application Error logs in the event Viewer says -
Faulting application name: w3wp.exe, version: 7.5.7601.17514, time stamp: 0x4ce7afa2
Faulting module name: ntdll.dll, version: 6.1.7601.17514, time stamp: 0x4ce7c8f9
Exception code: 0xc0000374
Fault offset: 0x00000000000c40f2
Faulting process id: 0x1414
Faulting application start time: 0x01ce5edada76109d
Faulting application path: c:\windows\system32\inetsrv\w3wp.exe
Faulting module path: C:\Windows\SYSTEM32\ntdll.dll
Report Id: 5a69ec1e-cace-11e2-9be2-441ea13bf8be
At the same time the SharePoint ULS logs says -
1)
06/13/2013 03:44:29.53 w3wp.exe (0x0808)                       0x2DF0 SharePoint Foundation
        General                                8e2s
Medium              Unknown SPRequest error occurred. More information: 0x80070005      8b343224-4aa6-490c-8a2a-ce06ac160773
06/13/2013 03:44:35.03 w3wp.exe (0x0808)                       0x2DF0 SharePoint Foundation
        General
8e25      Medium              Failed to look up string with key "FSAdmin_SiteSettings_UserContextManagement_ToolTip", keyfile Microsoft.Office.Server.Search.
8b343224-4aa6-490c-8a2a-ce06ac160773
06/13/2013 03:44:35.03 w3wp.exe (0x0808)                       0x2DF0 SharePoint Foundation
        General                                8l3c
Medium              Localized resource for token 'FSAdmin_SiteSettings_UserContextManagement_ToolTip' could not be found for file with path: "C:\Program Files\Common Files\Microsoft Shared\Web
Server Extensions\14\Template\Features\SearchExtensions\ExtendedSearchAdminLinks.xml".              8b343224-4aa6-490c-8a2a-ce06ac160773
2)
06/13/2013 03:44:29.01 w3wp.exe (0x0808)                       0x2DF0 SharePoint Foundation
        Web Parts
emt4     High       Error initializing Safe control - Assembly:Microsoft.Office.SharePoint.ClientExtensions, Version=14.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c TypeName: Microsoft.Office.SharePoint.ClientExtensions.Publishing.TakeListOfflineRibbonControl
Error: Could not load type 'Microsoft.Office.SharePoint.ClientExtensions.Publishing.TakeListOfflineRibbonControl' from assembly 'Microsoft.Office.SharePoint.ClientExtensions, Version=14.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c'.
8b343224-4aa6-490c-8a2a-ce06ac160773
06/13/2013 03:44:29.50 w3wp.exe (0x0808)
0x2DF0 SharePoint Foundation                 Logging Correlation Data
xmnv     Medium              Site=/    8b343224-4aa6-490c-8a2a-ce06ac160773
3)
06/13/2013 03:43:59.67 w3wp.exe (0x263C)                       0x24D8 SharePoint Foundation
        Performance                   9fx9
Medium              Performance degradation: unfetched field [PublishingPageContent] caused extra roundtrip.     at Microsoft.SharePoint.SPListItem.GetValue(SPField fld,
Int32 columnNumber, Boolean bRaw, Boolean bThrowException)     at Microsoft.SharePoint.SPListItem.GetValue(String strName, Boolean bThrowException)     at Microsoft.SharePoint.SPListItem.get_Item(String fieldName)
at Microsoft.SharePoint.WebControls.BaseFieldControl.get_ItemFieldValue()     at Microsoft.SharePoint.Publishing.WebControls.RichHtmlField.RenderFieldForDisplay(HtmlTextWriter output)     at Microsoft.SharePoint.WebControls.BaseFieldControl.Render(HtmlTextWriter
output)     at Microsoft.SharePoint.Publishing.WebControls.BaseRichField.Render(HtmlTextWriter output)     at Microsoft.SharePoint.Publishing.WebControls.RichHtmlField.R...
b8d0b8ca-8386-441f-8fce-d79fe72556e1
06/13/2013 03:43:59.67*               w3wp.exe (0x263C)
0x24D8 SharePoint Foundation                 Performance
9fx9       Medium              ...ender(HtmlTextWriter output)     at System.Web.UI.Control.RenderChildrenInternal(HtmlTextWriter writer, ICollection
children)     at System.Web.UI.Control.RenderChildrenInternal(HtmlTextWriter writer, ICollection children)     at System.Web.UI.Control.RenderChildrenInternal(HtmlTextWriter writer, ICollection children)
at System.Web.UI.HtmlControls.HtmlContainerControl.Render(HtmlTextWriter writer)     at System.Web.UI.Control.RenderChildrenInternal(HtmlTextWriter writer, ICollection children)     at System.Web.UI.HtmlControls.HtmlForm.RenderChildren(HtmlTextWriter
writer)     at System.Web.UI.HtmlControls.HtmlForm.Render(HtmlTextWriter output)     at System.Web.UI.HtmlControls.HtmlForm.RenderControl(HtmlTextWriter writer)     at System.Web.UI.Control.RenderChildrenInternal(HtmlTextWrit...
b8d0b8ca-8386-441f-8fce-d79fe72556e1
06/13/2013 03:43:59.67*               w3wp.exe (0x263C)
0x24D8 SharePoint Foundation                 Performance
9fx9       Medium              ...er writer, ICollection children)     at System.Web.UI.Control.RenderChildrenInternal(HtmlTextWriter writer,
ICollection children)     at System.Web.UI.Page.Render(HtmlTextWriter writer)     at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)
at System.Web.UI.Page.ProcessRequest(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)     at System.Web.UI.Page.ProcessRequest()     at System.Web.UI.Page.ProcessRequest(HttpContext context)
at Microsoft.SharePoint.Publishing.TemplateRedirectionPage.ProcessRequest(HttpContext context)     at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()     at System.Web.HttpApplication.ExecuteStep(IExecutionSte...
b8d0b8ca-8386-441f-8fce-d79fe72556e1
06/13/2013 03:43:59.67*               w3wp.exe (0x263C)
0x24D8 SharePoint Foundation                 Performance
9fx9       Medium              ...p step, Boolean& completedSynchronously)     at System.Web.HttpApplication.PipelineStepManager.ResumeSteps(Exception
error)     at System.Web.HttpApplication.BeginProcessRequestNotification(HttpContext context, AsyncCallback cb)     at System.Web.HttpRuntime.ProcessRequestNotificationPrivate(IIS7WorkerRequest wr, HttpContext context)
at System.Web.Hosting.PipelineRuntime.ProcessRequestNotificationHelper(IntPtr managedHttpContext, IntPtr nativeRequestContext, IntPtr moduleData, Int32 flags)     at System.Web.Hosting.PipelineRuntime.ProcessRequestNotification(IntPtr managedHttpContext,
IntPtr nativeRequestContext, IntPtr moduleData, Int32 flags)     at System.Web.Hosting.PipelineRuntime.ProcessRequestNotificationHelper(IntPtr managedHttpContext, IntPtr nativeRequestContext, IntPtr module...
b8d0b8ca-8386-441f-8fce-d79fe72556e1
06/13/2013 03:43:59.67*               w3wp.exe (0x263C)
0x24D8 SharePoint Foundation                 Performance
9fx9       Medium              ...Data, Int32 flags)     at System.Web.Hosting.PipelineRuntime.ProcessRequestNotification(IntPtr managedHttpContext,
IntPtr nativeRequestContext, IntPtr moduleData, Int32 flags)                b8d0b8ca-8386-441f-8fce-d79fe72556e1
06/13/2013 03:43:59.67 w3wp.exe (0x263C)
0x24D8 SharePoint Foundation                 Performance                   g4zd
High       Performance degradation: note field [PublishingPageContent] was not in demoted fields.          b8d0b8ca-8386-441f-8fce-d79fe72556e1
Anybody has any idea whats going on? I need to fix this ASAP as we are suppose to go live in next few days.
Soumalya

Hello Soumalya,
Do you have an update on your issue? We are actually experiencing a similar issue at a new customer.
- Dennis | Netherlands | Blog |
Twitter

Performance Degradation - High fetches and Prses

Hello,
My analysis on a particular job trace file drew my attention towards:
1) High rate of Parses instead of Bind variables usage.
2) High fetches and poor number/ low number of rows being processed
Please let me kno as to how the performance degradation can be minimised, Perhaps the high number of SQL* Net Client wait events may be due to multiple fetches and transactions with the client.
EXPLAIN PLAN FOR SELECT /*+ FIRST_ROWS (1) */ * FROM SAPNXP.INOB
WHERE MANDT = :A0
AND KLART = :A1
AND OBTAB = :A2
AND OBJEK LIKE :A3 AND ROWNUM <= :A4;
call     count       cpu    elapsed       disk      query    current        rows
Parse      119      0.00       0.00          0          0          0           0
Execute    239      0.16       0.13          0          0          0           0
Fetch      239   2069.31    2127.88          0   13738804          0           0
total      597   2069.47    2128.01          0   13738804          0           0
PLAN_TABLE_OUTPUT
Plan hash value: 1235313998
| Id | Operation                    | Name   | Rows | Bytes | Cost (%CPU)| Time     |
|   0 | SELECT STATEMENT             |        |     2 |   268 |     1   (0)| 00:00:01 |
|* 1 | COUNT STOPKEY               |        |       |       |            |          |
|* 2 |   TABLE ACCESS BY INDEX ROWID| INOB   |     2 |   268 |     1   (0)| 00:00:01 |
|* 3 |    INDEX SKIP SCAN           | INOB~2 | 7514 |       |     1   (0)| 00:00:01 |
Predicate Information (identified by operation id):
   1 - filter(ROWNUM<=TO_NUMBER(:A4))
   2 - filter("OBJEK" LIKE :A3 AND "KLART"=:A1)
   3 - access("MANDT"=:A0 AND "OBTAB"=:A2)
       filter("OBTAB"=:A2)
18 rows selected.
SQL> SELECT INDEX_NAME,TABLE_NAME,COLUMN_NAME FROM DBA_IND_COLUMNS WHERE INDEX_OWNER='SAPNXP' AND INDEX_NAME='INOB~2';
INDEX_NAME      TABLE_NAME                     COLUMN_NAME
INOB~2          INOB                           MANDT
INOB~2          INOB                           CLINT
INOB~2          INOB                           OBTAB
Is it possible to Maximise the rows/fetch
call     count       cpu    elapsed       disk      query    current        rows
Parse      163      0.03       0.00          0          0          0           0
Execute    163      0.01       0.03          0          0          0           0
Fetch   174899     55.26      59.14          0    1387649          0     4718932
total   175225     55.30      59.19          0    1387649          0     4718932
Misses in library cache during parse: 0
Optimizer mode: ALL_ROWS
Parsing user id: 27
Rows     Row Source Operation
28952 TABLE ACCESS BY INDEX ROWID EDIDC (cr=8505 pr=0 pw=0 time=202797 us)
28952   INDEX RANGE SCAN EDIDC~1 (cr=1457 pr=0 pw=0 time=29112 us)(object id 202995)
Elapsed times include waiting on following events:
Event waited on                             Times   Max. Wait Total Waited
----------------------------------------   Waited ---------- ------------
SQL*Net message to client                  174899        0.00          0.16
SQL*Net more data to client                155767        0.01          5.69
SQL*Net message from client                174899        0.11        208.21
latch: cache buffers chains                     2        0.00          0.00
latch free                                      4        0.00          0.00
********************************************************************************

user4566776 wrote:
My analysis on a particular job trace file drew my attention towards:
1) High rate of Parses instead of Bind variables usage.
But if you look at the text you are using bind variables.
The first query is executed 239 times - which matches the 239 fetches. You cut off some of the useful information from the tkprof output, but the figures show that you're executing more than once per parse call. The time is CPU time spent using a bad execution plan to find no data -- this looks like a bad choice of index, possibly a side effect of the first_rows(1) hint.
2) High fetches and poor number/ low number of rows being processedThe second query is doing a lot of fetches because in 163 executions it is fetching 4.7 million rows at roughly 25 rows per fetch. You might improve performance a little by increasing the array fetch size - but probably not by more than a factor of 2.
You'll notice that even though you record 163 parse calls for the second statement the number of " Misses in library cache during parse" is zero - so the parse calls are pretty irrelevant, the cursor is being re-used.
Regards
Jonathan Lewis
http://jonathanlewis.wordpress.com
http://www.jlcomp.demon.co.uk
To post code, statspack/AWR report, execution plans or trace files, start and end the section with the tag {noformat}{noformat} (lowercase, curly brackets, no spaces) so that the text appears in fixed format.
"Science is more than a body of knowledge; it is a way of thinking"
Carl Sagan

Performance degradation encountered while running BOE in clustered set up

Problem Statement:
We have a clustered BOE set up in Production with 2 CMS servers (named boe01 and boe02) . Mantenix application (Standard J2EE application in a clustered set up) points to these BOE services hosted on virtual machines to generate reports. As soon as BOE services on both boe01 and boe02 are up and running , performance degradation is observed i.e (response times varies from 7sec to 30sec) .
The same set up works fine when BOE services on boe02 is turned off i.e only boe01 is up and running.No drastic variation is noticed.
BOE Details : SAP BusinessObjects environment XIR2 SP3 running on Windows 2003 Servers.(Virtual machines)
Possible Problem Areas as per our analysis
1) Node 2 Virtual Machine Issue:
This currently being part of the Production infrastructure, any problem assessment testing is not possible.
2) BOE Configuration Issue
Comparison report to check the build between BOE 01 and BOE 02 - Support team has confirmed no major installation differences apart from a minor Operating System setting difference.Question being is there some configuration/setting that we are missing ?
3) Possible BOE Cluster Issue:
Tests in staging environment ( with a similar clustered BOE setup ) have proved inconclusive.
We require your help in
- Root cause Analysis for this problem.
- Any troubleshooting action henceforth.
Another observation from our Weblogic support engineers for the above set up which may or may not be related to the problem is mentioned below.
When the services on BOE_2 are shutdown and we try to fetch a particular report from BOE_1 (Which is running), the following WARNING/ERROR comes up:-
07/09/2011 10:22:26 AM EST> <WARN> <com.crystaldecisions.celib.trace.d.if(Unknown Source)> - getUnmanagedService(): svc=BlockingReportSourceRepository,spec=aps<BOE_1> ,cluster:@BOE_OLTP, kind:cacheserver, name:<BOE_2>.cacheserver.cacheserver, queryString:null, m_replaceable:true,uri=osca:iiop://<BOE_1>;SI_SESSIONID=299466JqxiPSPUTef8huXO
com.crystaldecisions.thirdparty.org.omg.CORBA.TRANSIENT: attempt to establish connection failed: java.net.ConnectException: Connection timed out: connect minor code: 0x4f4f0001 completed: No
     at com.crystaldecisions.thirdparty.com.ooc.OCI.IIOP.Connector_impl.connect(Connector_impl.java:150)
     at com.crystaldecisions.thirdparty.com.ooc.OB.GIOPClient.createTransport(GIOPClient.java:233)
     at com.crystaldecisions.thirdparty.com.ooc.OB.GIOPClientWorkersPool.next(GIOPClientWorkersPool.java:122)
     at com.crystaldecisions.thirdparty.com.ooc.OB.GIOPClient.getWorker(GIOPClient.java:105)
     at com.crystaldecisions.thirdparty.com.ooc.OB.GIOPClient.startDowncall(GIOPClient.java:409)
     at com.crystaldecisions.thirdparty.com.ooc.OB.Downcall.preMarshalBase(Downcall.java:181)
     at com.crystaldecisions.thirdparty.com.ooc.OB.Downcall.preMarshal(Downcall.java:298)
     at com.crystaldecisions.thirdparty.com.ooc.OB.DowncallStub.preMarshal(DowncallStub.java:250)
     at com.crystaldecisions.thirdparty.com.ooc.OB.DowncallStub.setupRequest(DowncallStub.java:530)
     at com.crystaldecisions.thirdparty.com.ooc.CORBA.Delegate.request(Delegate.java:556)
     at com.crystaldecisions.thirdparty.org.omg.CORBA.portable.ObjectImpl._request(ObjectImpl.java:118)
     at com.crystaldecisions.enterprise.ocaframework.idl.ImplServ._OSCAFactoryStub.getServices(_OSCAFactoryStub.java:806)
     at com.crystaldecisions.enterprise.ocaframework.ServiceMgr.do(Unknown Source)
     at com.crystaldecisions.enterprise.ocaframework.ServiceMgr.a(Unknown Source)
     at com.crystaldecisions.enterprise.ocaframework.ServiceMgr.getUnmanagedService(Unknown Source)
     at com.crystaldecisions.enterprise.ocaframework.AbstractStubHelper.getService(Unknown Source)
     at com.crystaldecisions.enterprise.ocaframework.e.do(Unknown Source)
     at com.crystaldecisions.enterprise.ocaframework.o.try(Unknown Source)
     at com.crystaldecisions.enterprise.ocaframework.o.a(Unknown Source)
     at com.crystaldecisions.enterprise.ocaframework.o.a(Unknown Source)
     at com.crystaldecisions.enterprise.ocaframework.p.a(Unknown Source)
     at com.crystaldecisions.enterprise.ocaframework.ServiceMgr.getManagedService(Unknown Source)
     at com.crystaldecisions.sdk.occa.managedreports.ps.internal.a$a.getService(Unknown Source)
     at com.crystaldecisions.enterprise.ocaframework.e.do(Unknown Source)
     at com.crystaldecisions.enterprise.ocaframework.o.try(Unknown Source)
     at com.crystaldecisions.enterprise.ocaframework.o.a(Unknown Source)
     at com.crystaldecisions.enterprise.ocaframework.o.a(Unknown Source)
     at com.crystaldecisions.enterprise.ocaframework.p.a(Unknown Source)
We see the above warning coming 2 or 3 times before the request is processed and then we see the report. We have checked our config's for the cluster but didn't find anything concrete.
Is this a normal behavior of the software or can we optimize it?
Any assistance that you can provide would be great

Rahul,
I have exactly the same problem running BO 3.1 SP3 in a 2 machine cluster on AIX. Exact same full install on both machines. When I take down one of the machines the performance is much better.
An example of the problem now is that when i run the command ./ccm.sh -display -username administrator -password xxx on the either box when they are both up and running, I sometimes receive a timeout error (over 15mins)
If I run SQLplus direct on the boxes to the CMS DB then the response is instant. Tnspings of course shows no problems
When I bring down one of the machines and run the command ./ccm.sh -display again then this brings back results in less than a minute...
I am baffled as to the problem so was wondering if you found anything from your end
Cheers
Chris

Performance Degradation of new Servers

Hi All,
We are experiencing massive performance degradation on production when the system is under heavy use. It seems to be at its worst around month end. The worst effected transactions are the Cost / Profit centre 'line item' reports using RCOPCA02 (and other similar programs).
We have the message server running on the database instance as well as 2x App servers. The Msg Srv and One of the App servers are very similar builds
4x AMD Opteron 875
16gb RAM (10gb Pagefile)
Both running Win Server 2003
We're using MS SQL 2000
The other message server is a bit weaker but has been around for some time (few years) and hasn't caused any issues.
We have recently moved the Msg Server from an old (much weaker) server to the new build and since then seem to have performance issues. Initially after the move we had issues with the number of Page Table Entries (where down at about 6000-8000). Using the /3GB /USERVA=2900 switches we have this up to about 49,000.
If anyone has had a similar experience or could offer some assistance it would be much appreciated!!!
Cheers,
Kye

While investigating a different issue we've found that we have 3 servers that share the same set of disks on the SAN. Two of these where Message Servers (R/3 and BW) when both of these where running the 'Average disk queue Length' was at times reaching 400! (should be around 1-3).
We've moved one of these instances back to DR which has relieved some of the pressure from the disks.
We have also added another index to the GLPCA table (which was causing most of the problems).

Database migrated from Oracle 10g to 11g Discoverer report performance issu

Hi All,
We are now getting issue in Discoverer Report performance as the report is keep on running when database got upgrade from 10g to 11g.
In database 10g the report is working fine but the same report is not working fine in 11g.
The query i have changed as I have passed the date format TO_CHAR("DD-MON-YYYY" and removed the NVL & TRUNC function from the existing query.
The report is now working fine in Database 11g backhand but when I am using the same query in Discoverer it is not working and report is keep on running.
Please advise.
Regards,

Pl post exact OS, database and Discoverer versions. After the upgrade, have statistics been updated ? Have you traced the Discoverer query to determine where the performance issue is ?
How To Find Oracle Discoverer Diagnostic and Tracing Guides [ID 290658.1]
How To Enable SQL Tracing For Discoverer Sessions [ID 133055.1]
Discoverer 11g: Performance degradation after Upgrade to Database 11g [ID 1514929.1]
HTH
Srini

Can System performance degraded by Z development

Hi,
Can system performance degrade by more z development? Is there any idle
ratio with standard system/reports to Z development in % terms?
Regards,
Shail

Z programs are just programs not written by SAP, but added by the customer.
The performance of the Z programs depends on the experience of the Z programmer.
There is no ideal ratio, I just depends how much must be programmed different from the standard.
And oyu should also keep in mind, that if a Z-program replaces a standard program that it will have broader/narrower functionality, so it will need a different amount of time.
There is no optimum defined, you must find out whether something can be done better.
Siegfried

JDBC, SQL*Net wait interface, performance degradation on 10g vs. 9i

Hi All,
I came across performance issue that I think results from mis-configuration of something between Oracle and JDBC. The logic of my system executes 12 threads in java. Each thread performs simple 'select a,b,c...f from table_xyz' on different tables. (so I have 12 different tables with cardinality from 3 to 48 millions and one working thread per table).
In each thread I'm creating result set that is explicitly marked as forward_only, transaction is set read only, fetch size is set to 100000 records. Java logic processes records in standard while(rs.next()) {...} loop.
I'm experiencing performance degradation between execution on Oracle 9i and Oracle 10g of the same java code, on the same machine, on the same data. The difference is enormous, 9i execution takes 26 hours while 10g execution takes 39 hours.
I have collected statspack for 9i and awr report for 10g. Below I've enclosed top wait events for 9i and 10g
===== 9i ===================
Avg
Total Wait wait Waits
Event Waits Timeouts Time (s) (ms) /txn
db file sequential read 22,939,988 0 6,240 0 0.7
control file parallel write 6,152 0 296 48 0.0
SQL*Net more data to client 2,877,154 0 280 0 0.1
db file scattered read 26,842 0 91 3 0.0
log file parallel write 3,528 0 83 23 0.0
latch free 94,845 0 50 1 0.0
process startup 93 0 5 50 0.0
log file sync 34 0 2 46 0.0
log file switch completion 2 0 0 215 0.0
db file single write 9 0 0 33 0.0
control file sequential read 4,912 0 0 0 0.0
wait list latch free 15 0 0 12 0.0
LGWR wait for redo copy 84 0 0 1 0.0
log file single write 2 0 0 18 0.0
async disk IO 263 0 0 0 0.0
direct path read 2,058 0 0 0 0.0
slave TJ process wait 1 1 0 12 0.0
===== 10g ==================
Avg
%Time Total Wait wait Waits
Event Waits -outs Time (s) (ms) /txn
db file scattered read 268,314 .0 2,776 10 0.0
SQL*Net message to client 278,082,276 .0 813 0 7.1
io done 20,715 .0 457 22 0.0
control file parallel write 10,971 .0 336 31 0.0
db file parallel write 15,904 .0 294 18 0.0
db file sequential read 66,266 .0 257 4 0.0
log file parallel write 3,510 .0 145 41 0.0
SQL*Net more data to client 2,221,521 .0 102 0 0.1
SGA: allocation forcing comp 2,489 99.9 27 11 0.0
log file sync 564 .0 23 41 0.0
os thread startup 176 4.0 19 106 0.0
latch: shared pool 372 .0 11 29 0.0
latch: library cache 537 .0 5 10 0.0
rdbms ipc reply 57 .0 3 49 0.0
log file switch completion 5 40.0 3 552 0.0
latch free 4,141 .0 2 0 0.0
I put full blame for the slowdown on SQL*Net message to client wait event. All I could find about this event is that it is a network related problem. I assume it would be true if database and client were on different machines.. However in my case they are on them very same machine.
I'd be very very grateful if someone could point me in the right direction, i.e. give a hint what statistics should I analyze further? what might cause this event to appear? why probable cause (that is said be outside db) affects only 10g instance?
Thanks in advance,
Rafi.

Hi Steven,
Thanks for the input. It's a fact that I did not gather statistics on my tables. My understanding is that statistics are useful for queries more complex than simple select * from table_xxx. In my case tables don't have indexes. There's no filtering condition as well. Full table scan is what I actually want as all software logic is inside the java code.
Explain plans are as follows:
======= 10g ================================
PLAN_TABLE_OUTPUT
Plan hash value: 1141003974
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
| 0 | SELECT STATEMENT | | 1 | 259 | 2 (0)| 00:00:01 |
| 1 | TABLE ACCESS FULL| xxx | 1 | 259 | 2 (0)| 00:00:01 |
In sqlplus I get:
SQL> set autotrace traceonly explain statistics;
SQL> select * from xxx;
36184384 rows selected.
Elapsed: 00:38:44.35
Execution Plan
Plan hash value: 1141003974
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
| 0 | SELECT STATEMENT | | 1 | 259 | 2 (0)| 00:00:01 |
| 1 | TABLE ACCESS FULL| xxx | 1 | 259 | 2 (0)| 00:00:01 |
Statistics
1 recursive calls
0 db block gets
3339240 consistent gets
981517 physical reads
116 redo size
26535700 bytes received via SQL*Net from client
2412294 SQL*Net roundtrips to/from client
0 sorts (memory)
0 sorts (disk)
36184384 rows processed
======= 9i =================================
PLAN_TABLE_OUTPUT
| Id | Operation | Name | Rows | Bytes | Cost |
| 0 | SELECT STATEMENT | | | | |
| 1 | TABLE ACCESS FULL | xxx | | | |
Note: rule based optimization
In sqlplus I get:
SQL> set autotrace traceonly explain statistics;
SQL> select * from xxx;
36184384 rows selected.
Elapsed: 00:17:43.06
Execution Plan
0 SELECT STATEMENT Optimizer=CHOOSE
1 0 TABLE ACCESS (FULL) OF 'xxx'
Statistics
0 recursive calls
1 db block gets
3306118 consistent gets
957515 physical reads
100 redo size
23659424 bytes sent via SQL*Net to client
26535867 bytes received via SQL*Net from client
2412294 SQL*Net roundtrips to/from client
0 sorts (memory)
0 sorts (disk)
36184384 rows processed
Thanks for pointing me difference in table scans. I infer that 9i is doing single-block full table scan (db file sequential read) while 10g is using multi-block full table scan (db file scattered read).
I now have theory that 9i is faster because sequential reads use continuous buffer space while scattered reads use discontinuous buffer space. Since I'm accessing data 'row by row' in jdbc 10g might have an overhead in providing data from discontinuous buffer space. This overhead shows itself as SQL*Net message to client wait. Is that making any sense?
Is there any way I could force 10g (i.e. with hint) to use sequential reads instead of scattered reads for full table scan?
I'll experiment with FTS tuning in 10g by enabling automatic multi-block reads tuning (i.e. db_file_multiblock_read_count=0 instead of 32 as it is now). I'll also check if response time improves after statistics are gathered.
Please advice if you have any other ideas.
Thanks & regards,
Rafi.

Report Performance - timeout short dump

Hello Experts,
i am trying to improve the performace of a report that was developed long time ago.
Issues i found:
1. The report has many select...Endselect combinations, and selects inside the loop statements.
2. Most of the selects have the addition 'into corresponding fields of' for selecting a few fields, without the table addition.
3. Also few selects have the 'select * from' syntax.
data: begin of itab occurs 0,
          f1,
          f2
          f3.....
          fn,
        end of itab.
Ex: loop at itab.
         select f1 f2 f3 from table1
               into corresponding fields of itab1.
           collect itab1.
         endselect.
          select f4 f5 from table2
              into corresponding fields of itab2.
           endselect.
      endloop.
All this leeds to performace issues.
i have checked ST05, and i have got the details of the error.
My question is which one of the reasons i mentioned above are a major factor in delaying the report performance?
Which one of the above should i conetrate first to get the long runtime down? My goal is to keep my changes to the minimum and improve the performance. Please advise.

> My question is which one of the reasons i mentioned above are a major factor in delaying the report
> performance?
Don't ask people for guesses, if you can see the facts!
Run the SQL Trace several times, and use go to 'Trace List' -> 'Summarize Trace by SQL Statement'
=> Shows you total DB time and time per statement (all executions), the problems are on top of the list.
Check ABAP, detail, and explain!
Read more here:
/people/siegfried.boes/blog/2007/09/05/the-sql-trace-st05-150-quick-and-easy
Siegfried

Report performance while creating report on BEx

All all!
I am creating a report on BOE 4.0 on top of BEx connection as a source. I have developed reports on top of universe in the past and i know that if we keep calculations on reporting end it hampers the report performance. Is this the same case with BEx? if we are following the best practices is it ok to say that we should keep all heavy calculations/ aggregation on BEx or backend for better report performance.
Can you guys please provide your opinion based on your experiance and knowledge. Any feedbacks will help! Thanks.

Hi,
Definitely best-practice to delegate a maximum of CKF to the Cube where possilble, put RKF in the BEx query, and Filters too.
also, add Default Values to your Variables (this will speed up generation of the bics transient universe)
also, since Patch2.10, we are seeing some significant performance improvements reducing 'document initialization' and 'time to prompts' by up to 50% (step such as these often took 1.5 minutes, even on sized systems)
Also, make sure you have BW corrections like this implemented: 1593802 Performance optimization when loading query views
In the BusinessObjects landscape - especially with BI 4.0 - it's all about Sizing and Tuning . Here is your bible the 'sizing companion' guide : http://service.sap.com/~form/sapnet?_SHORTKEY=01100035870000738725&_OBJECT=011000358700000307202011E
Pay particular attention to BICSChunkSize registry settings
Also, the -Xmx JVM Heap Size for the Adaptive Processing Server that is running the DSL_Bridge service.
Regards,
H

Report Performance for GL item level report.

Hi All,
I have a requirements to get GL line items
report based on GL Line items so have created data model like 0FI_GL_4->DSO-> cube and tested everything is fine but when execute in production the report performance is very bad.
Report contains document number, GL act, comp.code, posting date objects.
I have decided to do as follows to improve reporting performance
· Create Aggregate on Document, GL characteristic
· Compression.
Can I do aggregates 1st then do the compression.
Please let me know if I missing out anything.
Regards,
Naani.

Hi Naani,
First fill the Aggrigates then do Compression,run SAP_INFOCUBE_DESIGN Check the size of Dimension maintain Line item, High cordinality to the dimension, Set Cahe for query in RSRT,
Try to reduce Novigational Attr in report. Below document may help you.
http://www.sdn.sap.com/irj/scn/go/portal/prtroot/docs/library/uuid/6071ed5f-1057-2e10-deb6-d3426fec0219?QuickLink=index&…
Regards,
Jagadeesh

Bad reporting performance after compressing infocubes

Hi,
as I learned, we should compress requests in our infocubes. And since we're using Oracle 9.2.0.7 as database, we can use partitioning on the E-facttable to still increase reporting performance. So far all theory...
After getting complaints about worse reporting performance we tested this theory. I created four InfoCubes (same datamodel):
A - no compression
B - compression, but no partitioning
C - compression, one partition for each year
D - compression, one partition for each month
After loading 135 requests and compressing the cubes, we get this amount of data:
15.6 million records in each cube
Cube A: 135 partitions (one per request)
Cube B: 1 partition
Cube C: 8 partitions
Cube D: 62 partitions
Now I copied one query on each cube and with this I tested the performance (transaction rsrt, without aggregates and cache, comparing the database time QTIMEDB and DMTDBBASIC). In the query I selected always one month, some hierarchy nodes and one branch.
With this selection on each cube, I expected that cube D would be fastest, since we only have one (small) partition with relevant data. But reality shows some different picture:
Cube A is fastest with an avg. time of 8.15, followed by cube B (8.75, +8%), cube C (10.14, +24%) and finally cube D (26.75, +228%).
Does anyone have an idea what's going wrong? Are there same db-parameters to "activate" the partitioning for the optimizer? Or do we have to do some other customizing?
Thanks for your replies,
Knut

Hi Björn,
thanks for your hints.
1. after compressing the cubes I refreshed the statistics in the infocube administration.
2. cube C ist partitioned using 0CALMONTH, cube D ist partitioned using 0FISCPER.
3. here we are: alle queries are filtered using 0FISCPER. Therefor I could increase the performance on cube C, but still not on D. I will change the query on cube C and do a retest at the end of this week.
4. loaded data is joined from 10 months. The records are nearly equally distributed over this 10 months.
5. partitioning was done for the period 01.2005 - 14.2009 (01.2005 - 12.2009 on cube C). So I have 5 years - 8 partitions on cube C are the result of a slight miscalculation on my side: 5 years + 1 partion before + 1 partition after => I set max. no. of partitions on 7, not thinking of BI, which always adds one partition for the data after the requested period... So each partition on cube C does not contain one full year but something about 8 months.
6. since I tested the cubes one after another without much time between, the system load should be nearly the same (on top: it was a friday afternoon...). Our BI is clustered with several other SAP installations on a big unix server, so I cannot see the overall system load. But I did several runs with each query and the mentioned times are average times over all runs - and the average shows the same picture as the single runs (cube A is always fastest, cube D always the worst).
Any further ideas?
Greets,
Knut

Item/Drill Report Performance hinderance

I am having a problem with report performance. I have a report that I have to have 5 drop down menus on top of the report. It seems the more drop down menus I add, the slower the response time when the report is actually navigated. One of the drop downs has over 1,000 options, but the other 4 drop down menus have 4-5 options. Is there a way to improve performance?

And this is different from yesterday how?
Please help, Discoverer Performance.
Russ provided a few possible reasons, and asked for a bit of detail. Instead of asking the same question again, respond to Russ and others, and provide a bit more information to how things are set up.

Crystal Report Performance for dbf files.

We have a report which was designed 5 -6 years ago. This report has 4 linked word doc and dbf file as datasource. This report also as 3 subreports. The size of field in dbf is 80 chars and couple of field are memo field. The report performance was excellent before we migrated the crystall report to 2008. After CR2008 the system changed and it is suddenly really slow. We have not change our reports so much it should have an influence on performance. When the user presses the preview button on printing tool window the control is transferred to Crystal. Something has happened inside black box of Crystal ( IMO ). the dll we have are crdb_p2bxbse.dll 12.00.0000.0549 . The issues seems to be of xbase driver (not possible to use latest version of crdb_p2bxbse.dll and dbase files with memo fields).

Hi Kamlesh,
Odd that the word doc is opened before the RPT, I would think that the RPT would need to be opened first so it sees that the doc also needs to be opened. Once it's been loaded then the connection can be closed, CR embeds the DOC in the RPT so the original is no longer required.
Also, you should upgrade to Service Pack 3, it appears you are still using the original release. SP1 is required first but then you should be able to skip SP2 and install SP3.
You did not say what earlier version of Cr you were using? After CR 8.5 we went to full UNICODE support at which time they completely re-built the report designer and removed the database engines from the EXE and made them separate dll's now. OLE objecting also changed, you can use a formula and database field to point to linked objects now so they can be refreshed any time. Previously they were only refreshed when the report was opened.
You may want to see if linking them using a database field would speed up the process. Other than that I can't suggest anything else as a work around.
Thank you
Don

Report Performance degradation

Similar Messages

Maybe you are looking for