RAC 10.2.0.4, event gc cr block busy & log file switch
hello everybody,
i would like to know if there is any dependencies between gc cr block busy and log switch in the one node of the rac cluster.
i had a select and its completion time lasted 12 secs instead of 1, the start time of the select is the start time of the log switch on the node.
But when i looked into the active session history the session which was standing for that select had been waiting gc cr block busy instead log file switch completion.
While looking to the Google resources i ve noticed that "The gc current block busy and gc cr block busy wait events indicate that the
remote instance received the block after a remote instance processing delay.
In most cases, this is due to a log flush".
I would be really greatfull if anybody would be able to locate the initial dependancy i ve mantioned and explain the cause of the issue as i can not quite get why the selection took so long.
Thank you in advance!
Did you told "log file switch"?
you mean log file switch (checkpoint incomplete) or log file switch (archiving needed) or log file switch/archive or log file switch (clearing log file) or log file switch completion or log switch/archive
however a instance can wait ... if you find high values about waiting, you may tune your database.
please show us
- Top 5 Wait Events
SQL> alter session set nls_date_format='YYYY/MM/DD HH24:MI:SS';
SQL> select name, completion_time from V$ARCHIVED_LOG order by completion_time ;
Check How often do you switch logfile to archive log? ... Every switch log file... you may find "log file switch" waiting
I see... you no high DML activitiy.
But Please check High segment + object and query on AWR report... (example: Segments by Physical Writes )
just investigate
Good Luck
Similar Messages
-
Unhandled event loop exception (open a log file in NW Developer Studio)
Hi!
SAP NetWeaver Developer Studio
SAP NetWeaver 7.1 Composition Environment
When I try to open a log file at the SAP Management Console View (i.e. available.log) I get a "Unhandled event loop exception" and the log file content is not shown.
Any hints?
Example:
Log Viewing @ <a href="http://help.sap.com/saphelp_nwce10/helpdata/en/45/e3b0f5f76a2e99e10000000a155369/content.htm">http://help.sap.com/saphelp_nwce10/helpdata/en/45/e3b0f5f76a2e99e10000000a155369/content.htm</a>
Thanks,
ChrisUpdate:
I have a stack now
Caused by: java.lang.NullPointerException
at com.sap.managementconsole.jface.JFaceDataPublisher.getEditorForInput(JFaceDataPublisher.java:290)
at com.sap.managementconsole.jface.JFaceDataPublisher.access$000(JFaceDataPublisher.java:49)
at com.sap.managementconsole.jface.JFaceDataPublisher$7.run(JFaceDataPublisher.java:180)
at org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:35)
at org.eclipse.swt.widgets.Synchronizer.runAsyncMessages(Synchronizer.java:123)
... 20 more
-Chris -
Check Event Alert ends in error, log file has no error message
Hi,
We are upgrading from 11.5.10 to 12.1.3. Check Event Alert ends in error, log file doesn't have any information. This alert is on oe_order_headers_all. Alert working fine in 11i.
12.1.3
10g database
Appreciate any inputs.
I noticed that Operating Unit parameter is passed as NULL, not sure if this is causing the issue.
KHere is the workflow logfile from today, check event was run multiple times today as part of testing and i believe this log could be for these requests. I am not too sure how to read this....had to remove some lines from the log as it is more than 30000 characters.
LOG_ID_UNKNOWN : oracle.apps.fnd.cp.gsc.Logger.Logger(String, int) : Logging to System.out until necessary parameters are retrieved for Logger to be properly started.
LOG_ID_UNKNOWN : oracle.apps.fnd.cp.gsm.GSMSvcComponentContainer.initializeStateMachine() : BEGIN
LOG_ID_UNKNOWN : oracle.apps.fnd.cp.gsc.SvcComponentContainer.getNewWorkflowContext() : BEGIN
LOG_ID_UNKNOWN : oracle.apps.fnd.cp.gsc.SvcComponentContainer.getNewWorkflowContext() : BEGIN
LOG_ID_UNKNOWN : oracle.apps.fnd.cp.gsc.SvcComponentContainer.loadGlobalParameters() : BEGIN
LOG_ID_UNKNOWN : oracle.apps.fnd.cp.gsm.GSMSvcComponentContainer.loadContainerParameters() : BEGIN
LOG_ID_UNKNOWN : oracle.apps.fnd.cp.gsc.SvcComponentContainer.start() : Successfully retrieved global and container parameters
LOG_ID_UNKNOWN : oracle.apps.fnd.cp.gsc.SvcComponentContainer.validateParameterValues(Properties) : BEGIN
LOG_ID_UNKNOWN : oracle.apps.fnd.cp.gsc.SvcComponentContainer.validateParameterValues(Properties) : ({SVC_COMP_MAX_ERROR_COUNT=10, SVC_COMP_MONITOR_LOOP_SLEEP=60, SVC_CONTAINER_LOOP_SLEEP=10, SVC_CONTAINER_READ_TIMEOUT=10, SVC_COMP_MONITOR_ONDEMAND_FREQ=300, SVC_CONTAINER_LOG_LEVEL=4, SVC_WRITE_DIAG_TO_GSM_LOG=Y})
LOG_ID_UNKNOWN : oracle.apps.fnd.cp.gsc.SvcComponentContainer.start() : Successfully validated container parameters
LOG_ID_UNKNOWN : oracle.apps.fnd.cp.gsm.GSMSvcComponentContainer.loadDetails(Connection) : BEGIN (oracle.jdbc.driver.T4CConnection@8ae45a)
LOG_ID_UNKNOWN : oracle.apps.fnd.cp.gsc.SvcComponentContainer.start() : Successfully retrieved container details
LOG_ID_UNKNOWN : oracle.apps.fnd.cp.gsm.GSMSvcComponentContainer.startLogger() : BEGIN
LOG_ID_UNKNOWN : oracle.apps.fnd.cp.gsc.SvcComponentContainer.startLogger() : BEGIN [default implementation]
SVC-GSM-WFMLRSVC-27823-10010 : oracle.apps.fnd.cp.gsc.Logger.Logger(String, int) : Logging to System.out until necessary parameters are retrieved for Logger to be properly started.
3030 secs]
[GC 3198K->1790K(4992K), 0.0002970 secs]
[GC 3198K->1791K(4992K), 0.0002970 secs]
[GC 3199K->1792K(4992K), 0.0003130 secs]
[GC 3200K->1794K(4992K), 0.0003090 secs]
[GC 3202K->1792K(4992K), 0.0003030 secs]
[GC 3200K->1792K(4992K), 0.0003010 secs]
[GC 3200K->1793K(4992K), 0.0003040 secs]
[GC 3201K->1793K(4992K), 0.0003150 secs]
[GC 3201K->1793K(4992K), 0.0003050 secs]
[GC 3201K->1792K(4992K), 0.0003120 secs]
[GC 3200K->1790K(4992K), 0.0002960 secs]
[GC 3198K->1789K(4992K), 0.0003000 secs]
[GC 3197K->1789K(4992K), 0.0002980 secs]
[GC 4629K->3828K(5248K), 0.0015060 secs]
[Full GC 3828K->3466K(5248K), 0.0243510 secs]
[GC 5892K->4216K(8468K), 0.0016710 secs]
[GC 6648K->4971K(8468K), 0.0026010 secs]
[GC 7403K->5358K(8468K), 0.0036650 secs]
[GC 7790K->5610K(8468K), 0.0067030 secs]
[GC 8027K->6335K(8852K), 0.0047390 secs]
[Full GC 6335K->5444K(8852K), 0.0368670 secs]
[Aug 5, 2012 4:57:53 AM GMT+00:00]:1344142673430:-1:-1:ohioitp001:10.160.68.139:-1:-1:1:20420:SYSADMIN(0):-1:Thread[main,5,main]:10.160.68.139:80714:1344142672846:0:EXCEPTION:[SVC-GSM-WFMLRSVC-27823 : oracle.apps.fnd.cp.gsc.SvcComponentContainer.start()]:Successfully created new component control context
[Aug 5, 2012 4:57:53 AM GMT+00:00]:1344142673837:-1:-1:ohioitp001:10.160.68.139:-1:-1:1:20420:SYSADMIN(0):-1:Thread[main,5,main]:10.160.68.139:80714:1344142672846:0:EXCEPTION:[SVC-GSM-WFMLRSVC-27823 : oracle.apps.fnd.cp.gsc.SvcComponentContainer.startBusinessEventListener()]:Successfully asked for BES control connection to be established (asynchronous)
[Aug 5, 2012 4:57:55 AM GMT+00:00]:1344142675139:-1:-1:ohioitp001:10.160.68.139:-1:-1:1:20420:SYSADMIN(0):-1:Thread[main,5,main]:10.160.68.139:80714:1344142672846:0:EXCEPTION:[SVC-GSM-WFMLRSVC-27823 : oracle.apps.fnd.cp.gsc.SvcComponentContainer.startBusinessEventListener()]:Registered a subscription for oracle.apps.fnd.cp.gsc.SvcComponent.refresh
[Aug 5, 2012 4:57:55 AM GMT+00:00]:1344142675140:-1:-1:ohioitp001:10.160.68.139:-1:-1:1:20420:SYSADMIN(0):-1:Thread[main,5,main]:10.160.68.139:80714:1344142672846:0:EXCEPTION:[SVC-GSM-WFMLRSVC-27823 : oracle.apps.fnd.cp.gsc.SvcComponentContainer.startBusinessEventListener()]:Registered a subscription for oracle.apps.fnd.cp.gsc.SvcComponent.resume
[Aug 5, 2012 4:57:55 AM GMT+00:00]:1344142675140:-1:-1:ohioitp001:10.160.68.139:-1:-1:1:20420:SYSADMIN(0):-1:Thread[main,5,main]:10.160.68.139:80714:1344142672846:0:EXCEPTION:[SVC-GSM-WFMLRSVC-27823 : oracle.apps.fnd.cp.gsc.SvcComponentContainer.startBusinessEventListener()]:Registered a subscription for oracle.apps.fnd.cp.gsc.SvcComponent.start
[Aug 5, 2012 4:57:55 AM GMT+00:00]:1344142675140:-1:-1:ohioitp001:10.160.68.139:-1:-1:1:20420:SYSADMIN(0):-1:Thread[main,5,main]:10.160.68.139:80714:1344142672846:0:EXCEPTION:[SVC-GSM-WFMLRSVC-27823 : oracle.apps.fnd.cp.gsc.SvcComponentContainer.startBusinessEventListener()]:Registered a subscription for oracle.apps.fnd.cp.gsc.SvcComponent.stop
[Aug 5, 2012 4:57:55 AM GMT+00:00]:1344142675140:-1:-1:ohioitp001:10.160.68.139:-1:-1:1:20420:SYSADMIN(0):-1:Thread[main,5,main]:10.160.68.139:80714:1344142672846:0:EXCEPTION:[SVC-GSM-WFMLRSVC-27823 : oracle.apps.fnd.cp.gsc.SvcComponentContainer.startBusinessEventListener()]:Registered a subscription for oracle.apps.fnd.cp.gsc.SvcComponent.suspend
[Aug 5, 2012 4:57:55 AM GMT+00:00]:1344142675140:-1:-1:ohioitp001:10.160.68.139:-1:-1:1:20420:SYSADMIN(0):-1:Thread[main,5,main]:10.160.68.139:80714:1344142672846:0:EXCEPTION:[SVC-GSM-WFMLRSVC-27823 : oracle.apps.fnd.cp.gsc.SvcComponentContainer.startBusinessEventListener()]:Registered a subscription for oracle.apps.fnd.wf.mailer.Mailer.notification.summary
[Aug 5, 2012 4:57:55 AM GMT+00:00]:1344142675140:-1:-1:ohioitp001:10.160.68.139:-1:-1:1:20420:SYSADMIN(0):-1:Thread[main,5,main]:10.160.68.139:80714:1344142672846:0:EXCEPTION:[SVC-GSM-WFMLRSVC-27823 : oracle.apps.fnd.cp.gsc.SvcComponentContainer.startBusinessEventListener()]:Have waited 0 seconds for BES system to establish connection to the control queue. Waiting another 2 seconds
[GC 9025K->6456K(13172K), 0.0024970 secs]
[GC 10104K->6700K(13172K), 0.0028340 secs]
[Aug 5, 2012 4:57:57 AM GMT+00:00]:1344142677143:-1:-1:ohioitp001:10.160.68.139:-1:-1:1:20420:SYSADMIN(0):-1:Thread[main,5,main]:10.160.68.139:80714:1344142672846:0:EXCEPTION:[SVC-GSM-WFMLRSVC-27823 : oracle.apps.fnd.cp.gsc.SvcComponentContainer.startBusinessEventListener()]:Waited 2 seconds for BES system to establish connection to the control queue
[Aug 5, 2012 4:58:07 AM GMT+00:00]:1344142687008:-1:-1:ohioitp001:10.160.68.139:-1:-1:1:20420:SYSADMIN(0):-1:Thread[main,5,main]:10.160.68.139:80714:1344142672846:0:EXCEPTION:[SVC-GSM-WFMLRSVC-27823 : oracle.apps.fnd.cp.gsc.SvcComponentContainer.start()]:Successfully started BES listener
[Aug 5, 2012 4:58:07 AM GMT+00:00]:1344142687165:-1:-1:ohioitp001:10.160.68.139:-1:-1:1:20420:SYSADMIN(0):-1:Thread[main,5,main]:10.160.68.139:80714:1344142672846:0:EXCEPTION:[SVC-GSM-WFMLRSVC-27823 : oracle.apps.fnd.cp.gsc.SvcComponentContainer.start()]:Successfully reset component statuses for this container
[Aug 5, 2012 4:58:07 AM GMT+00:00]:1344142687171:-1:-1:ohioitp001:10.160.68.139:-1:-1:1:20420:SYSADMIN(0):-1:Thread[main,5,main]:10.160.68.139:80714:1344142672846:0:EXCEPTION:[SVC-GSM-WFMLRSVC-27823 : oracle.apps.fnd.cp.gsc.SvcComponentContainer.start()]:Successfully updated the container's database status, if any
[Aug 5, 2012 4:58:07 AM GMT+00:00]:1344142687173:-1:-1:ohioitp001:10.160.68.139:-1:-1:1:20420:SYSADMIN(0):-1:Thread[main,5,main]:10.160.68.139:80714:1344142672846:0:EXCEPTION:[SVC-GSM-WFMLRSVC-27823 : oracle.apps.fnd.cp.gsc.SvcComponentContainer.start()]:Successfully named container session (took logical lock)
[Aug 5, 2012 4:58:07 AM GMT+00:00]:1344142687276:-1:-1:ohioitp001:10.160.68.139:-1:-1:1:20420:SYSADMIN(0):-1:Thread[main,5,main]:10.160.68.139:80714:1344142672846:0:EXCEPTION:[SVC-GSM-WFMLRSVC-27823 : oracle.apps.fnd.cp.gsc.SvcComponentContainer.start()]:Successfully started the Component Monitor thread
[Aug 5, 2012 4:58:07 AM GMT+00:00]:1344142687282:-1:-1:ohioitp001:10.160.68.139:-1:-1:1:20420:SYSADMIN(0):-1:Thread[main,5,main]:10.160.68.139:80714:1344142672846:0:EXCEPTION:[SVC-GSM-WFMLRSVC-27823 : oracle.apps.fnd.cp.gsc.SvcComponentContainer.start()]:Successfully started the state machine
[Aug 5, 2012 4:58:07 AM GMT+00:00]:1344142687285:-1:-1:ohioitp001:10.160.68.139:-1:-1:1:20420:SYSADMIN(0):-1:Thread[main,5,main]:10.160.68.139:80714:1344142672846:0:EXCEPTION:[SVC-GSM-WFMLRSVC-27823 : oracle.apps.fnd.cp.gsc.SvcComponentContainer.start()]:Successfully started the container -> SvcComponentContainer{mComponentControlWorkflowContext=[oracle.apps.fnd.wf.apps.WorkflowContext@3a0ab1 $Revision: 120.14.12010000.18 $ extends [oracle.apps.fnd.common.Context@3a0ab1 $Revision: 120.7.12010000.15 $ {sessionId=0x3a0ab1,conn=oracle.jdbc.driver.T4CConnection@1de45e2,state=oracle.apps.fnd.common.StateStack@940f82}] {isDedicated=false}],mComponentMonitor=SvcComponentMonitor{Processor{mThread=Thread[ComponentMonitor,5,main],mCurrentLoopSleep=0,mMinLoopSleep=60,mMaxLoopSleep=0,mReadTimedOutFlag=false,mCurrentErrorCount=0,mMaxErrorCount=-2147483648,mErrorLoopSleep=60,mErrorFlag=false,mControlEvent=noEvent,mStopFlag=false,mSuspendFlag=false,mLastSuspendFlag=false},mCurrentLoopCount=4,mOnDemandLoopCount=5},mStateMachine=[email protected]18,mLogger=Logger{mLog=[$Header: AppsLog.java 120.4.12010000.2 2009/06/17 15:14:37 rsantis ship $ @9564165 {oracle.apps.fnd.profiles.ExtendedProfileStore@1250ff2}],mUniqueId=SVC-GSM-WFMLRSVC-27823,mLevel=4},mParameters={SVC_COMP_MAX_ERROR_COUNT=10, SVC_COMP_MONITOR_LOOP_SLEEP=60, SVC_CONTAINER_LOOP_SLEEP=10, SVC_CONTAINER_READ_TIMEOUT=10, SVC_COMP_MONITOR_ONDEMAND_FREQ=300, SVC_CONTAINER_LOG_LEVEL=4, SVC_WRITE_DIAG_TO_GSM_LOG=Y},mProcessId=27823,mType=GSM,mName=WFMLRSVC,mComponents={},mAutomaticComponentErrorCounts=null,mEventSubscriptions=[oracle.apps.fnd.wf.bes.Subscription@e2433b, oracle.apps.fnd.wf.bes.Subscription@abe654, oracle.apps.fnd.wf.bes.Subscription@19ea173, oracle.apps.fnd.wf.bes.Subscription@179a49f, oracle.apps.fnd.wf.bes.Subscription@1270107, oracle.apps.fnd.wf.bes.Subscription@1b48392]}
[Aug 5, 2012 4:58:07 AM GMT+00:00]:1344142687284:-1:-1:ohioitp001:10.160.68.139:-1:-1:1:20420:SYSADMIN(0):-1:Thread[GSMQueueProcessor,5,main]:10.160.68.139:80714:1344142687286:1:EXCEPTION:[SVC-GSM-WFMLRSVC-27823 : oracle.apps.fnd.cp.gsc.Processor.run()]:Processor has successfully been initialized. Loop will now start.
[GC 10319K->8190K(13172K), 0.0032630 secs]
[Aug 5, 2012 4:58:07 AM GMT+00:00]:1344142687320:-1:-1:ohioitp001:10.160.68.139:-1:-1:1:20420:SYSADMIN(0):-1:Thread[ComponentMonitor,5,main]:10.160.68.139:80714:1344142687320:2:EXCEPTION:[SVC-GSM-WFMLRSVC-27823 : oracle.apps.fnd.cp.gsc.SvcComponentMonitor.performInit()]:Successfully initialized -> SvcComponentMonitor{Processor{mThread=Thread[ComponentMonitor,5,main],mCurrentLoopSleep=0,mMinLoopSleep=60,mMaxLoopSleep=0,mReadTimedOutFlag=false,mCurrentErrorCount=0,mMaxErrorCount=-2147483648,mErrorLoopSleep=60,mErrorFlag=false,mControlEvent=noEvent,mStopFlag=false,mSuspendFlag=false,mLastSuspendFlag=false},mCurrentLoopCount=4,mOnDemandLoopCount=5}
[Aug 5, 2012 4:58:07 AM GMT+00:00]:1344142687320:-1:-1:ohioitp001:10.160.68.139:-1:-1:1:20420:SYSADMIN(0):-1:Thread[ComponentMonitor,5,main]:10.160.68.139:80714:1344142687320:2:EXCEPTION:[SVC-GSM-WFMLRSVC-27823 : oracle.apps.fnd.cp.gsc.Processor.run()]:Processor has successfully been initialized. Loop will now start.
[Aug 5, 2012 4:58:07 AM GMT+00:00]:1344142687454:-1:-1:ohioitp001:10.160.68.139:-1:-1:1:20420:SYSADMIN(0):-1:Thread[ComponentMonitor,5,main]:10.160.68.139:80714:1344142687320:2:EXCEPTION:[SVC-GSM-WFMLRSVC-27823 : oracle.apps.fnd.cp.gsc.SvcComponentMonitor.startAutomaticComponents()]:Starting automatic component 10010
[Aug 5, 2012 4:58:16 AM GMT+00:00]:1344142696950:-1:-1:ohioitp001:10.160.68.139:-1:-1:1:20420:SYSADMIN(0):-1:Thread[BES Dispatch Thread,5,main]:10.160.68.139:80714:1344142696950:3:EXCEPTION:[SVC-GSM-WFMLRSVC-27823 : oracle.apps.fnd.cp.gsc.SvcComponentContainer.onBusinessEvent(BusinessEvent)]:(BusinessEvent{name=oracle.apps.fnd.cp.gsc.SvcComponent.start, key=SVC:05-AUG-2012, priority=50, correlationId=null, sendDate=Sun Aug 05 04:58:08 GMT+00:00 2012, receiveDate=null, From Agent: , To Agent: , Last Subscription= , Error Message=null, Error Stack=null, CONTAINER_TYPE=GSM, CONTAINER_PROCESS_ID=27819, COMPONENT_ID=10004, BES_PAYLOAD_OBJECT=false, [email protected]})
[Aug 5, 2012 4:58:17 AM GMT+00:00]:1344142697016:-1:-1:ohioitp001:10.160.68.139:-1:-1:1:20420:SYSADMIN(0):-1:Thread[BES Dispatch Thread,5,main]:10.160.68.139:80714:1344142697016:4:EXCEPTION:[SVC-GSM-WFMLRSVC-27823 : oracle.apps.fnd.cp.gsc.SvcComponentContainer.onBusinessEvent(BusinessEvent)]:(BusinessEvent{name=oracle.apps.fnd.cp.gsc.SvcComponent.start, key=SVC:05-AUG-2012, priority=50, correlationId=null, sendDate=Sun Aug 05 04:58:08 GMT+00:00 2012, receiveDate=null, From Agent: , To Agent: , Last Subscription= , Error Message=null, Error Stack=null, CONTAINER_TYPE=GSM, CONTAINER_PROCESS_ID=27823, COMPONENT_ID=10010, BES_PAYLOAD_OBJECT=false, [email protected]})
[Aug 5, 2012 4:58:17 AM GMT+00:00]:1344142697031:-1:-1:ohioitp001:10.160.68.139:-1:-1:1:20420:SYSADMIN(0):-1:Thread[BES Dispatch Thread,5,main]:10.160.68.139:80714:1344142697016:4:EXCEPTION:[SVC-GSM-WFMLRSVC-27823 : oracle.apps.fnd.cp.gsc.SvcComponentContainer.handleComponentEvent(int, String, String)]:Successfully retrieved component details from the database
[GC 11838K->8860K(13172K), 0.0054540 secs]
[GC 12508K->9106K(13172K), 0.0025580 secs]
[GC 12754K->9621K(13432K), 0.0020510 secs]
[Full GC 9621K->8781K(13432K), 0.0354810 secs]
[Aug 5, 2012 4:58:36 AM GMT+00:00]:1344142716097:-1:-1:ohioitp001:10.160.68.139:-1:-1:1:20420:SYSADMIN(0):-1:Thread[BES Dispatch Thread,5,main]:10.160.68.139:80714:1344142697016:4:EXCEPTION:[SVC-GSM-WFMLRSVC-27823 : oracle.apps.fnd.cp.gsc.SvcComponentContainer.onBusinessEvent(BusinessEvent)]:Successfully handled component event, oracle.apps.fnd.cp.gsc.SvcComponent.start, for component 10010
[Aug 5, 2012 4:58:36 AM GMT+00:00]:1344142716098:-1:-1:ohioitp001:10.160.68.139:-1:-1:1:20420:SYSADMIN(0):-1:Thread[BES Dispatch Thread,5,main]:10.160.68.139:80714:1344142716098:5:EXCEPTION:[SVC-GSM-WFMLRSVC-27823 : oracle.apps.fnd.cp.gsc.SvcComponentContainer.onBusinessEvent(BusinessEvent)]:(BusinessEvent{name=oracle.apps.fnd.cp.gsc.SvcComponent.start, key=SVC:05-AUG-2012, priority=50, correlationId=null, sendDate=Sun Aug 05 04:58:10 GMT+00:00 2012, receiveDate=null, From Agent: , To Agent: , Last Subscription= , Error Message=null, Error Stack=null, CONTAINER_TYPE=GSM, CONTAINER_PROCESS_ID=27819, COMPONENT_ID=10011, BES_PAYLOAD_OBJECT=false, [email protected]})
[Aug 5, 2012 4:58:36 AM GMT+00:00]:1344142716098:-1:-1:ohioitp001:10.160.68.139:-1:-1:1:20420:SYSADMIN(0):-1:Thread[BES Dispatch Thread,5,main]:10.160.68.139:80714:1344142716098:6:EXCEPTION:[SVC-GSM-WFMLRSVC-27823 : oracle.apps.fnd.cp.gsc.SvcComponentContainer.onBusinessEvent(BusinessEvent)]:(BusinessEvent{name=oracle.apps.fnd.cp.gsc.SvcComponent.start, key=SVC:05-AUG-2012, priority=50, correlationId=null, sendDate=Sun Aug 05 04:58:10 GMT+00:00 2012, receiveDate=null, From Agent: , To Agent: , Last Subscription= , Error Message=null, Error Stack=null, CONTAINER_TYPE=GSM, CONTAINER_PROCESS_ID=27819, COMPONENT_ID=10009, BES_PAYLOAD_OBJECT=false, [email protected]})
[Aug 5, 2012 4:58:36 AM GMT+00:00]:1344142716098:-1:-1:ohioitp001:10.160.68.139:-1:-1:1:20420:SYSADMIN(0):-1:Thread[BES Dispatch Thread,5,main]:10.160.68.139:80714:1344142716098:7:EXCEPTION:[SVC-GSM-WFMLRSVC-27823 : oracle.apps.fnd.cp.gsc.SvcComponentContainer.onBusinessEvent(BusinessEvent)]:(BusinessEvent{name=oracle.apps.fnd.cp.gsc.SvcComponent.start, key=SVC:05-AUG-2012, priority=50, correlationId=null, sendDate=Sun Aug 05 04:58:10 GMT+00:00 2012, receiveDate=null, From Agent: , To Agent: , Last Subscription= , Error Message=null, Error Stack=null, CONTAINER_TYPE=GSM, CONTAINER_PROCESS_ID=27819, COMPONENT_ID=10008, BES_PAYLOAD_OBJECT=false, [email protected]})
[Aug 5, 2012 4:58:36 AM GMT+00:00]:1344142716099:-1:-1:ohioitp001:10.160.68.139:-1:-1:1:20420:SYSADMIN(0):-1:Thread[BES Dispatch Thread,5,main]:10.160.68.139:80714:1344142716099:8:EXCEPTION:[SVC-GSM-WFMLRSVC-27823 : oracle.apps.fnd.cp.gsc.SvcComponentContainer.onBusinessEvent(BusinessEvent)]:(BusinessEvent{name=oracle.apps.fnd.cp.gsc.SvcComponent.start, key=SVC:05-AUG-2012, priority=50, correlationId=null, sendDate=Sun Aug 05 04:58:10 GMT+00:00 2012, receiveDate=null, From Agent: , To Agent: , Last Subscription= , Error Message=null, Error Stack=null, CONTAINER_TYPE=GSM, CONTAINER_PROCESS_ID=27819, COMPONENT_ID=10007, BES_PAYLOAD_OBJECT=false, [email protected]})
[Aug 5, 2012 4:58:36 AM GMT+00:00]:1344142716099:-1:-1:ohioitp001:10.160.68.139:-1:-1:1:20420:SYSADMIN(0):-1:Thread[BES Dispatch Thread,5,main]:10.160.68.139:80714:1344142716099:9:EXCEPTION:[SVC-GSM-WFMLRSVC-27823 : oracle.apps.fnd.cp.gsc.SvcComponentContainer.onBusinessEvent(BusinessEvent)]:(BusinessEvent{name=oracle.apps.fnd.cp.gsc.SvcComponent.start, key=SVC:05-AUG-2012, priority=50, correlationId=null, sendDate=Sun Aug 05 04:58:10 GMT+00:00 2012, receiveDate=null, From Agent: , To Agent: , Last Subscription= , Error Message=null, Error Stack=null, CONTAINER_TYPE=GSM, CONTAINER_PROCESS_ID=27819, COMPONENT_ID=10006, BES_PAYLOAD_OBJECT=false, [email protected]})
[Aug 5, 2012 4:58:36 AM GMT+00:00]:1344142716099:-1:-1:ohioitp001:10.160.68.139:-1:-1:1:20420:SYSADMIN(0):-1:Thread[BES Dispatch Thread,5,main]:10.160.68.139:80714:1344142716099:10:EXCEPTION:[SVC-GSM-WFMLRSVC-27823 : oracle.apps.fnd.cp.gsc.SvcComponentContainer.onBusinessEvent(BusinessEvent)]:(BusinessEvent{name=oracle.apps.fnd.cp.gsc.SvcComponent.start, key=SVC:05-AUG-2012, priority=50, correlationId=null, sendDate=Sun Aug 05 04:58:10 GMT+00:00 2012, receiveDate=null, From Agent: , To Agent: , Last Subscription= , Error Message=null, Error Stack=null, CONTAINER_TYPE=GSM, CONTAINER_PROCESS_ID=27819, COMPONENT_ID=10005, BES_PAYLOAD_OBJECT=false, [email protected]})
[GC 14733K->11747K(21292K), 0.0039530 secs]
[GC 17672K->13394K(21292K), 0.0059880 secs]
[GC 19340K->14241K(21292K), 0.0027020 secs]
[GC 20193K->14421K(21292K), 0.0029050 secs]
[GC 20303K->15379K(21420K), 0.0014680 secs]
[Full GC[Unloading class sun.reflect.GeneratedMethodAccessor1]
[Unloading class sun.reflect.GeneratedSerializationConstructorAccessor4]
[Unloading class sun.reflect.GeneratedSerializationConstructorAccessor15]
[Unloading class sun.reflect.GeneratedSerializationConstructorAccessor10]
[Unloading class sun.reflect.GeneratedSerializationConstructorAccessor2]
[Unloading class sun.reflect.GeneratedMethodAccessor2]
[Unloading class sun.reflect.GeneratedSerializationConstructorAccessor3]
[Unloading class sun.reflect.GeneratedSerializationConstructorAccessor1]
[Unloading class sun.reflect.GeneratedSerializationConstructorAccessor14]
[Unloading class sun.reflect.GeneratedSerializationConstructorAccessor16]
15379K->10614K(21420K), 0.0507280 secs]
[GC 17822K->11024K(25756K), 0.0011190 secs]
[GC 18256K->10845K(25756K), 0.0009730 secs]
[GC 18003K->10806K(25756K), 0.0006760 secs]
[GC 17969K->10857K(25756K), 0.0010200 secs]
[GC 18053K->10940K(25756K), 0.0007280 secs]
[GC 18172K->10834K(25756K), 0.0010650 secs]
[GC 18049K->11012K(25756K), 0.0007780 secs]
[GC 18244K->11037K(25756K), 0.0008230 secs]
[GC 18173K->10927K(25756K), 0.0007040 secs]
[GC 18116K->10915K(25756K), 0.0006930 secs]
[GC 18142K->11114K(25756K), 0.0011650 secs]
[GC 18346K->10920K(25756K), 0.0008180 secs]
[GC 18152K->10908K(25756K), 0.0007900 secs]
[GC 18134K->11088K(25756K), 0.0007210 secs]
[GC 18242K->11023K(25756K), 0.0011560 secs]
[GC 18235K->11131K(25756K), 0.0010920 secs]
[GC 18342K->11105K(25756K), 0.0007430 secs]
[GC 18305K->11145K(25756K), 0.0010030 secs]
[GC 18302K->11045K(25756K), 0.0010750 secs]
[GC 18238K->11020K(25756K), 0.0009560 secs]
[GC 18252K->11136K(25756K), 0.0008380 secs]
[GC 18366K->10956K(25756K), 0.0014370 secs]
[GC 18098K->11075K(25756K), 0.0009240 secs]
[GC 18266K->10979K(25756K), 0.0009150 secs]
[GC 18171K->11088K(25756K), 0.0009690 secs]
[GC 18310K->11196K(25756K), 0.0009630 secs]
[GC 18428K->11171K(25756K), 0.0013610 secs]
[GC 18403K->11005K(25756K), 0.0007650 secs]
[GC 18223K->11101K(25756K), 0.0006870 secs] -
Log file sequential read and RFS ping/write - among Top 5 event
I have situation here to discuss. In a 3-node RAC setup which is Logical standby DB; one node is showing high CPU utilization around 40~50%. The CPU utilization was less than 20% 10 days back but from 9th oldest day it jumped and consistently shows the double figure. I ran AWR reports on all three nodes and found one node with high CPU utilization and shows below tops events-
EVENT WAITS TIME(S) AVG WAIT(MS) %TOTAL CALL TIME WAIT CLASS
CPU time 5,802 34.9
RFS ping 15 5,118 33,671 30.8 Other
Log file sequential read 234,831 5,036 21 30.3 System I/O
Sql*Net more data from
client 24,171 1,087 45 6.5 Network
Db file sequential read 130,939 453 3 2.7 User I/O
Findings:-
On AWR report(file attached) for node= sipd207; we can see that "RFS PING" wait event takes 30% of the waits and "log file sequential read" wait event takes 30% of the waits that occurs in database.
Environment :- (Oracle- 10.2.0.4.0, O/S - AIX .3)
1)other node awr shows "log file sync" - is it due to oversized log buffer?
2)Network wait events can be reduced by tweaking SDU & TDU values based on MDU.
3) Why ARCH processes taking much to archives filled redo logs; is it issue with slow disk I/O?
Regards
WORKLOAD REPOSITORY report for<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<DB Name DB Id Instance Inst Num Release RAC Host
XXXPDB 4123595889 XXX2p2 2 10.2.0.4.0 YES sipd207
Snap Id Snap Time Sessions Curs/Sess
Begin Snap: 1053 04-Apr-11 18:00:02 59 7.4
End Snap: 1055 04-Apr-11 20:00:35 56 7.5
Elapsed: 120.55 (mins)
DB Time: 233.08 (mins)
Cache Sizes
~~~~~~~~~~~ Begin End
Buffer Cache: 3,728M 3,728M Std Block Size: 8K
Shared Pool Size: 4,080M 4,080M Log Buffer: 14,332K
Load Profile
~~~~~~~~~~~~ Per Second Per Transaction
Redo size: 245,392.33 10,042.66
Logical reads: 9,080.80 371.63
Block changes: 1,518.12 62.13
Physical reads: 7.50 0.31
Physical writes: 44.00 1.80
User calls: 36.44 1.49
Parses: 25.84 1.06
Hard parses: 0.59 0.02
Sorts: 12.06 0.49
Logons: 0.05 0.00
Executes: 295.91 12.11
Transactions: 24.43
% Blocks changed per Read: 16.72 Recursive Call %: 94.18
Rollback per transaction %: 4.15 Rows per Sort: 53.31
Instance Efficiency Percentages (Target 100%)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Buffer Nowait %: 99.99 Redo NoWait %: 100.00
Buffer Hit %: 99.92 In-memory Sort %: 100.00
Library Hit %: 99.83 Soft Parse %: 97.71
Execute to Parse %: 91.27 Latch Hit %: 99.79
Parse CPU to Parse Elapsd %: 15.69 % Non-Parse CPU: 99.95
Shared Pool Statistics Begin End
Memory Usage %: 83.60 84.67
% SQL with executions>1: 97.49 97.19
% Memory for SQL w/exec>1: 97.10 96.67
Top 5 Timed Events Avg %Total
~~~~~~~~~~~~~~~~~~ wait Call
Event Waits Time (s) (ms) Time Wait Class
CPU time 4,503 32.2
RFS ping 168 4,275 25449 30.6 Other
log file sequential read 183,537 4,173 23 29.8 System I/O
SQL*Net more data from client 21,371 1,009 47 7.2 Network
RFS write 25,438 343 13 2.5 System I/O
RAC Statistics DB/Inst: UDAS2PDB/udas2p2 Snaps: 1053-1055
Begin End
Number of Instances: 3 3
Global Cache Load Profile
~~~~~~~~~~~~~~~~~~~~~~~~~ Per Second Per Transaction
Global Cache blocks received: 0.78 0.03
Global Cache blocks served: 1.18 0.05
GCS/GES messages received: 131.69 5.39
GCS/GES messages sent: 139.26 5.70
DBWR Fusion writes: 0.06 0.00
Estd Interconnect traffic (KB) 68.60
Global Cache Efficiency Percentages (Target local+remote 100%)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Buffer access - local cache %: 99.91
Buffer access - remote cache %: 0.01
Buffer access - disk %: 0.08
Global Cache and Enqueue Services - Workload Characteristics
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Avg global enqueue get time (ms): 0.5
Avg global cache cr block receive time (ms): 0.9
Avg global cache current block receive time (ms): 1.0
Avg global cache cr block build time (ms): 0.0
Avg global cache cr block send time (ms): 0.1
Global cache log flushes for cr blocks served %: 2.9
Avg global cache cr block flush time (ms): 4.6
Avg global cache current block pin time (ms): 0.0
Avg global cache current block send time (ms): 0.1
Global cache log flushes for current blocks served %: 0.1
Avg global cache current block flush time (ms): 5.0
Global Cache and Enqueue Services - Messaging Statistics
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Avg message sent queue time (ms): 0.1
Avg message sent queue time on ksxp (ms): 0.6
Avg message received queue time (ms): 0.0
Avg GCS message process time (ms): 0.0
Avg GES message process time (ms): 0.1
% of direct sent messages: 31.57
% of indirect sent messages: 5.17
% of flow controlled messages: 63.26
Time Model Statistics DB/Inst: UDAS2PDB/udas2p2 Snaps: 1053-1055
-> Total time in database user-calls (DB Time): 13984.6s
-> Statistics including the word "background" measure background process
time, and so do not contribute to the DB time statistic
-> Ordered by % or DB time desc, Statistic name
Statistic Name Time (s) % of DB Time
sql execute elapsed time 7,270.6 52.0
DB CPU 4,503.1 32.2
parse time elapsed 506.7 3.6
hard parse elapsed time 497.8 3.6
sequence load elapsed time 152.4 1.1
failed parse elapsed time 19.5 .1
repeated bind elapsed time 3.4 .0
PL/SQL execution elapsed time 0.7 .0
hard parse (sharing criteria) elapsed time 0.3 .0
connection management call elapsed time 0.3 .0
hard parse (bind mismatch) elapsed time 0.0 .0
DB time 13,984.6 N/A
background elapsed time 869.1 N/A
background cpu time 276.6 N/A
Wait Class DB/Inst: UDAS2PDB/udas2p2 Snaps: 1053-1055
-> s - second
-> cs - centisecond - 100th of a second
-> ms - millisecond - 1000th of a second
-> us - microsecond - 1000000th of a second
-> ordered by wait time desc, waits desc
Avg
%Time Total Wait wait Waits
Wait Class Waits -outs Time (s) (ms) /txn
System I/O 529,934 .0 4,980 9 3.0
Other 582,349 37.4 4,611 8 3.3
Network 279,858 .0 1,009 4 1.6
User I/O 54,899 .0 317 6 0.3
Concurrency 136,907 .1 58 0 0.8
Cluster 60,300 .0 41 1 0.3
Commit 80 .0 10 130 0.0
Application 6,707 .0 3 0 0.0
Configuration 17,528 98.5 1 0 0.1
Wait Events DB/Inst: UDAS2PDB/udas2p2 Snaps: 1053-1055
-> s - second
-> cs - centisecond - 100th of a second
-> ms - millisecond - 1000th of a second
-> us - microsecond - 1000000th of a second
-> ordered by wait time desc, waits desc (idle events last)
Avg
%Time Total Wait wait Waits
Event Waits -outs Time (s) (ms) /txn
RFS ping 168 .0 4,275 25449 0.0
log file sequential read 183,537 .0 4,173 23 1.0
SQL*Net more data from clien 21,371 .0 1,009 47 0.1
RFS write 25,438 .0 343 13 0.1
db file sequential read 54,680 .0 316 6 0.3
DFS lock handle 97,149 .0 214 2 0.5
log file parallel write 104,808 .0 157 2 0.6
db file parallel write 143,905 .0 149 1 0.8
RFS random i/o 25,438 .0 86 3 0.1
RFS dispatch 25,610 .0 56 2 0.1
control file sequential read 39,309 .0 55 1 0.2
row cache lock 130,665 .0 47 0 0.7
gc current grant 2-way 35,498 .0 23 1 0.2
wait for scn ack 50,872 .0 20 0 0.3
enq: WL - contention 6,156 .0 14 2 0.0
gc cr grant 2-way 16,917 .0 11 1 0.1
log file sync 80 .0 10 130 0.0
Log archive I/O 3,986 .0 9 2 0.0
control file parallel write 3,493 .0 8 2 0.0
latch free 2,356 .0 6 2 0.0
ksxr poll remote instances 278,473 49.4 6 0 1.6
enq: XR - database force log 2,890 .0 4 1 0.0
enq: TX - index contention 325 .0 3 11 0.0
buffer busy waits 4,371 .0 3 1 0.0
gc current block 2-way 3,002 .0 3 1 0.0
LGWR wait for redo copy 9,601 .2 2 0 0.1
SQL*Net break/reset to clien 6,438 .0 2 0 0.0
latch: ges resource hash lis 23,223 .0 2 0 0.1
enq: WF - contention 32 6.3 2 62 0.0
enq: FB - contention 660 .0 2 2 0.0
enq: PS - contention 1,088 .0 2 1 0.0
library cache lock 869 .0 1 2 0.0
enq: CF - contention 671 .1 1 2 0.0
gc current grant busy 1,488 .0 1 1 0.0
gc current multi block reque 1,072 .0 1 1 0.0
reliable message 618 .0 1 2 0.0
CGS wait for IPC msg 62,402 100.0 1 0 0.4
gc current block 3-way 998 .0 1 1 0.0
name-service call wait 18 .0 1 57 0.0
cursor: pin S wait on X 78 100.0 1 11 0.0
os thread startup 16 .0 1 53 0.0
enq: RO - fast object reuse 193 .0 1 3 0.0
IPC send completion sync 652 99.2 1 1 0.0
local write wait 194 .0 1 3 0.0
gc cr block 2-way 534 .0 0 1 0.0
log file switch completion 17 .0 0 20 0.0
SQL*Net message to client 258,483 .0 0 0 1.5
undo segment extension 17,282 99.9 0 0 0.1
gc cr block 3-way 286 .7 0 1 0.0
enq: TM - contention 76 .0 0 4 0.0
PX Deq: reap credit 15,246 95.6 0 0 0.1
kksfbc child completion 5 100.0 0 49 0.0
enq: TT - contention 141 .0 0 2 0.0
enq: HW - contention 203 .0 0 1 0.0
RFS create 2 .0 0 115 0.0
rdbms ipc reply 339 .0 0 1 0.0
PX Deq Credit: send blkd 452 20.1 0 0 0.0
gcs log flush sync 128 32.8 0 2 0.0
latch: cache buffers chains 128 .0 0 1 0.0
library cache pin 441 .0 0 0 0.0
Wait Events DB/Inst: UDAS2PDB/udas2p2 Snaps: 1053-1055
-> s - second
-> cs - centisecond - 100th of a second
-> ms - millisecond - 1000th of a second
-> us - microsecond - 1000000th of a second
-> ordered by wait time desc, waits desc (idle events last)We only apply on one node in a cluster so I would expect that the node running SQL Apply would have much higher usage and waits. Is this what you are asking?
Larry -
Log file sync top event during performance test -av 36ms
Hi,
During the performance test for our product before deployment into product i see "log file sync" on top with Avg wait (ms) being 36 which i feel is too high.
Avg
wait % DB
Event Waits Time(s) (ms) time Wait Class
log file sync 208,327 7,406 36 46.6 Commit
direct path write 646,833 3,604 6 22.7 User I/O
DB CPU 1,599 10.1
direct path read temp 1,321,596 619 0 3.9 User I/O
log buffer space 4,161 558 134 3.5 ConfiguratAlthough testers are not complaining about the performance of the appplication , we ,DBAs, are expected to be proactive about the any bad signals from DB.
I am not able to figure out why "log file sync" is having such slow response.
Below is the snapshot from the load profile.
Snap Id Snap Time Sessions Curs/Sess
Begin Snap: 108127 16-May-13 20:15:22 105 6.5
End Snap: 108140 16-May-13 23:30:29 156 8.9
Elapsed: 195.11 (mins)
DB Time: 265.09 (mins)
Cache Sizes Begin End
~~~~~~~~~~~ ---------- ----------
Buffer Cache: 1,168M 1,136M Std Block Size: 8K
Shared Pool Size: 1,120M 1,168M Log Buffer: 16,640K
Load Profile Per Second Per Transaction Per Exec Per Call
~~~~~~~~~~~~ --------------- --------------- ---------- ----------
DB Time(s): 1.4 0.1 0.02 0.01
DB CPU(s): 0.1 0.0 0.00 0.00
Redo size: 607,512.1 33,092.1
Logical reads: 3,900.4 212.5
Block changes: 1,381.4 75.3
Physical reads: 134.5 7.3
Physical writes: 134.0 7.3
User calls: 145.5 7.9
Parses: 24.6 1.3
Hard parses: 7.9 0.4
W/A MB processed: 915,418.7 49,864.2
Logons: 0.1 0.0
Executes: 85.2 4.6
Rollbacks: 0.0 0.0
Transactions: 18.4Some of the top background wait events:
^LBackground Wait Events DB/Inst: Snaps: 108127-108140
-> ordered by wait time desc, waits desc (idle events last)
-> Only events with Total Wait Time (s) >= .001 are shown
-> %Timeouts: value of 0 indicates value was < .5%. Value of null is truly 0
Avg
%Time Total Wait wait Waits % bg
Event Waits -outs Time (s) (ms) /txn time
log file parallel write 208,563 0 2,528 12 1.0 66.4
db file parallel write 4,264 0 785 184 0.0 20.6
Backup: sbtbackup 1 0 516 516177 0.0 13.6
control file parallel writ 4,436 0 97 22 0.0 2.6
log file sequential read 6,922 0 95 14 0.0 2.5
Log archive I/O 6,820 0 48 7 0.0 1.3
os thread startup 432 0 26 60 0.0 .7
Backup: sbtclose2 1 0 10 10094 0.0 .3
db file sequential read 2,585 0 8 3 0.0 .2
db file single write 560 0 3 6 0.0 .1
log file sync 28 0 1 53 0.0 .0
control file sequential re 36,326 0 1 0 0.2 .0
log file switch completion 4 0 1 207 0.0 .0
buffer busy waits 5 0 1 116 0.0 .0
LGWR wait for redo copy 924 0 1 1 0.0 .0
log file single write 56 0 1 9 0.0 .0
Backup: sbtinfo2 1 0 1 500 0.0 .0During a previous perf test , things didnt look this bad for "log file sync. Few sections from the comparision report(awrddprt.sql)
{code}
Workload Comparison
~~~~~~~~~~~~~~~~~~~ 1st Per Sec 2nd Per Sec %Diff 1st Per Txn 2nd Per Txn %Diff
DB time: 0.78 1.36 74.36 0.02 0.07 250.00
CPU time: 0.18 0.14 -22.22 0.00 0.01 100.00
Redo size: 573,678.11 607,512.05 5.90 15,101.84 33,092.08 119.13
Logical reads: 4,374.04 3,900.38 -10.83 115.14 212.46 84.52
Block changes: 1,593.38 1,381.41 -13.30 41.95 75.25 79.38
Physical reads: 76.44 134.54 76.01 2.01 7.33 264.68
Physical writes: 110.43 134.00 21.34 2.91 7.30 150.86
User calls: 197.62 145.46 -26.39 5.20 7.92 52.31
Parses: 7.28 24.55 237.23 0.19 1.34 605.26
Hard parses: 0.00 7.88 100.00 0.00 0.43 100.00
Sorts: 3.88 4.90 26.29 0.10 0.27 170.00
Logons: 0.09 0.08 -11.11 0.00 0.00 0.00
Executes: 126.69 85.19 -32.76 3.34 4.64 38.92
Transactions: 37.99 18.36 -51.67
First Second Diff
1st 2nd
Event Wait Class Waits Time(s) Avg Time(ms) %DB time Event Wait Class Waits Time(s) Avg Time
(ms) %DB time
SQL*Net more data from client Network 2,133,486 1,270.7 0.6 61.24 log file sync Commit 208,355 7,407.6
35.6 46.57
CPU time N/A 487.1 N/A 23.48 direct path write User I/O 646,849 3,604.7
5.6 22.66
log file sync Commit 99,459 129.5 1.3 6.24 log file parallel write System I/O 208,564 2,528.4
12.1 15.90
log file parallel write System I/O 100,732 126.6 1.3 6.10 CPU time N/A 1,599.3
N/A 10.06
SQL*Net more data to client Network 451,810 103.1 0.2 4.97 db file parallel write System I/O 4,264 784.7 1
84.0 4.93
-direct path write User I/O 121,044 52.5 0.4 2.53 -SQL*Net more data from client Network 7,407,435 279.7
0.0 1.76
-db file parallel write System I/O 986 22.8 23.1 1.10 -SQL*Net more data to client Network 2,714,916 64.6
0.0 0.41
{code}
*To sum it sup:
1. Why is the IO response getting such an hit during the new perf test? Please suggest*
2. Does the number of DB writer impact "log file sync" wait event? We have only one DB writer as the number of cpu on the host is only 4
{code}
select *from v$version;
BANNER
Oracle Database 11g Enterprise Edition Release 11.1.0.7.0 - 64bit Production
PL/SQL Release 11.1.0.7.0 - Production
CORE 11.1.0.7.0 Production
TNS for HPUX: Version 11.1.0.7.0 - Production
NLSRTL Version 11.1.0.7.0 - Production
{code}
Please let me know if you would like to see any other stats.
Edited by: Kunwar on May 18, 2013 2:20 PM1. A snapshot interval of 3 hours always generates meaningless results
Below are some details from the 1 hour interval AWR report.
Platform CPUs Cores Sockets Memory(GB)
HP-UX IA (64-bit) 4 4 3 31.95
Snap Id Snap Time Sessions Curs/Sess
Begin Snap: 108129 16-May-13 20:45:32 140 8.0
End Snap: 108133 16-May-13 21:45:53 150 8.8
Elapsed: 60.35 (mins)
DB Time: 140.49 (mins)
Cache Sizes Begin End
~~~~~~~~~~~ ---------- ----------
Buffer Cache: 1,168M 1,168M Std Block Size: 8K
Shared Pool Size: 1,120M 1,120M Log Buffer: 16,640K
Load Profile Per Second Per Transaction Per Exec Per Call
~~~~~~~~~~~~ --------------- --------------- ---------- ----------
DB Time(s): 2.3 0.1 0.03 0.01
DB CPU(s): 0.1 0.0 0.00 0.00
Redo size: 719,553.5 34,374.6
Logical reads: 4,017.4 191.9
Block changes: 1,521.1 72.7
Physical reads: 136.9 6.5
Physical writes: 158.3 7.6
User calls: 167.0 8.0
Parses: 25.8 1.2
Hard parses: 8.9 0.4
W/A MB processed: 406,220.0 19,406.0
Logons: 0.1 0.0
Executes: 88.4 4.2
Rollbacks: 0.0 0.0
Transactions: 20.9
Top 5 Timed Foreground Events
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Avg
wait % DB
Event Waits Time(s) (ms) time Wait Class
log file sync 73,761 6,740 91 80.0 Commit
log buffer space 3,581 541 151 6.4 Configurat
DB CPU 348 4.1
direct path write 238,962 241 1 2.9 User I/O
direct path read temp 487,874 174 0 2.1 User I/O
Background Wait Events DB/Inst: Snaps: 108129-108133
-> ordered by wait time desc, waits desc (idle events last)
-> Only events with Total Wait Time (s) >= .001 are shown
-> %Timeouts: value of 0 indicates value was < .5%. Value of null is truly 0
Avg
%Time Total Wait wait Waits % bg
Event Waits -outs Time (s) (ms) /txn time
log file parallel write 61,049 0 1,891 31 0.8 87.8
db file parallel write 1,590 0 251 158 0.0 11.6
control file parallel writ 1,372 0 56 41 0.0 2.6
log file sequential read 2,473 0 50 20 0.0 2.3
Log archive I/O 2,436 0 20 8 0.0 .9
os thread startup 135 0 8 60 0.0 .4
db file sequential read 668 0 4 6 0.0 .2
db file single write 200 0 2 9 0.0 .1
log file sync 8 0 1 152 0.0 .1
log file single write 20 0 0 21 0.0 .0
control file sequential re 11,218 0 0 0 0.1 .0
buffer busy waits 2 0 0 161 0.0 .0
direct path write 6 0 0 37 0.0 .0
LGWR wait for redo copy 380 0 0 0 0.0 .0
log buffer space 1 0 0 89 0.0 .0
latch: cache buffers lru c 3 0 0 1 0.0 .0 2 The log file sync is a result of commit --> you are committing too often, maybe even every individual record.
Thanks for explanation. +Actually my question is WHY is it so slow (avg wait of 91ms)+3 Your IO subsystem hosting the online redo log files can be a limiting factor.
We don't know anything about your online redo log configuration
Below is my redo log configuration.
GROUP# STATUS TYPE MEMBER IS_
1 ONLINE /oradata/fs01/PERFDB1/redo_1a.log NO
1 ONLINE /oradata/fs02/PERFDB1/redo_1b.log NO
2 ONLINE /oradata/fs01/PERFDB1/redo_2a.log NO
2 ONLINE /oradata/fs02/PERFDB1/redo_2b.log NO
3 ONLINE /oradata/fs01/PERFDB1/redo_3a.log NO
3 ONLINE /oradata/fs02/PERFDB1/redo_3b.log NO
6 rows selected.
04:13:14 perf_monitor@PERFDB1> col FIRST_CHANGE# for 999999999999999999
04:13:26 perf_monitor@PERFDB1> select *from v$log;
GROUP# THREAD# SEQUENCE# BYTES MEMBERS ARC STATUS FIRST_CHANGE# FIRST_TIME
1 1 40689 524288000 2 YES INACTIVE 13026185905545 18-MAY-13 01:00
2 1 40690 524288000 2 YES INACTIVE 13026185931010 18-MAY-13 03:32
3 1 40691 524288000 2 NO CURRENT 13026185933550 18-MAY-13 04:00Edited by: Kunwar on May 18, 2013 2:46 PM -
Archive log file size is varying in RAC 10g database.
---- Environment oracle 10g rac 9 node cluster database, with 3 log groups for each node with 500 mb size for each redo log file.
Question is why would be the archive log file size is varying, i know when ever there is log file switch the redo log will be archived, So as our redo log file size is of 500 MB
isn't the archive log file size should be the same as 500 MB?
Instead we are seeing the archive log file is varying from 20 MB to 500 MB this means the redo log file is not using the entire 500 MB space? What would be causing this to happen? how can we resolve this?
Some init parameter values.(just for information)
fast_start_mttr_target ----- 400
log_checkpoint_timeout ----- 0
log_checkpoint_interval ----- 0
fast_start_io_target ----- 0There was a similar discussion a few days back,
log file switch before it filled up
The guy later claimed it's because their log_buffer size. It's remain a mystery to me still. -
Performance Issue: Wait event "log file sync" and "Execute to Parse %"
In one of our test environments users are complaining about slow response.
In statspack report folowing are the top-5 wait events
Event Waits Time (cs) Wt Time
log file parallel write 1,046 988 37.71
log file sync 775 774 29.54
db file scattered read 4,946 248 9.47
db file parallel write 66 248 9.47
control file parallel write 188 152 5.80
And after runing the same application 4 times, we are geting Execute to Parse % = 0.10. Cursor sharing is forced and query rewrite is enabled
When I view v$sql, following command is parsed frequently
EXECUTIONS PARSE_CALLS
SQL_TEXT
93380 93380
select SEQ_ORDO_PRC.nextval from DUAL
Please suggest what should be the method to troubleshoot this and if I need to check some more information
Regards,
Sudhanshu BhandariWell, of course, you probably can't eliminate this sort of thing entirely: a setup such as yours is inevitably a compromise. What you can do is make sure your log buffer is a good size (say 10MB or so); that your redo logs are large (at least 100MB each, and preferably large enough to hold one hour or so of redo produced at the busiest time for your database without filling up); and finally set ARCHIVE_LAG_TARGET to something like 1800 seconds or more to ensure a regular, routine, predictable log switch.
It won't cure every ill, but that sort of setup often means the redo subsystem ceases to be a regular driver of foreground waits. -
Delete Log File: Correspondence in Training and Event Management ( t77vp)
Is there a standard way of deleting the Log File: Correspondence in Training and Event Management ( T77VP ) from the system.
Thanks for your help.
AndiHi Niladri,
Please open a new discussion for this as it's a different question. Not only is this stated in the guidelines and makes it easier for other members to search for the right things, but it also increases your chances of getting the right answers, because users know you are looking at LSO rather than TEM and because many users, sadly, are driven by points primarily for giving answers and know you could not mark their answer as correct, because it's not your post.
Please also give context info: which correspondence solution are you using (Smartforms, Adobe forms, SAPscript) and which version of LSO. -
Hi all,
I am using System Center 2012 R2 Operations Manager with the Key Management Service Management Pack at version 6.0.7234.0, and I keep receiving the following alert in Operations Manager:
"Idle Minutes Monitor Alert:
Key Management Service (KMS) inactivity exceeded threshold
Knowledge:
Summary:
The purpose of this rule is to alert system administrators to a possible KMS or network outage. This rule monitors the end-to-end operation of KMS activation. A notification event is created by KMS if no activation or renewal requests were logged by KMS
(activity event 12290) in the specified time interval. In addition to new activations, periodic renewal requests are expected to occur (default is 7 days). Whether or not this alert is serious depends on the number of machines in the KMS environment, how many
are actually connected, and the configured renewal interval.
Causes:
Any failure or incorrect configuration of the KMS service, other Windows components, firewall, hardware, network or routers can trigger the Idle Minutes Alert. This alert can also result from normal behavior, since it is possible that not enough machines
attempted to activate or renew during the specified time interval.
Resolutions:
The first step is to determine whether there really is a problem. Start with a known good KMS client and run (with elevated privileges) the script slmgr.vbs -ato . If the activation/renewal fails, it will report an error code. You can direct the client to
connect to a specific KMS machine by using the slmgr.vbs -skms option. The request event (12288) and response event (12289) in the Windows Application event log may provide additional information, including the identity of failing KMS machines. If there has
been a failure, check the following:
Software Licensing service (slsvc) is running.
Other KMS machine behavior is normal.
KMS firewall port is open (default is TCP 1688).
Attempt to connect to KMS using telnet to the KMS port(you won’t be able to do anything other than connect)
Use a network monitor (e.g. netmon) to capture and trace network problems.
There is one Idle Minutes Monitor that is used to monitor for activity. It may be desirable to adjust the time threshold, depending on expected KMS activity."
Despite what the alert says, there are activity events with ID 12290 being logged, but they are appearing under the 'Key Management Service' log instead of the general 'Application' log. I know that my clients are activating with the server without
any problems as I have run slmgr.vbs -ato with success on a number of them, and none are stating that activation is required. This issue was previously raised here:
http://social.technet.microsoft.com/Forums/systemcenter/en-US/1391acf8-f0be-4a48-9039-8d24e275f1fd/kms-idle-time-monitor-raise-wrong-alerts?forum=operationsmanagermgmtpacks, but I am running Windows Server 2008 R2 SP1 and the hotfix KB981314 comes
up as 'not applicable to this computer' so I assume it is part of SP1 now. I have also tried installing KB2692929, as that was cited as being a possible fix here:
http://social.technet.microsoft.com/Forums/systemcenter/en-US/8ec8ae5b-a310-4b0f-9a5f-e0599bceb93a/kms-managementpack?forum=operationsmanagermgmtpacks, however I am still seeing the same alerts.
Would greatly appreciate any further suggestions!
Many thanksHi,
I've checked for all the events that are generated by our servers, and there are periods of time where no requests are received for up to 18 hours, so that is no doubt why the alerts are appearing (we then sometimes see five requests in the same minute,
but there we go). I'll need to adjust the thresholds.
Thank you for your help, Chunky.1, I'll mark your reply as the answer, as it was very helpful in understanding the alert.
For anyone else who needs to check this, create a PowerShell script using the following content, and place your own details into the relevant sections in bold (without the brackets, of course). This will generate a csv file which will have the server
name in one column and the time that the request event was generated - you can then check the gaps between the requests. The script is quick and dirty, with no checks along the way, so feel free to embellish it as required.
Out-File "<directory to place file in>\KMS_check_results.csv"
$results = get-eventlog -ComputerName <KMS server hostname> -logname 'Key Management Service'|where{$_.message -match "<String which identifies servers instead of clients>"}
$filteredresults = @()
foreach($event in $results){
#This splits up the message into sections, separated by commas, and places each section as an element in an array
$event.Message -split ","|foreach{
if($_ -match "<server domain suffix - e.g. contoso.com>"){
$eventserver = $_
$timegenerated = $event.TimeGenerated
$filteredresults += New-Object PSObject -Property @{
Server = $eventserver
TimeGenerated = $timegenerated
$filteredresults|select-object server,timegenerated|export-csv "<directory to place file in>\KMS_check_results.csv" -NoTypeInformation -
How to capture the current info in the top-of-page event in Reuse block dis
How to capture the current info in the top-of-page event in Reuse block dis
Hi Geetha,
If you don't have any information to pass the Heading Block, then why you are using this event ?
please comment/ remove that TOP_OF_PAGE code. and use subtotal code in field catalog block.
you can use below code for subtotal.
FORM field_catalog .
gs_fcat-do_sum = &2.
fcat : 'WRBTR' '15' 'X' ' ' ' ' 'WRBTR' 'Amount',
ENDFORM.
Regards,
Kunjan -
Wait Events "log file parallel write" / "log file sync" during CREATE INDEX
Hello guys,
at my current project i am performing some performance tests for oracle data guard. The question is "How does a LGWR SYNC transfer influences the system performance?"
To get some performance values, that i can compare i just built up a normal oracle database in the first step.
Now i am performing different tests like creating "large" indexes, massive parallel inserts/commits, etc. to get the bench mark.
My database is an oracle 10.2.0.4 with multiplexed redo log files on AIX.
I am creating an index on a "normal" table .. i execute "dbms_workload_repository.create_snapshot()" before and after the CREATE INDEX to get an equivalent timeframe for the AWR report.
After the index is built up (round about 9 GB) i perform an awrrpt.sql to get the AWR report.
And now take a look at these values from the AWR
Avg
%Time Total Wait wait Waits
Event Waits -outs Time (s) (ms) /txn
log file parallel write 10,019 .0 132 13 33.5
log file sync 293 .7 4 15 1.0
......How can this be possible?
Regarding to the documentation
-> log file sync: http://download.oracle.com/docs/cd/B19306_01/server.102/b14237/waitevents003.htm#sthref3120
Wait Time: The wait time includes the writing of the log buffer and the post.-> log file parallel write: http://download.oracle.com/docs/cd/B19306_01/server.102/b14237/waitevents003.htm#sthref3104
Wait Time: Time it takes for the I/Os to complete. Even though redo records are written in parallel, the parallel write is not complete until the last I/O is on disk.This was also my understanding .. the "log file sync" wait time should be higher than the "log file parallel write" wait time, because of it includes the I/O and the response time to the user session.
I could accept it, if the values are close to each other (maybe round about 1 second in total) .. but the different between 132 seconds and 4 seconds is too noticeable.
Is the behavior of the log file sync/write different when performing a DDL like CREATE INDEX (maybe async .. like you can influence it with the initialization parameter COMMIT_WRITE??)?
Do you have any idea how these values come about?
Any thoughts/ideas are welcome.
Thanks and RegardsSurachart Opun (HunterX) wrote:
Thank you for Nice Idea.
In this case, How can we reduce "log file parallel write" and "log file sync" waited time?
CREATE INDEX with NOLOGGINGA NOLOGGING can help, can't it?Yes - if you create index nologging then you wouldn't be generating that 10GB of redo log, so the waits would disappear.
Two points on nologging, though:
<ul>
it's "only" an index, so you could always rebuild it in the event of media corruption, but if you had lots of indexes created nologging this might cause an unreasonable delay before the system was usable again - so you should decide on a fallback option, such as taking a new backup of the tablespace as soon as all the nologging operatons had completed.
If the database, or that tablespace, is in +"force logging"+ mode, the nologging will not work.
</ul>
Don't get too alarmed by the waits, though. My guess is that the +"log file sync"+ waits are mostly from other sessions, and since there aren't many of them the other sessions are probably not seeing a performance issue. The +"log file parallel write"+ waits are caused by your create index, but they are happeninng to lgwr in the background which is running concurrently with your session - so your session is not (directly) affected by them, so may not be seeing a performance issue.
The other sessions are seeing relatively high sync times because their log file syncs have to wait for one of the large writes that you have triggered to complete, and then the logwriter includes their (little) writes with your next (large) write.
There may be a performance impact, though, from the pure volume of I/O. Apart from the I/O to write the index you have LGWR writting (N copies) of the redo for the index and ARCH is reading and writing the completed log files caused by the index build. So the 9GB of index could easily be responsible for vastly more I/O than the initial 9GB.
Regards
Jonathan Lewis
http://jonathanlewis.wordpress.com
http://www.jlcomp.demon.co.uk
To post code, statspack/AWR report, execution plans or trace files, start and end the section with the tag {noformat}{noformat} (lowercase, curly brackets, no spaces) so that the text appears in fixed format.
"Science is more than a body of knowledge; it is a way of thinking"
Carl Sagan -
Hi All
In Message monitoring(RWB) in adapter engine i am getting the following error
SOAP: response message contains an error XIAdapter/PARSING/ADAPTER.SOAP_EXCEPTION - soap fault: Server was unable to process request. ---> The event log file is full
Can any one suggest me what might be the problem
Thanks
Jayaraman
Edited by: Jayaraman P on May 20, 2010 4:27 PM
Edited by: Jayaraman P on May 20, 2010 4:28 PM>
Jayaraman P wrote:
> SOAP: response message contains an error XIAdapter/PARSING/ADAPTER.SOAP_EXCEPTION - soap fault: Server was unable to process request. ---> The event log file is full
this is because of a problem at the WS server (it might mostly be a windows server).
You can request the WS team to have a look into this issue. it is not a PI problem. -
Check Event Alert failed with error - No errors in the log file.
Hi All,
I am developing a simple event based alert on PO_HEADERS table. I want to send alerts when a PO is created.
I did all the steps according to the metalink note How To Send An Email In A Simple Periodic Or Event Alert? [ID 1162153.1]
When i create the PO, the alert is triggering, and Check Event Alert concurrent program is running. But the program completes with error.
Checking the output file (empty) log file (no errors)
What can i do here to find out what is the problem? There is nothing in the Alert Manager - History form also. I have kept 7 days as days to keep.
Thanks!
MCan you find any details about the error from the "View Detail" button (the same window where you check the log and output files)?
I found the Workflow logs, I am not sure what I am looking for, but i am not seeing any errors reported.The event viewer is supposed to send an email, so do you see anything in the logs that could be related?
Thanks,
Hussein -
No of events in 1231/1131AG log file
Can anyone tell me how to increase the number of events on the log file of the 1231/1131AGs. It seem to be maxing out at 30, and I am unable to go back in the history to view past logs.
Probably your best bet would be to send the events to a logging machine.
Get a copy of Kiwi syslog (it's free) and install it on a PC.
In the WebGUI, bring up the Services page and enable syslog and give it the addres of the PC running Kiwi.
Memory/storage and processor is limited on the APs, it's a good thing to offload as many processes from the APs as possible, especially if you have a heavier traffic load or are situated in an area with a lot of interference.
The text logs created by Kiwi are pretty dense ... you may also want to shop around for a log parser / interpreter.
Kiwi also give you the ability to save the log to a database (mysql, mssql2k & others) and even give you some templates to create the tables.
I use MSSQL2K and report from that with Crystal Reports ... it might be easier to to find a log parser if you're not up on databases.
Good Luck
Scott -
Maximum number of events per audit log file must be greater than 0.
BOE-XI (R2)
Windows Server 2003
Running AUDIT features on all services.
Report Application Server (RAS) keeps giving the following error in the Windows Application Event Log.
Maximum number of events per audit log file must be greater than 0. Defaulting to 500.
I am assuming that this is because the RAS is not being used by anyone at this time - and there is nothing in the local-audit-log to be copied to the AUDIT database.
Is there any way to suppress this error...?
Thanks in advance for the advice!A couple more reboots after applying service pack 3 seemed to fix the issue.
Also had to go to IIS and set the BusinessObjects and CrystalEnterprise11 web sites to use ASP .NET 1.1 instead of 2.
Maybe you are looking for
-
Choice between UPK in-application help and standard EBS help
Hi, We are on a EBS 11.5.10 project, implementing UPK. We got UPK in-application help working; changed custom.pll, set the profile option, everything fine. Now we would like the users to have the choice between UPK help (default) and standard EBS hel
-
i decided to update itunes to itunes 8 and it download from the itunes website store .when i try to install itunes 8 it says "the itunes application could not be opened. A required itunes component is not installed.(-42404)"
-
I have a crystal report to convert in Webi. Universe build in ready. Is there a way to do flexible Grouping on the objects.Please find the screen shot of the crystal reports parameters http://www.forumtopics.com/busobj/viewtopic.php?p=770073#770073 I
-
Hi, I've got another question. Does anybody know, if it is possible to connect a number to a string in a sql-statement. Example: Select counter, counter || 'abc' From table Do I have to do a typecast? ?? Number->String Lg Anna
-
Christmas Greetings to you all! One of my Christmas downloads was McCartneys , The Fireman- Electric Arguments, and very nice it is too! Between track 3 and 4 the sound should be continuous, however i tunes and my 4g nano are both playing a silent ga