Performance of UDP Receiver during high load

Hello All,
I have the following problem and hopefully somebody can help me with this.
Our Application, SIP Proxy, is multi-threaded and this is the way we run it.
We have a UDPReceiver Thread that listens on a particular port. We have a ThreadPoolManager that has 50 Worker threads that are started when the SIP Proxy is brought up. These threads follow the wait-notify mechanism. When we get a packet ( in UDPReceiver),we add the packet to a Vector and notify the threads ( not notifyAll). The thread that is notified picks up the first job in the vector. If there are no jobs in the vectors, the workers all wait on the synchronized object.
So far no issues. Everything works fine when there is reasonable load. But when I reach a certain load I see UDP packets getting dropped (netstat -s -v 1 | grep udpInOverflows in T2000). The CPU utilization at this point is only 40%.)
I face the same packet drop issue in Linux as well. I analyzed the problem and this is what I found out.
Sometimes all the Workers are busy and UDP Receiver adds the jobs to the Vectors and there atleast 300 of these jobs in the queue ( Normally this would be 1 or 2). So when the Workers finish their existing job, they take the next one if there is any. Since there are 300+ jobs, the Workers (50 of them) pick their job. This is continuous till all the jobs in the queue are complete. Now I believe the UDP Receiver thread is not getting a chance to execute during this time
and hence the UDP buffer overflows and we get packet drops ( I have done all the OS tuning to increase the receive buffer etc).
I set a higher priority to the UDP Receiver Thread but still the same thing happened).. What is really puzzling me, how come this
thread is starved for CPU if it shows as only 40% in prstat?
Have any of you faced similar problems and if so how did you fix it?
I can paste code snippets if need be.
Thanks in advance
Gokul

We do have a similar setup to this. I think I would
try to determine what the best case max throughput
you can handle, by disconnecting the receiver from
the queuing and worker thread machinery and just let
it count how many packets it can copy into a buffer
and throw out. This will allow you to get a better
idea of what effect thread contention is having on
your setup.Hi,
Thanks for the quick response. I had done what you have suggested when I was trying out different combinations. The number of workers are something we have a control over at runtime and to ensure that there are no jobs being processed I set the Worker Count at '0'. This correlates to your suggestion. When I do this, there are no packet drops at even very high packet rate. ( Ofcourse the Vector keeps growing since there are no Workers to process it) and eventually I ran out of Memory.
So as far as I can see our UDP Receiver is Ok. So it basically comes down to how do I make sure that UDP Receiver always get top priority?
Have you faced this problem at all or is there is a threshold where you see this problem?
Thanks again
Gokul

Similar Messages

  • During high load the, many times pstack show's __1cDstdQ_STLP_mutex_spin4iA

    Hello,
    I am testing my application with 1500 msg/sec after a while the perfomance drops immediately. The pstack during the pstack show's the below output. Most of the time the pstack in STLP_mutex_spin. I am currently running my application on muti-processor with 32 cores. The application is compiled with stlport4 libraries.
    ffffffff72ed327c nanosleep (ffffffff7fffb800, 0)
    ffffffff7316a4a4 __1cDstdQ_STLP_mutex_spin4iA_N_S_nsec_sleep6Fi_v_ (6, 1, 40, 40, 1, 1) + 1c
    ffffffff7316a320 __1cDstdQ_STLP_mutex_spin4iA_K_M_do_lock6Fpol_v_ (ffffffff73318830, 1, 1, ffffffff7fffb8cc, ffffffff73318838, 1e) + 118
    ffffffff73169f40 __1cDstdM__node_alloc4BiA_N_M_deallocate6FpvL_v_ (100736b08, 8, 7, ffffffff733189b8, 0, ffffffff73310880) + 40
    ffffffff73178ae8 __1cDstdMbasic_string4Ccn0ALchar_traits4Cc__n0AJallocator4Cc___Gappend4Cpkc_6MTD4rkn0AUforward_iterator_tag__r1_ (100e79260, ffffffff7fffdb28, ffffffff7fffdb3e, ffffffff7fffc748, 100e79276, 16) + 140
    ffffffff7756902c __1cDstdMbasic_string4Ccn0ALchar_traits4Cc__n0AJallocator4Cc___2G6Mpkc_r1_ (ffffffff7fffc748, ffffffff7fffdb28, 16, 0, 0, 0) + ac
    ffffffff7ad8e154 __1cGImsDisPDIS_BaseContextFstore6MrnGImsMsgPPhMessagePacker_pkcI66n0AQDIS_ContextState_IIIIbkI_v_ (ffffffff7fffc6a0, 100d23f18, 0, ffffffff, ffffffff7b0026a4, ffffffff7fffdb28) + 124
    ffffffff7ad37f1c __1cGImsDisODIS_DispatcherPprocess_rtp_msg6MrnLRtpComMsg_t_rknPRtpComMsgInfo_t_b_n0AODIS_ReturnCode__ (ffffffff7b002538, ffffffff7fffdb90, ffffffff7fffdb28, 0, 0, 0) + f24
    ffffffff7ad5447c __1cGImsDisODIS_DispatcherMreceive_loop6M_n0AODIS_ReturnCode__ (ffffffff7b002538, ffffffff7fffd118, ffffffff79c29578, 20, 105514, 0) + 3c2c
    ffffffff7ad61cfc __1cGImsDisODIS_DispatcherDrun6Mn0ASDIS_DispatcherType_pn0ASDIS_FeatureManager__i_ (ffffffff7b002538, 0, ffffffff7fffe318, ffffffff79c29e20, ffffffff79c29e48, 1) + 20c
    000000010000d34c main (1, ffffffff7fffe3f8, ffffffff7fffe408, 0, ffffffff7920dfc0, ffffffff79400200) + 564
    The prstat -Lmp show the below oupt. The SLP column increases when the performance drops.
    26385 rtp99 0.0 0.0 0.0 0.0 0.0 100 0.0 0.0 0 0 0 0 UMSdid_cscf/2
    Total: 1 processes, 15 lwps, load averages: 5.63, 5.52, 4.90
    PID USERNAME USR SYS TRP TFL DFL LCK SLP LAT VCX ICX SCL SIG PROCESS/LWPID
    26385 rtp99 73 6.7 0.0 0.0 0.0 20 0.1 0.2 897 147 98K 0 UMSdid_cscf/1
    26385 rtp99 20 4.2 0.1 0.0 0.0 74 0.0 2.0 11K 371 19K 0 UMSdid_cscf/7
    26385 rtp99 20 4.2 0.0 0.0 0.0 74 0.0 2.1 11K 327 18K 0 UMSdid_cscf/4
    26385 rtp99 20 4.0 0.0 0.0 0.0 74 0.0 2.0 11K 290 18K 0 UMSdid_cscf/5
    26385 rtp99 19 4.1 0.0 0.0 0.0 74 0.0 2.1 11K 247 18K 0 UMSdid_cscf/6
    26385 rtp99 19 4.1 0.1 0.0 0.0 74 0.0 2.1 11K 378 18K 0 UMSdid_cscf/3
    26385 rtp99 3.1 3.4 0.0 0.0 0.0 0.0 93 0.6 2K 127 17K 0 UMSdid_cscf/11
    26385 rtp99 2.5 3.2 0.0 0.0 0.0 94 0.0 0.6 3K 24 7K 0 UMSdid_cscf/10
    26385 rtp99 1.8 2.1 0.0 0.0 0.0 0.0 96 0.3 1K 15 9K 0 UMSdid_cscf/15
    26385 rtp99 1.7 1.9 0.0 0.0 0.0 0.0 96 0.3 1K 0 9K 0 UMSdid_cscf/13
    26385 rtp99 1.4 2.0 0.0 0.0 0.0 96 0.0 0.4 2K 1 4K 0 UMSdid_cscf/14
    26385 rtp99 1.4 1.8 0.0 0.0 0.0 96 0.0 0.4 2K 1 4K 0 UMSdid_cscf/12
    26385 rtp99 0.0 0.0 0.0 0.0 0.0 100 0.2 0.0 3 0 19 0 UMSdid_cscf/8
    26385 rtp99 0.0 0.0 0.0 0.0 0.0 100 0.0 0.0 4 0 16 0 UMSdid_cscf/9
    26385 rtp99 0.0 0.0 0.0 0.0 0.0 100 0.0 0.0 0 0 0 0 UMSdid_cscf/2
    Total: 1 processes, 15 lwps, load averages: 5.61, 5.52, 4.91
    PID USERNAME USR SYS TRP TFL DFL LCK SLP LAT VCX ICX SCL SIG PROCESS/LWPID
    26385 rtp99 68 6.1 0.0 0.0 0.0 18 7.8 0.2 756 170 92K 0 UMSdid_cscf/1
    26385 rtp99 19 3.6 0.0 0.0 0.0 69 6.5 1.9 9K 287 16K 0 UMSdid_cscf/5
    26385 rtp99 18 3.6 0.0 0.0 0.0 70 6.3 1.8 9K 319 17K 0 UMSdid_cscf/3
    26385 rtp99 18 3.6 0.0 0.0 0.0 69 7.7 1.8 9K 308 16K 0 UMSdid_cscf/4
    26385 rtp99 18 3.6 0.0 0.0 0.0 69 7.0 1.8 9K 238 16K 0 UMSdid_cscf/6
    26385 rtp99 18 3.6 0.0 0.0 0.0 67 10 1.8 9K 257 16K 0 UMSdid_cscf/7
    26385 rtp99 2.9 3.2 0.0 0.0 0.0 0.1 93 0.6 2K 156 16K 0 UMSdid_cscf/11
    26385 rtp99 2.3 3.0 0.0 0.0 0.0 94 0.0 0.7 3K 18 6K 0 UMSdid_cscf/10
    26385 rtp99 1.7 2.0 0.0 0.0 0.0 0.2 96 0.3 1K 12 9K 0 UMSdid_cscf/15
    26385 rtp99 1.6 1.7 0.0 0.0 0.0 0.4 96 0.3 1K 11 9K 0 UMSdid_cscf/13
    26385 rtp99 1.4 1.9 0.0 0.0 0.0 96 0.0 0.4 1K 14 3K 0 UMSdid_cscf/14
    26385 rtp99 1.4 1.6 0.0 0.0 0.0 97 0.0 0.4 1K 6 3K 0 UMSdid_cscf/12
    26385 rtp99 0.0 0.0 0.0 0.0 0.0 100 0.2 0.0 3 0 19 0 UMSdid_cscf/8
    26385 rtp99 0.0 0.0 0.0 0.0 0.0 100 0.0 0.0 8 0 27 0 UMSdid_cscf/9
    Total: 1 processes, 15 lwps, load averages: 5.58, 5.52, 4.91
    PID USERNAME USR SYS TRP TFL DFL LCK SLP LAT VCX ICX SCL SIG PROCESS/LWPID
    26385 rtp99 60 5.1 0.1 0.0 0.0 12 23 0.2 613 226 80K 0 UMSdid_cscf/1
    26385 rtp99 16 2.7 0.0 0.0 0.0 59 21 1.3 6K 218 12K 0 UMSdid_cscf/7
    26385 rtp99 16 2.6 0.0 0.0 0.0 61 19 1.3 6K 175 12K 0 UMSdid_cscf/3
    26385 rtp99 16 2.6 0.0 0.0 0.0 57 23 1.4 6K 213 12K 0 UMSdid_cscf/6
    26385 rtp99 16 2.6 0.0 0.0 0.0 58 22 1.3 6K 203 12K 0 UMSdid_cscf/5
    26385 rtp99 16 2.6 0.0 0.0 0.0 58 23 1.3 6K 203 12K 0 UMSdid_cscf/4
    26385 rtp99 2.4 2.6 0.0 0.0 0.0 0.4 94 0.4 2K 61 13K 0 UMSdid_cscf/11
    26385 rtp99 2.1 2.6 0.0 0.0 0.0 95 0.0 0.6 2K 20 5K 0 UMSdid_cscf/10
    26385 rtp99 1.5 1.7 0.0 0.0 0.0 0.0 97 0.3 1K 23 7K 0 UMSdid_cscf/15
    26385 rtp99 1.4 1.5 0.0 0.0 0.0 0.6 96 0.3 1K 12 7K 0 UMSdid_cscf/13
    26385 rtp99 1.2 1.5 0.0 0.0 0.0 97 0.0 0.3 1K 3 3K 0 UMSdid_cscf/14
    26385 rtp99 1.2 1.4 0.0 0.0 0.0 97 0.0 0.3 1K 1 3K 0 UMSdid_cscf/12
    26385 rtp99 0.0 0.0 0.0 0.0 0.0 100 0.2 0.0 3 0 19 0 UMSdid_cscf/8
    26385 rtp99 0.0 0.0 0.0 0.0 0.0 100 0.0 0.0 8 0 25 0 UMSdid_cscf/9
    26385 rtp99 0.0 0.0 0.0 0.0 0.0 100 0.0 0.0 0 0 0 0 UMSdid_cscf/2
    Pls help, Thanks in Advance
    Regards,
    Baaasssi

    First, compiler and system messages are hard to read unless you enclose them in "code" blocks. The Forum software otherwise interprets some character sequences as formatting directives. But I was able to puzzle out the actual mangled names of functions.
    You appear to have hit the STLport default-allocator bottleneck. If multiple threads are frequently allocating and deallocating library objects (like strings), they block one another via a spin lock. Although the STLport source code has some alternative allocators, they don't seem to work well.
    Some alternatives:
    1. Try another implementation of the C++ Standard library, such as Apache stdcxx
    [http://stdcxx.apache.org/]
    Refer to the C++ Users Guide section 12.7 "Replacing the C++ Standard Library".
    2. Write or find a replacement allocator, and use it explicitly throughout your code instead of the default allocator.
    3. See whether you can reduce the number of actual object allocations and deallocations by managing objects on your own free list.

  • Strange Oracle Behaviour - only month end or high load periods

    Hi SAP Gurus
    In one of our systems I am expreriencing a strange behaviour from Oracle, in that it seems to be in reconnect, but yet it is not, it resets and carries on but do produce Dump and SM21 log messages, the dump is not really helpfull as it is the same dump but with different program everytime.
    I have looked in oracle alert log and found no event tying back to the SM21 log message or dump.
    The following is what I found in the developer trace of one of these insident, but there are many, about 125 - 200 for the day during high load:
    Fri Nov 28 10:05:55 2008
    C  *** ERROR => Failed to register application info: orc=3113
    [dboci.c      3042]
    C  *** ERROR => Error 3114 in stmt_fetch()
    [dbsloci.c    13001]
    C  *** ERROR => ORA-3114 occurred when executing SQL statement (parse error offset=0)
    [dbsloci.c    13002]
    C  sc_p=045B18B4,no=29,idc_p=00000000,con=0,act=1,slen=1291,smax=1536,#vars=4,stmt=34DA78C8,table=CATSDB                       
    C  SELECT "MANDT" , "COUNTER" , "PERNR" , "WORKDATE" , "SKOSTL" , "LSTAR" , "SEBELN" , "SEBELP" , "SPRZ\
    C  NR" , "LSTNR" , "RKOSTL" , "RPROJ" , "RAUFNR" , "RNPLNR" , "RAUFPL" , "RAPLZL" , "RKDAUF" , "RKDPOS"\
    C   , "RKSTR" , "RPRZNR" , "PAOBJNR" , "FUND" , "FUNC_AREA" , "GRANT_NBR" , "S_FUND" , "S_FUNC_AREA" , \
    C  "S_GRANT_NBR" , "AWART" , "LGART" , "KAPID" , "SPLIT" , "REINR" , "WABLNR" , "VERSL" , "WTART" , "BW\
    C  GRL" , "WAERS" , "AUFKZ" , "TRFGR" , "TRFST" , "PRAKN" , "PRAKZ" , "OTYPE" , "PLANS" , "KOKRS" , "ME\
    C  INH" , "TCURR" , "PRICE" , "ARBID" , "WERKS" , "AUTYP" , "HRCOSTASG" , "HRKOSTL" , "HRLSTAR" , "HRFU\
    C  ND" , "HRFUNC_AREA" , "HRGRANT_NBR" , "BEMOT" , "UNIT" , "STATKEYFIG" , "TASKTYPE" , "TASKLEVEL" , "\
    C  TASKCOMPONENT" , "BUKRS" , "ERSDA" , "ERSTM" , "ERNAM" , "LAEDA" , "LAETM" , "AENAM" , "APNAM" , "AP\
    C  DAT" , "WORKITEMID" , "LOGSYS" , "STATUS" , "REFCOUNTER" , "REASON" , "BELNR" , "EXTSYSTEM" , "EXTAP\
    C  PLICATION" , "EXTDOCUMENTNO" , "TASKCOUNTER" , "CATSHOURS" , "BEGUZ" , "ENDUZ" , "VTKEN" , "ALLDF" ,\
    C   "OFMNW" , "PEDD" , "AUERU" , "LTXA1" , "LONGTEXT" , "ERUZU" , "CATSAMOUNT" , "CATSQUANTITY" , "ZCAT\
    C  SPLAN" , "ROUTE" , "KUNNR" , "MVGR1" , "GSBER" , "SNAME" , "MAKTX" , "ZAVGHRS" , "ZQANT" , "ZPERNR" \
    C  FROM "CATSDB" WHERE "MANDT" = :A0 AND "WORKDATE" = :A1 AND "WERKS" = :A2 AND "STATUS" < :A3;
    C  sc_p=045B18B4,no=29,idc_p=00000000,con=0,act=1,slen=1291,smax=1536,#vars=4,stmt=34DA78C8,table=CATSDB                       
    C  prep=0,lit=0,nsql=0,lobret=0,#exec=1,dbcnt=0,upsh_p=00000000,ocistmth_p=345E20EC
    C  IN : cols=4,rmax=1,xcnt=0,rpc=0,rowi=0,rtot=0,upto=-1,rsize=17,vmax=32,bound=1,iobuf_p=045DFFA8,vda_p=3470A8D8
    C       lobs=0,lmax=0,lpcnt=0,larr=00000000,lcurr_p=00000000,rret=0
    C  OUT: cols=105,rmax=58,xcnt=58,rpc=1,rowi=0,rtot=1,upto=-1,rsize=895,vmax=128,bound=1,iobuf_p=0E0A4EC8,vda_p=34F8F7E8
    C       lobs=0,lmax=0,lpcnt=0,larr=00000000,lcurr_p=00000000,rret=0
    C  SELECT "MANDT" , "COUNTER" , "PERNR" , "WORKDATE" , "SKOSTL" , "LSTAR" , "SEBELN" , "SEBELP" , "SPRZ\
    C  NR" , "LSTNR" , "RKOSTL" , "RPROJ" , "RAUFNR" , "RNPLNR" , "RAUFPL" , "RAPLZL" , "RKDAUF" , "RKDPOS"\
    C   , "RKSTR" , "RPRZNR" , "PAOBJNR" , "FUND" , "FUNC_AREA" , "GRANT_NBR" , "S_FUND" , "S_FUNC_AREA" , \
    C  "S_GRANT_NBR" , "AWART" , "LGART" , "KAPID" , "SPLIT" , "REINR" , "WABLNR" , "VERSL" , "WTART" , "BW\
    C  GRL" , "WAERS" , "AUFKZ" , "TRFGR" , "TRFST" , "PRAKN" , "PRAKZ" , "OTYPE" , "PLANS" , "KOKRS" , "ME\
    C  INH" , "TCURR" , "PRICE" , "ARBID" , "WERKS" , "AUTYP" , "HRCOSTASG" , "HRKOSTL" , "HRLSTAR" , "HRFU\
    C  ND" , "HRFUNC_AREA" , "HRGRANT_NBR" , "BEMOT" , "UNIT" , "STATKEYFIG" , "TASKTYPE" , "TASKLEVEL" , "\
    C  TASKCOMPONENT" , "BUKRS" , "ERSDA" , "ERSTM" , "ERNAM" , "LAEDA" , "LAETM" , "AENAM" , "APNAM" , "AP\
    C  DAT" , "WORKITEMID" , "LOGSYS" , "STATUS" , "REFCOUNTER" , "REASON" , "BELNR" , "EXTSYSTEM" , "EXTAP\
    C  PLICATION" , "EXTDOCUMENTNO" , "TASKCOUNTER" , "CATSHOURS" , "BEGUZ" , "ENDUZ" , "VTKEN" , "ALLDF" ,\
    C   "OFMNW" , "PEDD" , "AUERU" , "LTXA1" , "LONGTEXT" , "ERUZU" , "CATSAMOUNT" , "CATSQUANTITY" , "ZCAT\
    C  SPLAN" , "ROUTE" , "KUNNR" , "MVGR1" , "GSBER" , "SNAME" , "MAKTX" , "ZAVGHRS" , "ZQANT" , "ZPERNR" \
    C  FROM "CATSDB" WHERE "MANDT" = :A0 AND "WORKDATE" = :A1 AND "WERKS" = :A2 AND "STATUS" < :A3;
    B  ***LOG BYM=> severe DB error 3114      ; work process in reconnect status [dbsh#3 @ 1123] [dbsh    1123 ]
    B  ***LOG BY4=> sql error 3114   performing FET on table CATSDB     [dbtran#10 @ 7292] [dbtran  7292 ]
    B  ***LOG BY0=> ORA-03114: not connected to ORACLE [dbtran#10 @ 7292] [dbtran  7292 ]
    B  dbtran ERROR LOG (hdl_dbsl_error): DbSl 'FET'
    B   RSLT: {dbsl=99, tran=1}
    B   FHDR: {tab='CATSDB', fcode=225, mode=2, bpb=0, dbcnt=0, crsr=1,
    B          hold=0, keep=1, xfer=0, pkg=0, upto=0, init:b=0,
    B          init:p=00000000, init:#=871, wa:p=0XA000DC88, wa:#=871}
    B  dbtran ERROR LOG (hdl_dbsl_error): DbSl 'FET'
    B   STMT:
    B   CRSR: {tab='CATSDB', id=1, hold=0, prop=0x1000, max.in@0=1, fae:blk=1,
    B          con:id=0, con:vndr=5, val=2,
    B          key:#=4, xfer=0, xin:#=0, row:#=0, upto=0, wa:p=0XA000DC88}
    C  *** ERROR => Error 3114 in stmt_fetch()
    [dbsloci.c    13001]
    C  *** ERROR => ORA-3114 occurred when executing SQL statement (parse error offset=0)
    [dbsloci.c    13002]
    C  sc_p=045B52E4,no=225,idc_p=045C400C,con=0,act=1,slen=184,smax=2048,#vars=3,stmt=35BBB778,table=REPOLOAD                     
    C  SELECT "UNAM" , "UDAT" , "UTIME" , "L_DATALG" , "Q_DATALG" , "SDAT" , "STIME" , "MINOR_VERS" , "MAJO\
    C  R_VERS" FROM "REPOLOAD" WHERE "PROGNAME" = :A0 AND "R3STATE" = :A1 AND "MACH" = :A2 ;
    C  sc_p=045B52E4,no=225,idc_p=045C400C,con=0,act=1,slen=184,smax=2048,#vars=3,stmt=35BBB778,table=REPOLOAD                     
    C  prep=0,lit=0,nsql=0,lobret=0,#exec=29,dbcnt=0,upsh_p=00000000,ocistmth_p=34B9ADD4
    C  IN : cols=3,rmax=1,xcnt=0,rpc=0,rowi=0,rtot=0,upto=-1,rsize=45,vmax=32,bound=1,iobuf_p=0E0A4EC8,vda_p=31361B18
    C       lobs=0,lmax=0,lpcnt=0,larr=00000000,lcurr_p=00000000,rret=0
    C  OUT: cols=9,rmax=1,xcnt=1,rpc=1,rowi=0,rtot=1,upto=-1,rsize=56,vmax=32,bound=1,iobuf_p=045DFFA8,vda_p=345C39B8
    C       lobs=0,lmax=0,lpcnt=0,larr=00000000,lcurr_p=00000000,rret=0
    C  SELECT "UNAM" , "UDAT" , "UTIME" , "L_DATALG" , "Q_DATALG" , "SDAT" , "STIME" , "MINOR_VERS" , "MAJO\
    C  R_VERS" FROM "REPOLOAD" WHERE "PROGNAME" = :A0 AND "R3STATE" = :A1 AND "MACH" = :A2 ;
    B  ***LOG BYM=> severe DB error 3114      ; work process in reconnect status [dbsh#3 @ 1123] [dbsh    1123 ]
    B  ***LOG BY4=> sql error 3114   performing SEL on table REPOLOAD   [dbrepo#6 @ 2657] [dbrepo  2657 ]
    B  ***LOG BY0=> ORA-03114: not connected to ORACLE [dbrepo#6 @ 2657] [dbrepo  2657 ]
    A  TH VERBOSE LEVEL FULL
    A  ** RABAX: level LEV_RX_PXA_RELEASE_MTX entered.
    A  ** RABAX: level LEV_RX_PXA_RELEASE_MTX completed.
    A  ** RABAX: level LEV_RX_VMIT_CLEANUP entered.
    A  ** RABAX: level LEV_RX_VMIT_CLEANUP completed.
    A  ** RABAX: level LEV_RX_ROLLBACK entered.
    C  *** ERROR => OCI-call 'OCITransRollback' failed: rc = 3114
    [dboci.c      3127]
    B  ***LOG BYM=> severe DB error 3114      ; work process in reconnect status [dbsh#3 @ 1123] [dbsh    1123 ]
    B  Reconnect state is entered by connection:
    B  0: name = R/3, con_id = 000000000 state = ACTIVE      , perm = YES, reco = YES, timeout = 000, con_max = 255, con_opt = 255, occ = NO
    B  hdl_error_on_commit_rollback: DB-ROLLBACK detected RECONNECT state
    M  ThShortCommit: db unusable
    B  db_con_reconnect disconnecting connection:
    B  0: name = R/3, con_id = 000000000 state = INACTIVE    , perm = YES, reco = YES, timeout = 000, con_max = 255, con_opt = 255, occ = NO
    C  Disconnecting from connection 0 ...
    C  Close user session (con_hdl=0,svchp=045D7FA4,usrhp=345E67A0)
    C  Detaching from DB Server (con_hdl=0,svchp=045D7FA4,srvhp=045D0174)
    C  Now I'm disconnected from ORACLE
    B  db_con_reconnect performing the reconnect for con:
    B  0: name = R/3, con_id = 000000000 state = DISCONNECTED, perm = YES, reco = YES, timeout = 000, con_max = 255, con_opt = 255, occ = NO
    C  got NLS_LANG='AMERICAN_AMERICA.US7ASCII' from environment
    C  Client NLS settings:
    C  Logon as OPS$-user to get SAPR3's password
    C  Connecting as /@PRD on connection 0 (nls_hdl 0) ... (dbsl 640 250407)
    C  Nls CharacterSet                 NationalCharSet              C      EnvHp      ErrHp ErrHpBatch
    C    0 US7ASCII                                                  1   045CBB38   0E0A4648   0E0A40B0
    C  Attaching to DB Server PRD (con_hdl=0,svchp=045
    SM51 log entries:
         09:00:32     MS                         Q0I     Operating system call recv failed (error no. 10054)
         09:00:32     DP                         Q0K     Connection to message server (on FCMSPROD) established
         09:00:32     DIA     004     040     181JOLENES     VA02     BYM     SQL error 3114 . Work processes in reconnect status
         09:00:32     DP                         Q1C     MsgServer Hardware ID Was Determined
         09:00:32     DIA     001     040     181ANNELINEJ     VA42     BYY     Work process has left reconnect status
         09:00:32     DIA     001     040     181ANNELINEJ     VA42     AB0     Run-time error "DBIF_REPO_SQL_ERROR" occurred
         09:00:32     DIA     014     040     161ADRIV     VA02     BYY     Work process has left reconnect status
         09:00:32     DIA     013     040     140ELECIA          BYY     Work process has left reconnect status
         09:00:32     DIA     003     040     133MIRANDAV     KO88     BYY     Work process has left reconnect status
         09:00:32     DIA     003     040     133MIRANDAV     KO88     AB0     Run-time error "DBIF_REPO_SQL_ERROR" occurred
         09:00:32     DIA     001     040     181ANNELINEJ     VA42     AB1     > Short dump "081128 090032 FCMSPROD 181ANNELINEJ" generated
         09:00:32     DIA     003     040     133MIRANDAV     KO88     AB1     > Short dump "081128 090032 FCMSPROD 133MIRANDAV " generated
         09:00:32     DIA     009     040     120ELIZEP     FB01     BYY     Work process has left reconnect status
         09:00:33     DIA     009     040     120ELIZEP     FB01     AB0     Run-time error "DBIF_REPO_SQL_ERROR" occurred
         09:00:33     DIA     004     040     181JOLENES     VA02     BYY     Work process has left reconnect status
         09:00:33     DIA     002     040     133KOEKIEJ          BYY     Work process has left reconnect status
         09:00:33     DIA     002     040     133KOEKIEJ          AB0     Run-time error "DBIF_REPO_SQL_ERROR" occurred
         09:00:33     DIA     004     040     181JOLENES     VA02     AB0     Run-time error "DBIF_REPO_SQL_ERROR" occurred
         09:00:33     DIA     009     040     120ELIZEP     FB01     AB1     > Short dump "081128 090033 FCMSPROD 120ELIZEP " generated
         09:00:33     DIA     005     040     181CLIVEB     VA02     BYY     Work process has left reconnect status
         09:00:33     DIA     003                    R0Z     The update dispatch info was reset
         09:00:33     DIA     003                    R0Z     The update dispatch info was reset
    Please can you assist in maybe you have seen this before.
    Regards
    Pieter

    Hi
    Yes I have gone through the Oracle Alert log several time, It is huge, but I am pasting an extract for the entire day this was happening into this reply.  Oracle version is 9.2.0.8.0 with the only intrim path I can see being installed : 6130293.
    Following is the extract from Alert Log for the day this occurred, again I must stress that this happens only on high system load days like month end:
    Fri Nov 28 03:43:33 2008
    ALTER TABLESPACE "PSAPBTABI" END BACKUP
    Fri Nov 28 03:43:33 2008
    Completed: ALTER TABLESPACE "PSAPBTABI" END BACKUP
    Fri Nov 28 03:43:37 2008
    ALTER TABLESPACE "PSAPCLUD" BEGIN BACKUP
    Completed: ALTER TABLESPACE "PSAPCLUD" BEGIN BACKUP
    Fri Nov 28 03:53:38 2008
    ALTER TABLESPACE "PSAPCLUD" END BACKUP
    Fri Nov 28 03:53:38 2008
    Completed: ALTER TABLESPACE "PSAPCLUD" END BACKUP
    Fri Nov 28 03:53:42 2008
    ALTER TABLESPACE "PSAPCLUI" BEGIN BACKUP
    Completed: ALTER TABLESPACE "PSAPCLUI" BEGIN BACKUP
    Fri Nov 28 03:54:36 2008
    ALTER TABLESPACE "PSAPCLUI" END BACKUP
    Fri Nov 28 03:54:36 2008
    Completed: ALTER TABLESPACE "PSAPCLUI" END BACKUP
    Fri Nov 28 03:54:38 2008
    ALTER TABLESPACE "PSAPDDICD" BEGIN BACKUP
    Completed: ALTER TABLESPACE "PSAPDDICD" BEGIN BACKUP
    Fri Nov 28 03:55:47 2008
    ALTER TABLESPACE "PSAPDDICD" END BACKUP
    Fri Nov 28 03:55:47 2008
    Completed: ALTER TABLESPACE "PSAPDDICD" END BACKUP
    Fri Nov 28 03:55:50 2008
    ALTER TABLESPACE "PSAPDDICI" BEGIN BACKUP
    Completed: ALTER TABLESPACE "PSAPDDICI" BEGIN BACKUP
    Fri Nov 28 03:56:41 2008
    ALTER TABLESPACE "PSAPDDICI" END BACKUP
    Fri Nov 28 03:56:41 2008
    Completed: ALTER TABLESPACE "PSAPDDICI" END BACKUP
    Fri Nov 28 03:56:45 2008
    ALTER TABLESPACE "PSAPDOCUD" BEGIN BACKUP
    Completed: ALTER TABLESPACE "PSAPDOCUD" BEGIN BACKUP
    Fri Nov 28 03:56:49 2008
    ALTER TABLESPACE "PSAPDOCUD" END BACKUP
    Completed: ALTER TABLESPACE "PSAPDOCUD" END BACKUP
    Fri Nov 28 03:56:52 2008
    ALTER TABLESPACE "PSAPDOCUI" BEGIN BACKUP
    Completed: ALTER TABLESPACE "PSAPDOCUI" BEGIN BACKUP
    Fri Nov 28 03:56:55 2008
    ALTER TABLESPACE "PSAPDOCUI" END BACKUP
    Completed: ALTER TABLESPACE "PSAPDOCUI" END BACKUP
    Fri Nov 28 03:56:59 2008
    ALTER TABLESPACE "PSAPEL640D" BEGIN BACKUP
    Completed: ALTER TABLESPACE "PSAPEL640D" BEGIN BACKUP
    Fri Nov 28 04:03:46 2008
    ALTER TABLESPACE "PSAPEL640D" END BACKUP
    Fri Nov 28 04:03:46 2008
    Completed: ALTER TABLESPACE "PSAPEL640D" END BACKUP
    Fri Nov 28 04:03:50 2008
    ALTER TABLESPACE "PSAPEL640I" BEGIN BACKUP
    Completed: ALTER TABLESPACE "PSAPEL640I" BEGIN BACKUP
    Fri Nov 28 04:04:37 2008
    ALTER TABLESPACE "PSAPEL640I" END BACKUP
    Fri Nov 28 04:04:37 2008
    Completed: ALTER TABLESPACE "PSAPEL640I" END BACKUP
    Fri Nov 28 04:04:40 2008
    ALTER TABLESPACE "PSAPES640D" BEGIN BACKUP
    Completed: ALTER TABLESPACE "PSAPES640D" BEGIN BACKUP
    Fri Nov 28 04:20:17 2008
    ALTER TABLESPACE "PSAPES640D" END BACKUP
    Fri Nov 28 04:20:17 2008
    Completed: ALTER TABLESPACE "PSAPES640D" END BACKUP
    Fri Nov 28 04:20:20 2008
    ALTER TABLESPACE "PSAPES640I" BEGIN BACKUP
    Completed: ALTER TABLESPACE "PSAPES640I" BEGIN BACKUP
    Fri Nov 28 04:36:26 2008
    ALTER TABLESPACE "PSAPES640I" END BACKUP
    Fri Nov 28 04:36:26 2008
    Completed: ALTER TABLESPACE "PSAPES640I" END BACKUP
    Fri Nov 28 04:36:29 2008
    ALTER TABLESPACE "PSAPLOADD" BEGIN BACKUP
    Completed: ALTER TABLESPACE "PSAPLOADD" BEGIN BACKUP
    Fri Nov 28 04:36:40 2008
    ALTER TABLESPACE "PSAPLOADD" END BACKUP
    Fri Nov 28 04:36:40 2008
    Completed: ALTER TABLESPACE "PSAPLOADD" END BACKUP
    Fri Nov 28 04:36:43 2008
    ALTER TABLESPACE "PSAPLOADI" BEGIN BACKUP
    Completed: ALTER TABLESPACE "PSAPLOADI" BEGIN BACKUP
    Fri Nov 28 04:36:53 2008
    ALTER TABLESPACE "PSAPLOADI" END BACKUP
    Completed: ALTER TABLESPACE "PSAPLOADI" END BACKUP
    Fri Nov 28 04:36:56 2008
    ALTER TABLESPACE "PSAPPOOLD" BEGIN BACKUP
    Completed: ALTER TABLESPACE "PSAPPOOLD" BEGIN BACKUP
    Fri Nov 28 04:41:04 2008
    ALTER TABLESPACE "PSAPPOOLD" END BACKUP
    Fri Nov 28 04:41:04 2008
    Completed: ALTER TABLESPACE "PSAPPOOLD" END BACKUP
    Fri Nov 28 04:41:08 2008
    ALTER TABLESPACE "PSAPPOOLI" BEGIN BACKUP
    Completed: ALTER TABLESPACE "PSAPPOOLI" BEGIN BACKUP
    Fri Nov 28 04:45:17 2008
    ALTER TABLESPACE "PSAPPOOLI" END BACKUP
    Fri Nov 28 04:45:17 2008
    Completed: ALTER TABLESPACE "PSAPPOOLI" END BACKUP
    Fri Nov 28 04:45:21 2008
    ALTER TABLESPACE "PSAPPROTD" BEGIN BACKUP
    Completed: ALTER TABLESPACE "PSAPPROTD" BEGIN BACKUP
    Fri Nov 28 04:47:25 2008
    ALTER TABLESPACE "PSAPPROTD" END BACKUP
    Fri Nov 28 04:47:25 2008
    Completed: ALTER TABLESPACE "PSAPPROTD" END BACKUP
    Fri Nov 28 04:47:29 2008
    ALTER TABLESPACE "PSAPPROTI" BEGIN BACKUP
    Completed: ALTER TABLESPACE "PSAPPROTI" BEGIN BACKUP
    Fri Nov 28 04:48:09 2008
    ALTER TABLESPACE "PSAPPROTI" END BACKUP
    Fri Nov 28 04:48:09 2008
    Completed: ALTER TABLESPACE "PSAPPROTI" END BACKUP
    Fri Nov 28 04:48:13 2008
    ALTER TABLESPACE "PSAPSOURCED" BEGIN BACKUP
    Completed: ALTER TABLESPACE "PSAPSOURCED" BEGIN BACKUP
    Fri Nov 28 04:48:46 2008
    ALTER TABLESPACE "PSAPSOURCED" END BACKUP
    Fri Nov 28 04:48:46 2008
    Completed: ALTER TABLESPACE "PSAPSOURCED" END BACKUP
    Fri Nov 28 04:48:49 2008
    ALTER TABLESPACE "PSAPSOURCEI" BEGIN BACKUP
    Completed: ALTER TABLESPACE "PSAPSOURCEI" BEGIN BACKUP
    Fri Nov 28 04:49:29 2008
    ALTER TABLESPACE "PSAPSOURCEI" END BACKUP
    Fri Nov 28 04:49:29 2008
    Completed: ALTER TABLESPACE "PSAPSOURCEI" END BACKUP
    Fri Nov 28 04:49:34 2008
    ALTER TABLESPACE "PSAPSTABD" BEGIN BACKUP
    Completed: ALTER TABLESPACE "PSAPSTABD" BEGIN BACKUP
    Fri Nov 28 05:05:16 2008
    ALTER TABLESPACE "PSAPSTABD" END BACKUP
    Fri Nov 28 05:05:16 2008
    Completed: ALTER TABLESPACE "PSAPSTABD" END BACKUP
    Fri Nov 28 05:05:23 2008
    ALTER TABLESPACE "PSAPSTABI" BEGIN BACKUP
    Completed: ALTER TABLESPACE "PSAPSTABI" BEGIN BACKUP
    Fri Nov 28 05:21:15 2008
    ALTER TABLESPACE "PSAPSTABI" END BACKUP
    Fri Nov 28 05:21:15 2008
    Completed: ALTER TABLESPACE "PSAPSTABI" END BACKUP
    Fri Nov 28 05:21:19 2008
    ALTER TABLESPACE "PSAPUSER1D" BEGIN BACKUP
    Completed: ALTER TABLESPACE "PSAPUSER1D" BEGIN BACKUP
    Fri Nov 28 05:21:42 2008
    ALTER TABLESPACE "PSAPUSER1D" END BACKUP
    Fri Nov 28 05:21:42 2008
    Completed: ALTER TABLESPACE "PSAPUSER1D" END BACKUP
    Fri Nov 28 05:21:46 2008
    ALTER TABLESPACE "PSAPUSER1I" BEGIN BACKUP
    Completed: ALTER TABLESPACE "PSAPUSER1I" BEGIN BACKUP
    Fri Nov 28 05:23:03 2008
    ALTER TABLESPACE "PSAPUSER1I" END BACKUP
    Fri Nov 28 05:23:03 2008
    Completed: ALTER TABLESPACE "PSAPUSER1I" END BACKUP
    Fri Nov 28 05:23:06 2008
    ALTER TABLESPACE "SYSTEM" BEGIN BACKUP
    Completed: ALTER TABLESPACE "SYSTEM" BEGIN BACKUP
    Fri Nov 28 05:24:03 2008
    ALTER TABLESPACE "SYSTEM" END BACKUP
    Fri Nov 28 05:24:03 2008
    Completed: ALTER TABLESPACE "SYSTEM" END BACKUP
    Fri Nov 28 05:24:07 2008
    ALTER TABLESPACE "UNDOTBS1" BEGIN BACKUP
    Completed: ALTER TABLESPACE "UNDOTBS1" BEGIN BACKUP
    Fri Nov 28 05:42:06 2008
    ALTER TABLESPACE "UNDOTBS1" END BACKUP
    Fri Nov 28 05:42:06 2008
    Completed: ALTER TABLESPACE "UNDOTBS1" END BACKUP
    Fri Nov 28 05:42:07 2008
    ALTER DATABASE BACKUP CONTROLFILE TO 'C:\Program Files\CA\BrightStor ARCserve Backup Enterprise Option for SAP R3 for Oracle\CONTROL.PRD' REUSE
    Completed: ALTER DATABASE BACKUP CONTROLFILE TO 'C:\Program F
    Fri Nov 28 05:42:14 2008
    Beginning log switch checkpoint up to RBA [0x77f2.2.10], SCN: 0x0000.3c0ae436
    Thread 1 advanced to log sequence 30706
      Current log# 2 seq# 30706 mem# 0: C:\ORACLE\PRD\ORIGLOGB\LOGPRD02_M1.ORA
      Current log# 2 seq# 30706 mem# 1: F:\ORACLE\PRD\MIRRLOGB\LOGPRD02_M2.ORA
    Fri Nov 28 05:42:14 2008
    ARC0: Evaluating archive   log 3 thread 1 sequence 30705
    ARC0: Beginning to archive log 3 thread 1 sequence 30705
    Creating archive destination LOG_ARCHIVE_DEST_1: 'F:\ORACLE\PRD\SAPARCH\ARC30705.001'
    Fri Nov 28 05:42:27 2008
    ARC1: Evaluating archive   log 3 thread 1 sequence 30705
    ARC1: Unable to archive log 3 thread 1 sequence 30705
          Log actively being archived by another process
    Fri Nov 28 05:42:29 2008
    ARC0: Completed archiving  log 3 thread 1 sequence 30705
    Fri Nov 28 06:11:26 2008
    Completed checkpoint up to RBA [0x77f2.2.10], SCN: 0x0000.3c0ae436
    Fri Nov 28 09:20:59 2008
    Beginning log switch checkpoint up to RBA [0x77f3.2.10], SCN: 0x0000.3c0b739c
    Thread 1 advanced to log sequence 30707
      Current log# 1 seq# 30707 mem# 0: C:\ORACLE\PRD\ORIGLOGA\LOGPRD01_M1.ORA
      Current log# 1 seq# 30707 mem# 1: F:\ORACLE\PRD\MIRRLOGA\LOGPRD01_M2.ORA
    Fri Nov 28 09:20:59 2008
    ARC0: Evaluating archive   log 2 thread 1 sequence 30706
    ARC0: Beginning to archive log 2 thread 1 sequence 30706
    Creating archive destination LOG_ARCHIVE_DEST_1: 'F:\ORACLE\PRD\SAPARCH\ARC30706.001'
    Fri Nov 28 09:21:18 2008
    ARC0: Completed archiving  log 2 thread 1 sequence 30706
    Fri Nov 28 09:51:06 2008
    Completed checkpoint up to RBA [0x77f3.2.10], SCN: 0x0000.3c0b739c
    Fri Nov 28 10:44:43 2008
    Beginning log switch checkpoint up to RBA [0x77f4.2.10], SCN: 0x0000.3c0bb876
    Thread 1 advanced to log sequence 30708
      Current log# 4 seq# 30708 mem# 0: C:\ORACLE\PRD\ORIGLOGB\LOGPRD04_M1.ORA
      Current log# 4 seq# 30708 mem# 1: F:\ORACLE\PRD\MIRRLOGB\LOGPRD04_M2.ORA
    Fri Nov 28 10:44:43 2008
    ARC0: Evaluating archive   log 1 thread 1 sequence 30707
    ARC0: Beginning to archive log 1 thread 1 sequence 30707
    Creating archive destination LOG_ARCHIVE_DEST_1: 'F:\ORACLE\PRD\SAPARCH\ARC30707.001'
    Fri Nov 28 10:45:07 2008
    ARC1: Evaluating archive   log 1 thread 1 sequence 30707
    ARC1: Unable to archive log 1 thread 1 sequence 30707
          Log actively being archived by another process
    Fri Nov 28 10:45:09 2008
    ARC0: Completed archiving  log 1 thread 1 sequence 30707
    Fri Nov 28 11:13:20 2008
    Completed checkpoint up to RBA [0x77f4.2.10], SCN: 0x0000.3c0bb876
    Fri Nov 28 11:45:09 2008
    Beginning log switch checkpoint up to RBA [0x77f5.2.10], SCN: 0x0000.3c0bfc29
    Thread 1 advanced to log sequence 30709
      Current log# 3 seq# 30709 mem# 0: C:\ORACLE\PRD\ORIGLOGA\LOGPRD03_M1.ORA
      Current log# 3 seq# 30709 mem# 1: F:\ORACLE\PRD\MIRRLOGA\LOGPRD03_M2.ORA
    Fri Nov 28 11:45:09 2008
    ARC0: Evaluating archive   log 4 thread 1 sequence 30708
    ARC0: Beginning to archive log 4 thread 1 sequence 30708
    Creating archive destination LOG_ARCHIVE_DEST_1: 'F:\ORACLE\PRD\SAPARCH\ARC30708.001'
    Fri Nov 28 11:45:13 2008
    ARC1: Evaluating archive   log 4 thread 1 sequence 30708
    ARC1: Unable to archive log 4 thread 1 sequence 30708
          Log actively being archived by another process
    Fri Nov 28 11:45:29 2008
    ARC0: Completed archiving  log 4 thread 1 sequence 30708
    Fri Nov 28 12:15:17 2008
    Completed checkpoint up to RBA [0x77f5.2.10], SCN: 0x0000.3c0bfc29
    Fri Nov 28 12:50:23 2008
    Beginning log switch checkpoint up to RBA [0x77f6.2.10], SCN: 0x0000.3c0c3298
    Thread 1 advanced to log sequence 30710
      Current log# 2 seq# 30710 mem# 0: C:\ORACLE\PRD\ORIGLOGB\LOGPRD02_M1.ORA
      Current log# 2 seq# 30710 mem# 1: F:\ORACLE\PRD\MIRRLOGB\LOGPRD02_M2.ORA
    Fri Nov 28 12:50:24 2008
    ARC0: Evaluating archive   log 3 thread 1 sequence 30709
    ARC0: Beginning to archive log 3 thread 1 sequence 30709
    Creating archive destination LOG_ARCHIVE_DEST_1: 'F:\ORACLE\PRD\SAPARCH\ARC30709.001'
    Fri Nov 28 12:50:40 2008
    ARC0: Completed archiving  log 3 thread 1 sequence 30709
    Fri Nov 28 13:20:29 2008
    Completed checkpoint up to RBA [0x77f6.2.10], SCN: 0x0000.3c0c3298
    Fri Nov 28 14:27:01 2008
    Beginning log switch checkpoint up to RBA [0x77f7.2.10], SCN: 0x0000.3c0c75c1
    Thread 1 advanced to log sequence 30711
      Current log# 1 seq# 30711 mem# 0: C:\ORACLE\PRD\ORIGLOGA\LOGPRD01_M1.ORA
      Current log# 1 seq# 30711 mem# 1: F:\ORACLE\PRD\MIRRLOGA\LOGPRD01_M2.ORA
    Fri Nov 28 14:27:01 2008
    ARC0: Evaluating archive   log 2 thread 1 sequence 30710
    ARC0: Beginning to archive log 2 thread 1 sequence 30710
    Creating archive destination LOG_ARCHIVE_DEST_1: 'F:\ORACLE\PRD\SAPARCH\ARC30710.001'
    Fri Nov 28 14:27:25 2008
    ARC0: Completed archiving  log 2 thread 1 sequence 30710
    Fri Nov 28 14:57:08 2008
    Completed checkpoint up to RBA [0x77f7.2.10], SCN: 0x0000.3c0c75c1
    Fri Nov 28 15:57:26 2008
    Beginning log switch checkpoint up to RBA [0x77f8.2.10], SCN: 0x0000.3c0cb34a
    Thread 1 advanced to log sequence 30712
      Current log# 4 seq# 30712 mem# 0: C:\ORACLE\PRD\ORIGLOGB\LOGPRD04_M1.ORA
      Current log# 4 seq# 30712 mem# 1: F:\ORACLE\PRD\MIRRLOGB\LOGPRD04_M2.ORA
    Fri Nov 28 15:57:26 2008
    ARC0: Evaluating archive   log 1 thread 1 sequence 30711
    ARC0: Beginning to archive log 1 thread 1 sequence 30711
    Creating archive destination LOG_ARCHIVE_DEST_1: 'F:\ORACLE\PRD\SAPARCH\ARC30711.001'
    Fri Nov 28 15:57:40 2008
    ARC1: Evaluating archive   log 1 thread 1 sequence 30711
    ARC1: Unable to archive log 1 thread 1 sequence 30711
          Log actively being archived by another process
    Fri Nov 28 15:57:40 2008
    ARC0: Completed archiving  log 1 thread 1 sequence 30711
    Fri Nov 28 16:27:31 2008
    Completed checkpoint up to RBA [0x77f8.2.10], SCN: 0x0000.3c0cb34a
    Fri Nov 28 17:02:12 2008
    Beginning log switch checkpoint up to RBA [0x77f9.2.10], SCN: 0x0000.3c0cc245
    Thread 1 advanced to log sequence 30713
      Current log# 3 seq# 30713 mem# 0: C:\ORACLE\PRD\ORIGLOGA\LOGPRD03_M1.ORA
      Current log# 3 seq# 30713 mem# 1: F:\ORACLE\PRD\MIRRLOGA\LOGPRD03_M2.ORA
    Fri Nov 28 17:02:12 2008
    ARC0: Evaluating archive   log 4 thread 1 sequence 30712
    ARC0: Beginning to archive log 4 thread 1 sequence 30712
    Creating archive destination LOG_ARCHIVE_DEST_1: 'F:\ORACLE\PRD\SAPARCH\ARC30712.001'
    Fri Nov 28 17:02:25 2008
    ARC0: Completed archiving  log 4 thread 1 sequence 30712
    Fri Nov 28 17:32:17 2008
    Completed checkpoint up to RBA [0x77f9.2.10], SCN: 0x0000.3c0cc245
    Fri Nov 28 18:00:19 2008
    /* BRCONNECT */ ALTER TABLESPACE "PSAPBTABD" COALESCE
    Fri Nov 28 18:00:19 2008
    Completed: /* BRCONNECT */ ALTER TABLESPACE "PSAPBTABD" COALE
    Fri Nov 28 18:00:19 2008
    /* BRCONNECT */ ALTER TABLESPACE "PSAPBTABI" COALESCE
    Completed: /* BRCONNECT */ ALTER TABLESPACE "PSAPBTABI" COALE
    Fri Nov 28 18:00:19 2008
    /* BRCONNECT */ ALTER TABLESPACE "PSAPCLUD" COALESCE
    Completed: /* BRCONNECT */ ALTER TABLESPACE "PSAPCLUD" COALES
    Fri Nov 28 18:00:20 2008
    /* BRCONNECT */ ALTER TABLESPACE "PSAPCLUI" COALESCE
    Completed: /* BRCONNECT */ ALTER TABLESPACE "PSAPCLUI" COALES
    Fri Nov 28 18:00:21 2008
    /* BRCONNECT */ ALTER TABLESPACE "PSAPDDICD" COALESCE
    Completed: /* BRCONNECT */ ALTER TABLESPACE "PSAPDDICD" COALE
    Fri Nov 28 18:00:23 2008
    /* BRCONNECT */ ALTER TABLESPACE "PSAPDDICI" COALESCE
    Completed: /* BRCONNECT */ ALTER TABLESPACE "PSAPDDICI" COALE
    Fri Nov 28 18:00:23 2008
    /* BRCONNECT */ ALTER TABLESPACE "PSAPDOCUD" COALESCE
    Completed: /* BRCONNECT */ ALTER TABLESPACE "PSAPDOCUD" COALE
    Fri Nov 28 18:00:23 2008
    /* BRCONNECT */ ALTER TABLESPACE "PSAPDOCUI" COALESCE
    Completed: /* BRCONNECT */ ALTER TABLESPACE "PSAPDOCUI" COALE
    Fri Nov 28 18:00:24 2008
    /* BRCONNECT */ ALTER TABLESPACE "PSAPEL640D" COALESCE
    Completed: /* BRCONNECT */ ALTER TABLESPACE "PSAPEL640D" COAL
    Fri Nov 28 18:00:26 2008
    /* BRCONNECT */ ALTER TABLESPACE "PSAPEL640I" COALESCE
    Completed: /* BRCONNECT */ ALTER TABLESPACE "PSAPEL640I" COAL
    Fri Nov 28 18:00:27 2008
    /* BRCONNECT */ ALTER TABLESPACE "PSAPES640D" COALESCE
    Completed: /* BRCONNECT */ ALTER TABLESPACE "PSAPES640D" COAL
    Fri Nov 28 18:00:27 2008
    /* BRCONNECT */ ALTER TABLESPACE "PSAPES640I" COALESCE
    Completed: /* BRCONNECT */ ALTER TABLESPACE "PSAPES640I" COAL
    Fri Nov 28 18:00:27 2008
    /* BRCONNECT */ ALTER TABLESPACE "PSAPLOADD" COALESCE
    Completed: /* BRCONNECT */ ALTER TABLESPACE "PSAPLOADD" COALE
    Fri Nov 28 18:00:27 2008
    /* BRCONNECT */ ALTER TABLESPACE "PSAPLOADI" COALESCE
    Completed: /* BRCONNECT */ ALTER TABLESPACE "PSAPLOADI" COALE
    Fri Nov 28 18:00:27 2008
    /* BRCONNECT */ ALTER TABLESPACE "PSAPPOOLD" COALESCE
    Completed: /* BRCONNECT */ ALTER TABLESPACE "PSAPPOOLD" COALE
    Fri Nov 28 18:00:28 2008
    /* BRCONNECT */ ALTER TABLESPACE "PSAPPOOLI" COALESCE
    Completed: /* BRCONNECT */ ALTER TABLESPACE "PSAPPOOLI" COALE
    Fri Nov 28 18:00:28 2008
    /* BRCONNECT */ ALTER TABLESPACE "PSAPPROTD" COALESCE
    Completed: /* BRCONNECT */ ALTER TABLESPACE "PSAPPROTD" COALE
    Fri Nov 28 18:00:28 2008
    /* BRCONNECT */ ALTER TABLESPACE "PSAPPROTI" COALESCE
    Completed: /* BRCONNECT */ ALTER TABLESPACE "PSAPPROTI" COALE
    Fri Nov 28 18:00:28 2008
    /* BRCONNECT */ ALTER TABLESPACE "PSAPSOURCED" COALESCE
    Completed: /* BRCONNECT */ ALTER TABLESPACE "PSAPSOURCED" COA
    Fri Nov 28 18:00:28 2008
    /* BRCONNECT */ ALTER TABLESPACE "PSAPSOURCEI" COALESCE
    Completed: /* BRCONNECT */ ALTER TABLESPACE "PSAPSOURCEI" COA
    Fri Nov 28 18:00:28 2008
    /* BRCONNECT */ ALTER TABLESPACE "PSAPSTABD" COALESCE
    Completed: /* BRCONNECT */ ALTER TABLESPACE "PSAPSTABD" COALE
    Fri Nov 28 18:00:28 2008
    /* BRCONNECT */ ALTER TABLESPACE "PSAPSTABI" COALESCE
    Completed: /* BRCONNECT */ ALTER TABLESPACE "PSAPSTABI" COALE
    Fri Nov 28 18:00:28 2008
    /* BRCONNECT */ ALTER TABLESPACE "PSAPUSER1D" COALESCE
    Completed: /* BRCONNECT */ ALTER TABLESPACE "PSAPUSER1D" COAL
    Fri Nov 28 18:00:28 2008
    /* BRCONNECT */ ALTER TABLESPACE "PSAPUSER1I" COALESCE
    Completed: /* BRCONNECT */ ALTER TABLESPACE "PSAPUSER1I" COAL
    Fri Nov 28 18:00:28 2008
    /* BRCONNECT */ ALTER TABLESPACE "SYSTEM" COALESCE
    Completed: /* BRCONNECT */ ALTER TABLESPACE "SYSTEM" COALESCE
    Fri Nov 28 20:00:17 2008
    /* BRCONNECT */ ALTER TABLESPACE "PSAPBTABD" COALESCE
    Completed: /* BRCONNECT */ ALTER TABLESPACE "PSAPBTABD" COALE
    Fri Nov 28 20:00:18 2008
    /* BRCONNECT */ ALTER TABLESPACE "PSAPBTABI" COALESCE
    Completed: /* BRCONNECT */ ALTER TABLESPACE "PSAPBTABI" COALE
    Fri Nov 28 20:00:18 2008
    /* BRCONNECT */ ALTER TABLESPACE "PSAPCLUD" COALESCE
    Completed: /* BRCONNECT */ ALTER TABLESPACE "PSAPCLUD" COALES
    Fri Nov 28 20:00:18 2008
    /* BRCONNECT */ ALTER TABLESPACE "PSAPCLUI" COALESCE
    Completed: /* BRCONNECT */ ALTER TABLESPACE "PSAPCLUI" COALES
    Fri Nov 28 20:00:18 2008
    /* BRCONNECT */ ALTER TABLESPACE "PSAPDDICD" COALESCE
    Completed: /* BRCONNECT */ ALTER TABLESPACE "PSAPDDICD" COALE
    Fri Nov 28 20:00:18 2008
    /* BRCONNECT */ ALTER TABLESPACE "PSAPDDICI" COALESCE
    Completed: /* BRCONNECT */ ALTER TABLESPACE "PSAPDDICI" COALE
    Fri Nov 28 20:00:18 2008
    /* BRCONNECT */ ALTER TABLESPACE "PSAPDOCUD" COALESCE
    Completed: /* BRCONNECT */ ALTER TABLESPACE "PSAPDOCUD" COALE
    Fri Nov 28 20:00:18 2008
    /* BRCONNECT */ ALTER TABLESPACE "PSAPDOCUI" COALESCE
    Completed: /* BRCONNECT */ ALTER TABLESPACE "PSAPDOCUI" COALE
    Fri Nov 28 20:00:18 2008
    /* BRCONNECT */ ALTER TABLESPACE "PSAPEL640D" COALESCE
    Completed: /* BRCONNECT */ ALTER TABLESPACE "PSAPEL640D" COAL
    Fri Nov 28 20:00:18 2008
    /* BRCONNECT */ ALTER TABLESPACE "PSAPEL640I" COALESCE
    Completed: /* BRCONNECT */ ALTER TABLESPACE "PSAPEL640I" COAL
    Fri Nov 28 20:00:18 2008
    /* BRCONNECT */ ALTER TABLESPACE "PSAPES640D" COALESCE
    Completed: /* BRCONNECT */ ALTER TABLESPACE "PSAPES640D" COAL
    Fri Nov 28 20:00:18 2008
    /* BRCONNECT */ ALTER TABLESPACE "PSAPES640I" COALESCE
    Completed: /* BRCONNECT */ ALTER TABLESPACE "PSAPES640I" COAL
    Fri Nov 28 20:00:18 2008
    /* BRCONNECT */ ALTER TABLESPACE "PSAPLOADD" COALESCE
    Completed: /* BRCONNECT */ ALTER TABLESPACE "PSAPLOADD" COALE
    Fri Nov 28 20:00:18 2008
    /* BRCONNECT */ ALTER TABLESPACE "PSAPLOADI" COALESCE
    Completed: /* BRCONNECT */ ALTER TABLESPACE "PSAPLOADI" COALE
    Fri Nov 28 20:00:18 2008
    /* BRCONNECT */ ALTER TABLESPACE "PSAPPOOLD" COALESCE
    Completed: /* BRCONNECT */ ALTER TABLESPACE "PSAPPOOLD" COALE
    Fri Nov 28 20:00:18 2008
    /* BRCONNECT */ ALTER TABLESPACE "PSAPPOOLI" COALESCE
    Completed: /* BRCONNECT */ ALTER TABLESPACE "PSAPPOOLI" COALE
    Fri Nov 28 20:00:18 2008
    /* BRCONNECT */ ALTER TABLESPACE "PSAPPROTD" COALESCE
    Completed: /* BRCONNECT */ ALTER TABLESPACE "PSAPPROTD" COALE
    Fri Nov 28 20:00:18 2008
    /* BRCONNECT */ ALTER TABLESPACE "PSAPPROTI" COALESCE
    Completed: /* BRCONNECT */ ALTER TABLESPACE "PSAPPROTI" COALE
    Fri Nov 28 20:00:18 2008
    /* BRCONNECT */ ALTER TABLESPACE "PSAPSOURCED" COALESCE
    Completed: /* BRCONNECT */ ALTER TABLESPACE "PSAPSOURCED" COA
    Fri Nov 28 20:00:18 2008
    /* BRCONNECT */ ALTER TABLESPACE "PSAPSOURCEI" COALESCE
    Completed: /* BRCONNECT */ ALTER TABLESPACE "PSAPSOURCEI" COA
    Fri Nov 28 20:00:18 2008
    /* BRCONNECT */ ALTER TABLESPACE "PSAPSTABD" COALESCE
    Completed: /* BRCONNECT */ ALTER TABLESPACE "PSAPSTABD" COALE
    Fri Nov 28 20:00:18 2008
    /* BRCONNECT */ ALTER TABLESPACE "PSAPSTABI" COALESCE
    Completed: /* BRCONNECT */ ALTER TABLESPACE "PSAPSTABI" COALE
    Fri Nov 28 20:00:18 2008
    /* BRCONNECT */ ALTER TABLESPACE "PSAPUSER1D" COALESCE
    Completed: /* BRCONNECT */ ALTER TABLESPACE "PSAPUSER1D" COAL
    Fri Nov 28 20:00:18 2008
    /* BRCONNECT */ ALTER TABLESPACE "PSAPUSER1I" COALESCE
    Completed: /* BRCONNECT */ ALTER TABLESPACE "PSAPUSER1I" COAL
    Fri Nov 28 20:00:18 2008
    /* BRCONNECT */ ALTER TABLESPACE "SYSTEM" COALESCE
    Completed: /* BRCONNECT */ ALTER TABLESPACE "SYSTEM" COALESCE
    Fri Nov 28 20:00:36 2008
    Beginning log switch checkpoint up to RBA [0x77fa.2.10], SCN: 0x0000.3c0cf8e3
    Thread 1 advanced to log sequence 30714
      Current log# 2 seq# 30714 mem# 0: C:\ORACLE\PRD\ORIGLOGB\LOGPRD02_M1.ORA
      Current log# 2 seq# 30714 mem# 1: F:\ORACLE\PRD\MIRRLOGB\LOGPRD02_M2.ORA
    Fri Nov 28 20:00:36 2008
    ARC0: Evaluating archive   log 3 thread 1 sequence 30713
    ARC0: Beginning to archive log 3 thread 1 sequence 30713
    Creating archive destination LOG_ARCHIVE_DEST_1: 'F:\ORACLE\PRD\SAPARCH\ARC30713.001'
    Fri Nov 28 20:00:44 2008
    ARC1: Evaluating archive   log 3 thread 1 sequence 30713
    ARC1: Unable to archive log 3 thread 1 sequence 30713
          Log actively being archived by another process
    Fri Nov 28 20:00:47 2008
    ARC0: Completed archiving  log 3 thread 1 sequence 30713
    Dump file f:\oracle\prd\saptrace\background\alert_prd.log
    Mon Dec 01 09:03:16 2008
    ORACLE V9.2.0.8.0 - Production vsnsta=0
    vsnsql=12 vsnxtr=3
    Windows 2000 Version 5.2 Service Pack 2, CPU type 586
    Mon Dec 01 09:03:16 2008
    Starting ORACLE instance (normal)
    LICENSE_MAX_SESSION = 0
    LICENSE_SESSIONS_WARNING = 0
    SCN scheme 2
    LICENSE_MAX_USERS = 0
    SYS auditing is disabled
    Starting up ORACLE RDBMS Version: 9.2.0.8.0.
    System parameters with non-default values:
      processes                = 120
      sessions                 = 150
      timed_statistics         = TRUE
      shared_pool_size         = 822083584
      large_pool_size          = 8388608
      java_pool_size           = 8388608
      enqueue_resources        = 8000
      filesystemio_options     = SETALL
      control_files            = C:\oracle\PRD\origlogA\ctrlPRD.ctl, F:\oracle\PRD\saparch\ctrlPRD.ctl, E:\oracle\PRD\sapdata1\ctrlPRD.ctl
      db_block_checksum        = FALSE
      db_block_size            = 8192
      db_cache_size            = 1073741824
      compatible               = 9.2.0
      log_archive_start        = TRUE
      log_archive_dest         = F:\oracle\PRD\saparch
      log_buffer               = 4194304
      log_checkpoint_interval  = 0
      db_files                 = 254
      db_file_multiblock_read_count= 8
      log_checkpoints_to_alert = TRUE
      control_file_record_keep_time= 30
      dml_locks                = 2500
      row_locking              = always
      replication_dependency_tracking= FALSE
      transactions_per_rollback_segment= 20
      transaction_auditing     = FALSE
      undo_management          = AUTO
      undo_tablespace          = UNDOTBS1
      undo_retention           = 43200
      remote_os_authent        = TRUE
      db_domain                = world
      service_names            = PRD.world, PRD.WORLD
      hash_join_enabled        = FALSE
      background_dump_dest     = F:\Oracle\PRD\saptrace\background
      user_dump_dest           = F:\Oracle\PRD\saptrace\usertrace
      max_dump_file_size       = 1240
      core_dump_dest           = F:\Oracle\PRD\saptrace\background
      optimizer_features_enable= 9.2.0
      sort_area_size           = 0
      sort_area_retained_size  = 0
      db_name                  = PRD
      open_cursors             = 800
      optimizer_mode           = CHOOSE
      sortelimination_cost_ratio= 10
      btree_bitmap_plans     = FALSE
      eliminatecommon_subexpr= FALSE
      pushjoin_predicate     = FALSE
      optimizerjoin_sel_sanity_check= TRUE
      parallel_threads_per_cpu = 4
      optimizer_index_cost_adj = 10
      optimizer_index_caching  = 50
      optimpeek_user_binds   = FALSE
      pga_aggregate_target     = 536870912
      optimizeror_expansion  = DEPTH
    PMON started with pid=2, OS id=17336
    DBW0 started with pid=3, OS id=16972
    LGWR started with pid=4, OS id=16900
    CKPT started with pid=5, OS id=17236
    SMON started with pid=6, OS id=17436
    Regards
    Pieter

  • Performance Bottleneck at HIgh Loads

    Hi,
    We are currently load testing our web application and have been facing an issue that has ben bothering us for sometime now.
    If we load test the application at high load for sustained amounts of time, (around 2 hours), the performance suddenly dips to Zero hits/sec, Nothing is processed , no output generated. It seems as though all the processes some how are waiting for a resource to get free.After sometime the performance goes up again and again there is a race condition because of which the performance goes down again. Here on the dipping becomes highly erratic.
    We are trying to find out if this is a known bug with the oracle jdbc driver. "oracle.jdbc.driver.OracleDriver" .
    Would appreciate if anybody could provide any inputs as to why this is happening and how we could alleviate the situation.
    Thanks in advance.
    Deepanjan

    Hi I vaguely remember something like this.... but I thought it typically occurred during lots of heavy insert/update activity
    which version of the jdbc drivers are you using.... are you using thin/thick
    Have you checked the SQL or the database to determine whether any locks are blocking activity or whether you have large database wait issues....
    Dom

  • Weblogic Server hanging at high load!!!!!!!!!!!

    Folks,
    We have a weblogic server 9.2 which is running fine but after quiet a while at high load we observe that the server goes down and following exception is displayed in the server logs:
    ####<Nov 13, 2009 2:42:48 AM EST> <Error> <WebLogicServer> <ibmrxcptpnp-wb1> <AdminServer> <[ACTIVE] ExecuteThread: '2' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1258098168215> <BEA-000337> <[STUCK] ExecuteThread: '9' for queue: 'weblogic.kernel.Default (self-tuning)' has been busy for "617" seconds working on the request "weblogic.work.ServerWorkManagerImpl$WorkAdapterImpl@1cec1cec", which is more than the configured time (StuckThreadMaxTime) of "600" seconds. Stack trace:
         java.net.SocketOutputStream.socketWrite0(Native Method)
         java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:105)
         java.net.SocketOutputStream.write(SocketOutputStream.java:149)
         oracle.net.ns.DataPacket.send(Unknown Source)
         oracle.net.ns.NetOutputStream.flush(Unknown Source)
         oracle.net.ns.NetInputStream.getNextPacket(Unknown Source)
         oracle.net.ns.NetInputStream.read(Unknown Source)
         oracle.net.ns.NetInputStream.read(Unknown Source)
         oracle.net.ns.NetInputStream.read(Unknown Source)
         oracle.jdbc.driver.T4CMAREngine.unmarshalUB1(T4CMAREngine.java:1099)
         oracle.jdbc.driver.T4CMAREngine.unmarshalSB1(T4CMAREngine.java:1070)
         oracle.jdbc.driver.T4C8Oall.receive(T4C8Oall.java:478)
         oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:216)
         oracle.jdbc.driver.T4CPreparedStatement.executeForRows(T4CPreparedStatement.java:955)
         oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:1060)
         oracle.jdbc.driver.T4CPreparedStatement.executeMaybeDescribe(T4CPreparedStatement.java:839)
         oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1132)
         oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3316)
         oracle.jdbc.driver.OraclePreparedStatement.execute(OraclePreparedStatement.java:3422)
         weblogic.jdbc.common.internal.ConnectionEnv.test(ConnectionEnv.java:718)
         weblogic.jdbc.common.internal.ConnectionEnv.test(ConnectionEnv.java:460)
         weblogic.common.resourcepool.ResourcePoolImpl.checkResource(ResourcePoolImpl.java:1455)
         weblogic.common.resourcepool.ResourcePoolImpl.checkAndReturnResource(ResourcePoolImpl.java:1372)
         weblogic.common.resourcepool.ResourcePoolImpl.checkAndReturnResource(ResourcePoolImpl.java:1362)
         weblogic.common.resourcepool.ResourcePoolImpl.testUnusedResources(ResourcePoolImpl.java:1767)
         weblogic.common.resourcepool.ResourcePoolImpl.access$700(ResourcePoolImpl.java:37)
         weblogic.common.resourcepool.ResourcePoolImpl$ResourcePoolMaintanenceTask.timerExpired(ResourcePoolImpl.java:1935)
         weblogic.timers.internal.TimerImpl.run(TimerImpl.java:265)
         weblogic.work.ServerWorkManagerImpl$WorkAdapterImpl.run(ServerWorkManagerImpl.java:518)
         weblogic.work.ExecuteThread.execute(ExecuteThread.java:209)
         weblogic.work.ExecuteThread.run(ExecuteThread.java:181)
    >
    Any answers?????

    Hi,
    I have a similar issue on ODSI 10gR3 (WLS 10.3). I am having stuck threads - but, in my case it is clear that the database is down (not at the startup)
    Here is some more info:
    1. Test Connections On Reserve - Enabled
    2. Default values for other parameters
    Using Oracle thin driver for the connection pool "MyPool" -
    ./wlserver_10.3/server/lib/ojdbc6.jar
    My question is:
    1. How do I avoid the stuck thread caused when the database is down? During peak load, the stuck threads are causing some service requests to fail.
    I did my homework and the closest option I could find is set the statementTimeout (which is set currently to the default value of -1 i.e. never timeout). Oracle documentation (http://www.oracle.com/technology/software/tech/java/sqlj_jdbc/htdocs/111070_readme.html) says that the ojdbc6.jar supports setQueryTimeout() method.
    Is there any better way to attack this problem?
    <Dec 21, 2009 12:11:18 AM EST> <Error> <JDBC> <BEA-001112> <Test "SELECT 1 FROM DUAL" set up for pool "MyPool" failed with exception: "java.sql.SQLRecoverableException: Io exception: Read failed: Connection timed out".>
    <Dec 21, 2009 12:14:27 AM EST> <Warning> <JDBC> <BEA-001129> <Received exception while creating connection for pool "MyPool": Io exception: The Network Adapter could not establish the connection>
    <Dec 21, 2009 12:18:38 AM EST> <Error> <WebLogicServer> <BEA-000337> <[STUCK] ExecuteThread: '8' for queue: 'weblogic.kernel.Default (self-tuning)' has been busy for "600" seconds working on the request "weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl@297ac2b", which is more than the configured time (StuckThreadMaxTime) of "600" seconds. Stack trace:
    Thread-52 "[STUCK] ExecuteThread: '8' for queue: 'weblogic.kernel.Default (self-tuning)'" <alive, in native, suspended, waiting, priority=1, DAEMON> {
    -- Waiting for notification on: javax.management.remote.rmi.RMIConnector$RMINotifClient@271498f[fat lock]
    java.lang.Object.wait(Object.java:485)
    com.sun.jmx.remote.internal.ClientNotifForwarder.postReconnection(ClientNotifForwarder.java:304)
    javax.management.remote.rmi.RMIConnector$RMIClientCommunicatorAdmin.reconnectNotificationListeners(RMIConnector.java:1488)
    javax.management.remote.rmi.RMIConnector$RMIClientCommunicatorAdmin.doStart(RMIConnector.java:1568)
    com.sun.jmx.remote.internal.ClientCommunicatorAdmin.restart(ClientCommunicatorAdmin.java:72)
    com.sun.jmx.remote.internal.ClientCommunicatorAdmin.gotIOException(ClientCommunicatorAdmin.java:34)
    javax.management.remote.rmi.RMIConnector$RMIClientCommunicatorAdmin.gotIOException(RMIConnector.java:1420)
    javax.management.remote.rmi.RMIConnector$RemoteMBeanServerConnection.getAttribute(RMIConnector.java:857)
    weblogic.management.mbeanservers.domainruntime.internal.ManagedMBeanServerConnection.getAttribute(ManagedMBeanServerConnection.java:288)
    javax.management.MBeanServerInvocationHandler.invoke(MBeanServerInvocationHandler.java:235)
    weblogic.management.jmx.MBeanServerInvocationHandler.doInvoke(MBeanServerInvocationHandler.java:477)
    weblogic.management.jmx.MBeanServerInvocationHandler.invoke(MBeanServerInvocationHandler.java:294)
    $Proxy69.getServerRuntime(Unknown Source)
    weblogic.management.mbeanservers.domainruntime.internal.DomainRuntimeServiceMBeanImpl.lookupServerRuntime(DomainRuntimeServiceMBeanImpl.java:242)
    sun.reflect.GeneratedMethodAccessor1880.invoke(Unknown Source)
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    java.lang.reflect.Method.invoke(Method.java:575)
    weblogic.management.jmx.modelmbean.WLSModelMBean.invoke(WLSModelMBean.java:355)
    com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:831)
    com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:761)
    weblogic.management.mbeanservers.domainruntime.internal.FederatedMBeanServerInterceptor.invoke(FederatedMBeanServerInterceptor.java:255)
    weblogic.management.jmx.mbeanserver.WLSMBeanServerInterceptorBase$16.run(WLSMBeanServerInterceptorBase.java:447)
    weblogic.management.jmx.mbeanserver.WLSMBeanServerInterceptorBase.invoke(WLSMBeanServerInterceptorBase.java:441)
    weblogic.management.mbeanservers.internal.SecurityMBeanMgmtOpsInterceptor.invoke(SecurityMBeanMgmtOpsInterceptor.java:55)
    weblogic.management.jmx.mbeanserver.WLSMBeanServerInterceptorBase$16.run(WLSMBeanServerInterceptorBase.java:447)
    weblogic.management.jmx.mbeanserver.WLSMBeanServerInterceptorBase.invoke(WLSMBeanServerInterceptorBase.java:441)
    weblogic.management.mbeanservers.internal.SecurityInterceptor.invoke(SecurityInterceptor.java:437)
    weblogic.management.mbeanservers.internal.AuthenticatedSubjectInterceptor$10$1.run(AuthenticatedSubjectInterceptor.java:582)
    weblogic.management.mbeanservers.internal.AuthenticatedSubjectInterceptor$10.run(AuthenticatedSubjectInterceptor.java:576)
    weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:350)
    weblogic.management.mbeanservers.internal.AuthenticatedSubjectInterceptor.invoke(AuthenticatedSubjectInterceptor.java:570)
    weblogic.management.jmx.mbeanserver.WLSMBeanServer.invoke(WLSMBeanServer.java:305)
    javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1378)
    javax.management.remote.rmi.RMIConnectionImpl.access$200(RMIConnectionImpl.java:72)
    javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1264)
    javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1338)
    javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:761)
    javax.management.remote.rmi.RMIConnectionImpl_WLSkel.invoke(Unknown Source)
    weblogic.rmi.internal.ServerRequest.sendReceive(ServerRequest.java:136)
    weblogic.rmi.internal.BasicRemoteRef.invoke(BasicRemoteRef.java:211)
    javax.management.remote.rmi.RMIConnectionImpl_1030_WLStub.invoke(Unknown Source)
    javax.management.remote.rmi.RMIConnector$RemoteMBeanServerConnection.invoke(RMIConnector.java:969)
    com.bea.diagnostics.server.MBeanServerUtil.processRecordsFromArchive(MBeanServerUtil.java:177)
    com.bea.diagnostics.server.MetricEarliestTimestampLocator$RecordProcessorDesc.identifyEarliestTimestamps(MetricEarliestTimestampLocator.java:261)
    com.bea.diagnostics.server.MetricEarliestTimestampLocator.identifyEarliestTimestamps(MetricEarliestTimestampLocator.java:84)
    com.bea.diagnostics.server.MetricEarliestTimestampLocator.timerExpired(MetricEarliestTimestampLocator.java:79)
    weblogic.timers.internal.TimerImpl.run(TimerImpl.java:253)
    weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:516)
    weblogic.work.ExecuteThread.execute(ExecuteThread.java:198)
    weblogic.work.ExecuteThread.run(ExecuteThread.java:165)
    <Dec 21, 2009 12:24:04 AM EST> <Error> <JDBC> <BEA-001112> <Test "SELECT 1 FROM DUAL" set up for pool "MyPool" failed with exception: "java.sql.SQLRecoverableException: Io exception: Connection reset".>
    <Dec 21, 2009 12:24:04 AM EST> <Warning> <JDBC> <BEA-001129> <Received exception while creating connection for pool "MyPool": Io exception: The Network Adapter could not establish the connection>
    <Dec 21, 2009 12:24:04 AM EST> <Warning> <JDBC> <BEA-001129> <Received exception while creating connection for pool "MyPool": Io exception: The Network Adapter could not establish the connection>
    <Dec 21, 2009 12:24:05 AM EST> <Warning> <JDBC> <BEA-001129> <Received exception while creating connection for pool "MyPool": Io exception: The Network Adapter could not establish the connection>
    <Dec 21, 2009 12:24:07 AM EST> <Warning> <JDBC> <BEA-001129> <Received exception while creating connection for pool "MyPool": Io exception: The Network Adapter could not establish the connection>
    <Dec 21, 2009 12:24:12 AM EST> <Warning> <JDBC> <BEA-001129> <Received exception while creating connection for pool "MyPool": Io exception: The Network Adapter could not establish the connection>
    <Dec 21, 2009 12:24:17 AM EST> <Warning> <JDBC> <BEA-001129> <Received exception while creating connection for pool "MyPool": Io exception: The Network Adapter could not establish the connection>
    <Dec 21, 2009 12:24:22 AM EST> <Warning> <JDBC> <BEA-001129> <Received exception while creating connection for pool "MyPool": Io exception: The Network Adapter could not establish the connection>

  • Oracle ODBC  - Internal Error - unable to initialize NLS during driver load

    I'm having some trouble with my ODBC connections which I hope someone can please help me with!
    About 6 weeks ago all was working as normal.
    As far as I know there have been no updates to the ORacle DB, the Windows XP operating system or the ODBC Drivers.
    Today when I opened access and visual case 2 to connect to Oracle I was at first greeted with a:
    unable to connect SQLState=IM004 SQL_HANDLE_ENV
    error. ODBC also kept crashing.
    I restarted the computer and was confronted with a different error:
    odbc SQLSTate 08004 ORA 12154 TNS could not resolve the connect identifier specified
    I was able to fix this error by setting the environment variable TNS_ADMIN in windows xp environment variables. I'm extremely confused about how this happened though as it was working and I don't think anything has changed.
    I was then able to connect to the database via Microsoft Access but when I opened Visual Case 2 and tried to make an update, I was confronted with the following error:
    Oracle ODBC Driver - internal error - unable to initialize NLS during driver load
    I looked in the registry at:
    HKEY_LOCAL_MACHINE\SOFTWARE\Oracle\KEY_OraClient10g_home1
    HKEY_LOCAL_MACHINE\SOFTWARE\Oracle\KEY_OraClient10g_home2
    HKEY_LOCAL_MACHINE\SOFTWARE\Oracle\KEY_OraDb10g_home1
    and NLS_LANG was set to "AMERICAN_AMERICA.WE8MSWIN1252" in all 3 places.
    (Though KEY_OraClient10g_home2 only had 4 entries as opposed to KEY_OraClient10g_home1's 13 entries)...
    Since I made those changes I can no longer connect through Access.
    I just receive a ODBC - connection to 'xxx' failed
    Advice greatly appreciated!!!!
    Edited by: user11150264 on Aug 25, 2009 10:21 PM

    Actually it sort of does...
    I switched the ODBC connection to use instant client and now it's all working again.
    The biggest mystery is what changed to make it suddenly stop working the old way...

  • Error during database load in ECC 5.0

    Hello All,
    i am trying to install ECC.5 with oracle 10g during database load phase he is giving me a error when i saw in log file i found this.
    C:\usr\sap\EPT\SYS\exe\run/R3load.exe: START OF LOG: 20090611085559
    C:\usr\sap\EPT\SYS\exe\run/R3load.exe: sccsid @(#) $Id: //bas/640_REL/src/R3ld/R3load/R3ldmain.c#6 $ SAP
    C:\usr\sap\EPT\SYS\exe\run/R3load.exe: version R6.40/V1.4
    C:\usr\sap\EPT\SYS\exe\run/R3load.exe -ctf I E:/sap5/51031332_1/EXP1/DATA/SAP0000.STR C:\Program Files\sapinst_instdir\ECC50\SYSTEM\ABAP\ORA\NUC\DB/DDLORA.TPL C:\Program Files\sapinst_instdir\ECC50\SYSTEM\ABAP\ORA\NUC\DB/SAP0000.TSK ORA -l C:\Program Files\sapinst_instdir\ECC50\SYSTEM\ABAP\ORA\NUC\DB/SAP0000.log -o D
    C:\usr\sap\EPT\SYS\exe\run/R3load.exe: job completed
    C:\usr\sap\EPT\SYS\exe\run/R3load.exe: END OF LOG: 20090611085559
    C:\usr\sap\EPT\SYS\exe\run/R3load.exe: START OF LOG: 20090611103425
    C:\usr\sap\EPT\SYS\exe\run/R3load.exe: sccsid @(#) $Id: //bas/640_REL/src/R3ld/R3load/R3ldmain.c#6 $ SAP
    C:\usr\sap\EPT\SYS\exe\run/R3load.exe: version R6.40/V1.4
    C:\usr\sap\EPT\SYS\exe\run/R3load.exe -dbcodepage 1100 -i C:\Program Files\sapinst_instdir\ECC50\SYSTEM\ABAP\ORA\NUC\DB/SAP0000.cmd -l C:\Program Files\sapinst_instdir\ECC50\SYSTEM\ABAP\ORA\NUC\DB/SAP0000.log -stop_on_error
    DbSl Trace: OCI-call 'OCISessionBegin' failed: rc = 1034
    DbSl Trace: CONNECT failed with sql error '1034'
    DbSl Trace: OCI-call 'OCISessionBegin' failed: rc = 1034
    DbSl Trace: CONNECT failed with sql error '1034'
    (DB) ERROR: db_connect rc = 256
    DbSl Trace: OCI-call 'OCISessionBegin' failed: rc = 1034
    DbSl Trace: CONNECT failed with sql error '1034'
    DbSl Trace: OCI-call 'OCISessionBegin' failed: rc = 1034
    DbSl Trace: CONNECT failed with sql error '1034'
    (DB) ERROR: DbSlErrorMsg rc = 99
    C:\usr\sap\EPT\SYS\exe\run/R3load.exe: job finished with 1 error(s)
    C:\usr\sap\EPT\SYS\exe\run/R3load.exe: END OF LOG: 20090611103428
    can anybody help me to sort this out.
    Any help highly appreciated.
    Thanks & Regards
    Subhash

    First of all thanks for helping me
    i have tried this
    Connected to an idle instance.
    SQL> STARTUP MOUNT
    ORACLE instance started.
    Total System Global Area  205520896 bytes
    Fixed Size                  1248116 bytes
    Variable Size             117441676 bytes
    Database Buffers           83886080 bytes
    Redo Buffers                2945024 bytes
    Database mounted.
    SQL> SET AUTORECOVERY ON
    SQL> RECOVER DATABASE
    ORA-00279: change 262364 generated at 06/11/2009 09:43:56 needed for thread 1
    ORA-00289: suggestion : C:\ORACLE\EPT\ORAARCH\EPTARCHARC00052_0689209324.001
    ORA-00280: change 262364 for thread 1 is in sequence #52
    ORA-00308: cannot open archived log
    'C:\ORACLE\EPT\ORAARCH\EPTARCHARC00052_0689209324.001'
    ORA-27041: unable to open file
    OSD-04002: unable to open file
    O/S-Error: (OS 2) The system cannot find the file specified.
    But i m not able to see any archive file on oraarch folder
    Regards
    Subhash

  • Strange Error During Page Load in Debug Mode (only) - Please Help!

    Hi All,
    Data base version: oracle 11g
    Apex version: Apex 4.1.1
    Webserver: Apache
    Need help with how to troubleshoot a Critical problem. The following error only occurs during page load in "Debug" mode. And, only occurs on a specific page within the application. A web page is served-up containing the following message and the application is blocked from running the page. The browser's (IE 8.0) back button must be clicked to proceed outside of "Debug" mode.
    "Error occurred while painting error page: ORA-06502: PL/SQL: numeric or value error: character string buffer too small ORA-06502: PL/SQL: numeric or value error: character string buffer too small"
    Debug log follows:
    "S H O W: application="2006" page="6" workspace="" request="" session="500549669426301"
    Computation point: Before Header
    ...Perform computation of item: APP_SERVER, type=FUNCTION_BODY
    ...Performing function body computation
    ...Execute Statement: declare function x return varchar2 is begin return owa_util.get_cgi_env('SERVER_NAME'); return null; end; begin wwv_flow.g_computation_result_vc := x; end;
    ......Result = 156.9.122.214
    ...Session State: Save "APP_SERVER" - saving same value: "156.9.122.214"
    Processes - point: BEFORE_HEADER
    ...Process "GET_POSITION" - Type: PLSQL
    ...Execute Statement: begin wwv_flow.g_boolean := :F109_POSITION_ID IS NULL and :APP_PAGE_ID != 101; end;
    ......Result = FALSE
    ......Skip because condition or authorization evaluates to FALSE
    ...Process "Get JARS Sifter Log File Record Count" - Type: PLSQL
    ...Execute Statement: begin DECLARE vcnt NUMBER := 0; BEGIN d('Get JARS Sifter Log File Record Count'); Select count(*) into vcnt From JARS.JARS_SIFTER_LOG Where moveid = to_number(:P6_MOVEID) and sifter_status IN ('F','J'); :F1000_P6_SIFTER_LOG_COUNT := to_char(vcnt); END; end;
    Custom: Get JARS Sifter Log File Record Count
    ...Process "Set PTM Planned Trip Status" - Type: PLSQL
    ......Skip because condition or authorization evaluates to FALSE
    ...compatibility mode - do not set mime type
    ...compatibility mode - do not set additional http headers
    ...close http header
    ...metadata, fetch item type settings
    ...metadata, fetch items
    Show page template header
    Rendering form open tag and internal values
    Add error onto error stack
    ...Error data:
    ......message: Error processing request.
    ......additional_info: ORA-06502: PL/SQL: numeric or value error: character string buffer too small
    ......display_location: ON_ERROR_PAGE
    ......is_internal_error: true
    ......apex_error_code: APEX.UNHANDLED_ERROR
    ......ora_sqlcode: -6502
    ......ora_sqlerrm: ORA-06502: PL/SQL: numeric or value error: character string buffer too small
    ......error_backtrace: ORA-06512: at "APEX_040100.WWV_FLOW", line 3027 ORA-06512: at "APEX_040100.WWV_FLOW", line 7867
    ...Show Error on Error Page
    ......Performing rollback
    Rendering form open tag and internal values
    ...Unhandled Error while painting error page: ORA-06502: PL/SQL: numeric or value error: character string buffer too small ORA-06502: PL/SQL: numeric or value error: character string buffer too small
    ...Error Backtrace: ORA-06512: at "APEX_040100.WWV_FLOW", line 2707 ORA-06512: at "APEX_040100.WWV_FLOW_ERROR", line 185
    End Page Rendering"
    Thanks!
    Bernard

    All,
    It appears that the page Javascript maximum limit size was reached. The error stopped appearing after some of the page Javascript code was removed out to Application Static Files. I wonder if there exists any "direct" indicator by the system whenever the size limit has been reached?
    Again, the run error only occurred when the page was loaded in "Debug" mode.
    Bernard

  • SAP R/3 4.7 EX 2 SR1 - IDES (Unicode), Problem during DB load

    Dear Experts,
    Details of the system:
    Solaris 10, Oracle 9.2.0.7.
    R/3 Enterprise 4.7 X 2 SR1 - IDES (Unicode)
    Please help me in solving this problem. I have received this error during DB load Phase 25/37. Errors are related to code page conversion failed.
    Message from SAPCLUST.log:
    myCluster (3.5.Imp): 1329: inconsistent field count detected.
    myCluster (3.5.Imp): 1332: nametab says field count (TDESCR) is 305.
    myCluster (3.5.Imp): 1335: alternate nametab says field count (TDESCR) is 311.
    myCluster (3.5.Imp): 1130: unable to retrieve nametab info for logic table BSEG      .
    myCluster (3.5.Imp): 7305: unable to retrieve nametab info for logic table BSEG      .
    myCluster (3.5.Imp): 2090: failed to convert cluster data of cluster item.
    myCluster: RFBLG      *800**0001**0100000000**1995*
    myCluster (3.5.Imp): 319: error during conversion of cluster item.
    myCluster (3.5.Imp): 322: affected physical table is RFBLG.
    (CNV) ERROR: code page conversion failed
                 rc = 2
    ^M
    .--============--
    .^M
    |                              RSCP - Error                            |^M
    | Error from:             Codepage handling (RSCP)                     |^M
    | code:   64  RSCPECALL    Illegal mixture of procedure calls.         |^M
    | Table UMGSETTING of no interest in Unicode system                    |^M
    | module: rscpexcc no:   12 line:   673                    T100: TS007 |^M
    | TSL01: CQ3  p3: UMGSETTING  p4: &                                    |^M
    |----
    |^M
    | Some OS information, which may or may not belong to this error:      |^M
    | errno    17  File exists                                             |^M
    `----
    '^M
    (DB) INFO: disconnected from DB
    Message from SAPPOOL.log:
    failed to read short nametab of table T2200                          (rc=32)conversion failed for row 5836 of table T184L      VARKEY = \343\2
    40\260\343\200\260\343\200\260\343\200\261\342\200\240\342\200\240\342\200\240\342\200\240\342\200\240\342\200\240\342\200\240\342\200\240\342
    \200\240\342\200\240\342\200\240\342\200\240\342\200\240\342\200\240\342\200\240\342\200\240\342\200\240\342\200\240\342\200\240\342\200\240\3
    42\200\240
    (CNV) ERROR: code page conversion failed
                 rc = 2
    ^M
    .--============--
    .^M
    |                              RSCP - Error                            |^M
    | Error from:             Codepage handling (RSCP)                     |^M
    | code:   64  RSCPECALL    Illegal mixture of procedure calls.         |^M
    | Table UMGSETTING of no interest in Unicode system                    |^M
    | module: rscpexcc no:   12 line:   673                    T100: TS007 |^M
    | TSL01: CQ3  p3: UMGSETTING  p4: &                                    |^M
    |----
    |^M
    | Some OS information, which may or may not belong to this error:      |^M
    | errno    17  File exists                                             |^M
    `----
    '^M
    (DB) INFO: disconnected from DB
    /sapmnt/S01/exe/R3load: job finished with 1 error(s)
    /sapmnt/S01/exe/R3load: END OF LOG: 20071123101328
    Thanks for your attention
    Rgds,
    Fausto

    This process can't be done in Unicode. Has to be done in Non-UC.

  • Lock Object error during batch load

    Batch load was delayed about an hour. First error message in the Server Log said "Object is already locked by user""Error 1053010 processing request [Lock Object]-Disconnecting"Then it went through a series of "Object locked by user admin" and "received client request: create Temporary Object(from user admin)"It did this several times until connection.Although from time to time we have had problems with Objects getting locked and preventing maintenance in the Application Manager. This is the first time that is has prevented a batch load. Does anyone know what could have triggered this during the load process?

    This happened to us a lot, I think it's due to people opening objects, then letting their session time out, leaving a phantom lock.We solved the problem by calling the UNLOCKOBJECT function before any dimbuilds. 99.9% of the time this raises an error, "..object x is not locked..", but the rare time that the otl is inadvertently locked, it prevents the script from failing.<br><br>HTH,<br><br>Jeff McAhren<br>Dallas, Texas<br><br>ex to unlock the outline for app/db flash:<br><br>UNLOCKOBJECT 1, "FLASH" "FLASH" "FLASH"

  • Turning off sleep mode during high CPU usage

    There isn't option to prevent going to sleep mode during high CPU or network usage. This is not professional for serious work like 3D rendering, video montage, downloading or sending large files. Windows is going to sleep during performing serious tasks.
    Setting different sleep mode values every time is no way because we didn't know how long these tasks will running.

    There isn't option to prevent going to sleep mode during high CPU or network usage. This is not professional for serious work like 3D rendering, video montage, downloading or sending large files. Windows is going to sleep during performing serious tasks.
    Setting different sleep mode values every time is no way because we didn't know how long these tasks will running.

  • 2008 R2/ Win7 Offline Files Sync causing High Load on server

    Hi Everyone,
    I have recently been investigating extremely high CPU Usage from the System Process on my company's main File Cluster.
    We managed to track SRV2.sys Threads causing high CPU load within the system process, but was having issues identifying why this was the case.
    As per Microsoft's direction via support call, we have installed the latest SRV2.sys Hotfixes, but this does not appear to have allivated the issue we are experiencing. We have also added more CPU and Memory into both nodes, which has not helped either.
    We have since managed to create a system dump and is being sent to MS Support for analysis.
    I have noticed the following that appears to happen on our cluster:
    Whenever our CAD/Design department run certain functions within their apps running on a windows 7 client (apps include MicroStation, Revit, AutoCAD etc) we see a massive spike and flatline of the system process.
    We found several users with Windows 7 clients that have Configured Offline File to sync an entire Network Volume (some volumes are 2TB Plus, so would never fit on a users computer anyway, i was quite shocked when I found this). How we spotted this was through
    Resource Monitor, the System Process was trolling through all the folders and files in a given volume (it was "reading every single folder). Now, while this was the system process, we could identify the user by using the Open Files view in Server Manager's
    Share and Storage Management tool.
    I have done a fair bit of research and found that a lot of CAD/Drawing applications in the market have issues with using SMB2 (srv2.sys). When reviewing the products that we use, I noticed that a lot of these products actually recommended disabling SMB2
    and reverting to SMB1 on File Server and/or the clients.
    I have gone down the path of disabling SMB2 on all Windows 7 clients that have these CAD Applications installed to assist with lowering the load (our other options are to potentially shift the CAD Volumes off our main file cluster to further isolation these
    performance issues we have been experiencing.) We will be testing this again tomorrow to confirm that the issue is resolved when mass amounts of CAD users access data on these volumes via their CAD Application.
    We only noticed the issue with Offline Files today with trying to sync an ENTIRE Volume. My questions are:
    Should Offline File sync's really cause this much load on a File Server?
    Would the the size of the volume the sync is trying to complete create this additional load within the system process?
    What is the industry considered "Best Practice" regarding Offline Files setup for Volumes which could have up to 1000+ users attached? (My personal opinion is that Offline Files should only be sync of user "Personal/Home" Folders
    as they users themselves have a 1 to 1 relationship with this data.)
    Is there an easier way to identify what users have Offline Files enabled and "actually being used" (from my understanding, Sync Center and Offline Files are enabled by default, but you obviously have to add the folders/drives you wish to sync)?
    If I disable the ability for Offline Files on the volumes, what will the user experience be when/if they try to sync their offline files config after this has been disabled on the volume?
    Hoping for some guidance regarding this setup with Offline Files.
    Thanks in Advance!
    Simon

    Hi Everyone,
    Just thought I would give an update on this.
    While we're still deploying http://support.microsoft.com/kb/2733363/en-us to
    the remainder of our Windows 7 SP1 fleet, according to some Network Traces and XPerf that were sent to MS Support, it looks as though the issu with the Offline Files is now resolved.
    However, we are still having some issues with High CPU with the System Process in particular. Upon review of the traces, they have found a lot of ABE related traffic, leading them to believe that the issue may also be caused by ABE Access on our File Cluster.
    We already have the required hotfix for ABE on 2008 R2 installed (please see
    http://support.microsoft.com/kb/2732618/en-us), although we have it set to a certain value that MS believe may be too high. Our current value is set to "5", as that is the lowest level of any type of permission is set (i.e. where the lowest level of inheritance
    is broken).
    I think I will mark this as resolved regarding the Offline Files issue (as it did resolve the problem with the Offline Files)...
    Fingers crossed we can identify the rest of the high load within the System Process!!!

  • Replacing / Compiling Database Triggers in a HA/High Load system

    Hi there,
    My collegue has just asked me if downtime needs to be scheduled to replace an After IUD Database trigger where I've made a minor change.
    If this were a package or procedure, the answer would be an obvious yes, as I'd be wary of my users getting the dreaded: ORA-04068... State of Package has been disguarded.
    What about when working with database triggers?
    If the trigger is firing at the same time that I perform my Create or Replace Trigger DDL, will everything fall into a screaming heap?
    Notes:
    Oracle 10g R2 db.
    High DML rates on the table that the After IUD Trigger is 'attached' to.
    It's a near certainty that when the Create or Replace Trigger DDL runs, the trigger will be active at the time.
    The Trigger is 'simple' in that it's use: is to call a DB package with some of the :new values as params.
    +(I'm happy to rtfm - if someone can tell me where!!)+
    Cheers -
    Ron Marks

    If this were a package or procedure, the answer would be an obvious yes, as I'd be wary of my users getting the dreaded: ORA-04068... State of Package has been disguarded.Not true. This exception is thrown only if sessions have some state represented in package variables. If there is no associated session state, then no exception would be thrown after recompile.
    If the trigger is firing at the same time that I perform my Create or Replace Trigger DDL, will everything fall into a screaming heap?You'll wait - to lock an object in library cache which represents your trigger. Not sure if this would require a library cache lock on underlying table, but I would guess that it is required. BTW, locking a table would make no any sense: CREATE TRIGGER is a DDL, which means before it began it issues a COMMIT => all locks are released, and it is quite possible that a TM lock for CREATE TRIGGER won't be acquired since resource would be busy.
    Oracle 11gR2 claims to address problems of application upgrades on a high-load environments with a feature called "Editions" (not a good name - try to find something about it via Google...)

  • Sync messages failed due to high load

    Hi!
    On our test system we had a very high load of async messages for 3 - 4 hours. The async messages were processed successfully but all sync messages failed during this time frame! As soon as the high load of async messages stopped, the sync messages started executing successfully again.
    Has anybody also experienced this kind of problem and what can be done to avoid this issue? Thanks for any input and suggestions.
    Regards, Tanja

    hey Tanja,
    i had a similar issue when testing the box with Load runner, that a simple sync scenario fails when the size of payload or attachment is high, the thing that i did is just increased the virtual memory ,
    You can also check the trace files to figure out problem ?
    This might not be a definitive answer , hope this helps,
    Vara
    [PLEASE REWARD FOR POSTING]

  • Safari causes system freeze during page load

    Description:
    I've been trying to figure this one out for a while now (about 2 weeks), without any luck. I've tried reading a couple of threads describing similar problems, but it looks like I've got something else going on here seeing as it affect the entire system. I appreciate any suggestion!
    When a window in Safari is open and you're loading a page, it sometimes (almost always) causes a system freeze. Any action performed on any input device within OSX while a page is loading gets delayed by 5-20 seconds. Anything from right-clicking, to opening a new tab in Safari, to navigating to a new folder in Finder.... literally anything. The actual loading of the page is also affected by this. So when I click a link, the page becomes white and the system stalls. After X seconds, the system resumes and the page is displayed fully loaded. Once the page is loaded, everything is back to normal. I can just leave the Safari-window open without anything happening. It's just during the loading of a page.
    Using the mouse to move and switch between open windows are the only actions I seem to be able to perform during the system freeze.
    When it doesn't cause a system-wide freeze, it will still take forever for any page to load. Which isn't a problem in any other browser and I've never had any speed issues in general on any device.
    What I know for sure is that I can solve the problem by closing all open Safari-tabs and windows or closing the app completely, and use Chrome instead. But I do like Safari, so I hope there's another way to fix this.
    I should also note that I've made no changes to the system for many weeks now, other than installing whatever updates might have been available on the App Store.
    What I've tried within Safari:
    - program restart
    - reset settings (every option checked)
    - disable and remove extensions (only had one installed, LiveReload for Adobe Brackets)
    - empty cache
    - delete cookies
    What I've tried within OSX:
    - system restart
    - complete OSX-recovery
    - activity monitor (CPU, HDD and RAM always within normal values during all of this)
    - checked HDD (successfull)
    - checked internet-connection (got 50Mbit/s connection, no speed issues, checked on multiple devices)
    - tried other browsers (Chrome (OSX) and IE (Win 7, Parallels) both run smoothly)
    - turned off IPv6 in terminal (based on a suggestion here at the forums)
    - disabled iCloud-sync
    - disabled Dropbox
    - probably a couple of other things i can't remember right now

    So... I finally fixed it.
    In addition to the things I mentioned I tried, I also checked if it could be malware-related. It wasnt. I then checked if it was just affecting my user-account or OSX globally. I created a new account and Safari ran perfectly. I therefore assumed it had to be something in my library-folder.
    I deleted just about any mention of Safari in the Library-folder (the visible one) and just to be absolutely sure, I deleted the Contents-folder inside Safari.app in the Applications-folder. I then downloaded and installed Safari 7.0.3. Everything runs smooth now.

Maybe you are looking for

  • 10.4.6 AFP & PhotoShop CS2 crashes

    Hello all, I'm trying to save PhotoShop files to a Windows 2003 AFP (ExtremeZ-IP) volume and PhotShop occasionally will crash, but after saving the file. It doesn't crash all the time. What I've tried: I've updated the ExtremeZ-IP application running

  • QoS on Catalyst 6500

    We have the following QoS config running on Edge, Distributions and Cores and got the following error. “priority command is not supported in output direction for this interface Configuration failed on: Port-channel” We had opened a TAC case and t

  • Manual image transformation: Losing top of the image

    With LR3 I often use the manual image transformations and observe an undesirable effect, which I cannot explain to myself: When changing the Vertical Perspective to a negative value, the image will be transformed to cope with the perspective as inten

  • Exception occurred during event dispatching:java.lang.ArrayIndexOutOfBounds

    hi suppose i have 4 rows in my table i m trying to access first colunm in all rows with a for loop for(r0w=0;row<4;row++) getValueAt(row,0); its throwing ArrayIndexOutOfBoundsException for teh last row plz help me its urgent

  • Upconverting Stereo sound to Surround 5.1 or higher?

    Does something of the sort exist for linux? I'm just having a hard time living without the "CMSS-3D" feature of the windows soundblaster drivers. They also have an alternative called DTS:Connect which includes DTS:Neo PC (which i could use) and DTS:I