Opmn cannot start coreapplication_obis1

Question

Hi after installing obiee11g on win2008r2 64-bit and updating rpd and catalog I cannot start the coreapplication_obis1 using opmn:
-->snip
C:\Windows\system32>opmnctl startproc ias-component=coreapplication_obis1
opmnctl startproc: starting opmn managed processes...
================================================================================
opmn id=DBY-OBIEE11G:9501
0 of 1 processes started.
ias-instance id=instance2
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
ias-component/process-type/process-set:
coreapplication_obis1/OracleBIServerComponent/coreapplication_obis1/
Error
--> Process (index=1,uid=1920012172,pid=1404)
time out while waiting for a managed process to start
Log:
none
C:\Windows\system32>
<-- snap
I have checked all log and out files but I don't see any error and therefore I'm stuck.
Can I set a debug mode or do anything else to get information about what's happening ?
Thanks,
Knut

Answer 1

As explained in the Oracle documentation, it normally should be very easy to upgrade using the UpgradeAssistant (ua.bat).
But when trying to apply the upgrade, we encountered an error and the UpgradeAssistant would cancel the process and return “failed” as upgrade state.
Several tries later and analyzing the log files of the assistant, it was obvious that the java components of the BI weblogic server
were not started properly, which led to the failure of the upgrade. The log file(s) are located at $ORACLE_HOME\upgrade\logs .
Just for clarifying, this is the error the assistant threw at us:
[2011-03-31T15:11:31.500+01:00] [BIEE] [ERROR] [] [upgrade.BIEE] [tid: 15] [ecid:0000IwCDvHF3r2o_sTS4yX1D_8XV000006,0] javax.management.OperationsException: Unable to start process: instance1: coreapplication_obis1[[
at ….
at ….
at ….
Unfortunately, you will not be able to find help or guidiance among this error message on the internet or the Oracle forums. I really don’t know why, but all you find are “Upgrade-How-To’s” with zero error explanation.
Looking at Window’s own server logs using the Server Manager, it was discovered that there was some kind of problem with
the OraPlc11.dll, which is part of these required components. I actually discovered this because I tried to startup BI weblogic-server
separately, which causes these java components to be started, too.
Their health status can easily be monitored by the OracleProcessManager utility (opmnctl.bat can normally be found under $ORACLE_INSTANCE\bin .
Using ‘opmnctl status’ while starting the weblogic server throws the components health as output inside the console.
As expected, the components (in this particular case OBIS1) would not go any further than initializing, while in the Server Manager tool under “Diagnostics\Event Viewer\Windows Logs\Application” there would appear a new error message throwing the following error related to our component(s):
OraPlc11.Dll cannot be loaded. OS Error message: The specified procedure could not be found.
We double checked our complete installation for the existence of this file (we simply used the windows search and looked up OraPlc11.dll).
We encountered this file three times; First in our 11g installation folder (D:\oracle\client11gr2\product\11.2.0\client_1\bin), being 110 KB in size, second in our $ORACLE_HOME\bin folder being 97 KB in size and third in the rcu directory under “transfer” (D:\transfer\ofm_rcu_win32_11.1.1.3.3_disk1_1of1\rcuHome\BIN), 80 KB in size.
We knew that the 110 KB OraPlc.dll was used at the moment, so we tried using the 80 KB OraPlc.dll by renaming the 110 KB – dll to OraPlc.dll.old and copying the 80 KB – dll into our oracle installation (D:\oracle\ client11gr2\product\11.2.0\client_1\bin).
Then, we restarted the OS, started the BI weblogic-server via console (the start/stop script is found under $ORACLE_INSTANCE\bifoundation\OracleBIApplication\coreapplication), monitored the component status until all 5 reached the state “alive”, and then we started the UpgradeAssistant via console (ua.bat is located in $ORACLE_HOME\bin) with the additional parameter “–logLevel TRACE” for more detailed log information. We then went through the assistant as usual and described in the Oracle documentation. The upgrade process succeeded this time (ignore warnings).
Hope this will help other people encountering this problem in the future.
Greetz,
Felix