JMS Channel Cluster Nodes-INACTIVE

Hello All,
We have a Sender - JMS Channel which is green state but the Cluster Node (10 of them) are in WAITING STATE - Channel_Inactive. And Nodes are in GREEN STATE.
I have checked the Cache in Integration Directory where I could see RED entries and I have tried 'Repeat Cache Instance' , but in vain. Is it a fair idea to Run the Function-module 'LCR_CLEAR_CACHE' ? Does it have any impact?!
Due to this 1 of the message is lying in the MS system (JMS stream) in Uncommitted State.
ALL HAPPENING IN A PROD SYSTEM!!!
Please find the screenshots attached.
Regards
KarthiSP

Hi ,
Check the central adapter engine cache status in Cache monitoring from RWB...its having green or red...if it is red check with basis team...
Thanks,
Naveen

Similar Messages

SOAP Receiver Channel 2 nodes inactive in Production system

Hi All,
We are facing an issue with the SOAP receiver channels in production system. There are 4 nodes which can be seen in the communication channel monitoring and out of which 2 have the status as inactive and the short log says "inactive or never used".
But the same nodes are active on the sender SOAP channel and all other channels.We have also checked the Status of the SOAP service that is also set to active.
Now when the server (App server5 in our case) is switched off, which has the 2 active nodes, the messages are not processed since the nodes on the other server are inactive.
Request you to please help us out in finding out why the nodes are in inactive status only for SOAP receiver channels. Do we need to change any settings at the adapter engine level.
Thank you.
Regards,
Sarvesh Desai
Edited by: Sarvesh Nispat Desai on Aug 25, 2010 5:25 PM

There is an error in reporting the status of SOAP channels. This error has already been fixed by SAP and is available as a fix in higher releases (PI 7.10 and above, latest SPs) .
The status "inactive or never used" is shown in channel monitoring at the following two times:
1. After creating a new SOAP channel (sender/receiver) till this channel processes any new message.
2. After the PI system is restarted, all the SOAP channels show this status, though they are ready to handle any incoming/outgoing messages. The channels continue to display this status till they process any new message.
In both these situations, the SOAP Channel can handle/process messages successfully. So, kindly ignore this channel status ("inactive or never used") reporting error in channel monitoring. This will not at all hamper the normal functioning of the SOAP channels.
Hope this answers your query.

Sender File Channel Cluster Node issue in CC monitoring

Hi,
I have a issue in Sender File CC. The channel is active and polling every 1 min. The issue is in the CC monitoring it is just showing one single line i.e "Polling interval started.Length: 60.0 seconds" the other two lines Processing started and Processing Finished Successfully are not visible...
Please let me know the solution if you have faced a similar issue.
http://i50.tinypic.com/28sv4vr.jpg This is the refrence to the screen shot from CC monitoring...

Deepak,
Please let me know the solution if you have faced a similar issue.
Delete the Enqueue-lock related to the stopped Communication Channel. Refer SAP Note 821267.
For the 7.1 versions, please note that the Lock Manager is under the /nwa/locks, or on the location "Availability and Performance Management -> Locks".
@ Ramg
If no files are placed in the source directory and the channel will keep try polling l!!! Place files in the source and check.
Even if there are no files in the source directory, the 3 steps of - polling interval started, processing started and processing finished - should take place by the file channel.
Regards,
Neetesh

Concurrent nodes reading from JMS topic (cluster environment)

Hi.
Need some help on this:
Concurrent nodes reading from JMS topic (cluster environment)
Thanks
Denis

After some thinking, I noted the following:
1 - It's correct that only one node subscribes to a topic at a time. Otherwise, the same message would be processed by the nodes many times.
2 - In order to solve the load balancing problem, I think that the Topic should be changed by a Queue. This way, each BPEL process from the node would poll for a message, and as soon as the message arrives, only one BPEL node gets the message and take if off the Queue.
The legacy JMS provider I mentioned in the post above is actually the Retek Integration Bus (RIB). I'm integrating Retek apps with E-Business Suite.
I'll try to configure the RIB to provide both a Topic (for the existing application consumers) and a Queue (an exclusive channel for BPEL)
Do you guys have already tried to do an integration like this??
Thanks
Denis

Cluster node (1 out of 6) is in error for a file channel - SAP XI

Hi,
In one of a sender file channel for an interface, one java node out of 6 configured ones failed. The error message says "Login Incorrect" while the other cluster nodes are polling properly. I have tried to update the password in the channel's configuration in Integration Directory and activated it. This doesnt help. Please advice!
Thanks in advance!
Regards,
Kumaran

Hello Kumaran,
The status of the file adapter is not reflected properly but this should not have an impact. No message might have arrived at that node yet. Once it receives a message for processing, the status will be changed.
Anil

Cluster node problems in communciation channel

Hello,
I often have node problems where all messags get stuck in particular cluster node of communication channel.
Once it is restarted, all messages are working problem, With respect to that were the following issues. Error when getting an FTP connection from connection pool: com.sap.aii.af.service.util.concurrent.ResourcePoolException: Unable to create new pooled resource: ConnectException: Connection refused (errno:239)
Please do let me know for the solution to avoid it in the future.
Regards,
Anandh

Hello,
Question 43 of file adapter faq,
Q: J2EE engine hangs with the File/FTP sender channels. How to resolve this ?
A: The reason for this is, some times due to network level issues, a message is waiting forever for a response from a FTP server which is down for some time. Adapter does not know this and try to poll FTP server again with second message. This goes on and on and eventually we would have J2EE engine hanging. To solve this, following things need to be applied:
    1) Set FTP timeout to appropraite value if channel is FTP sender channel.
        2) In advanced mode table options, add a new parameter 'clusterSyncMode' and its value set to 'lock'. This parameter             is without single quotes and case-sensitive.
        3) Last but not least, make sure that you are in latest patches            of SP19/SP20/SP21/SP22 for SAPXIAFC component of XI 3.0                 release and SP10/SP11/SP12/SP13/SP14 for SAPXIAFC component             of XI 7.0 release. Any patch which is released after 11th of    Febraury 2008 for the above releases is fine.
We had the same issue and this did the trick.
Regards,
Bhavesh

Processing in Multiple Cluster Nodes

Hi All,
In our PI system we have 2 Java nodes due to some requirement. When the communication channel runs and we check the message log, in one Cluster node we have a successful message. In other Cluster Node we have an error message that says "File not found".
The file processing is completeing successfully on one Cluster node. But I wanted to know if there is any way to suppress the processing of the same file by same channel on another Node. Some setting in administration or IB where we can get this done.
Is there any way to get this done by some setting?
Thanks,
Rashmi.

Hello!
As per note #801926, please set the clusterSyncMode parameter on Advanced tab of the communication channel with LOCK value.
And also check the entries 4 and 48 of the FAQ note #821267:
4. FTP Sender File Processing in Cluster Environment
48. File System(NFS) File Sender Processing in Cluster Environment
Best regards,
Lucas

Receiver JMS Channel Error

hi,
I am facing an issue with the JMS Receiver Communication channel.Data is not getting inserted in the JMS Queue.CC Monitoring is showing the below ERROR:
Message processing failed. Cause: com.sap.aii.af.ra.ms.api.RecoverableException: Connector for ConnectionProfile of channel: JMS_r_CustomerLedgeron node: 7065651 having object id: 340fa47e4dc33a04a4f8a5a108708f07 encountered error: Error sending message in sending to destination custLedgerReplication, the message message: TextMessage[null, <?xml version="1.0" encoding="...]: weblogic.jms.common.JMSException: Error sending message.
can someone helpme out.
thanks,
robin

Hello Robin,
did you resolve this issue? I have now the same problem.
Do you have connections with two (or more) BEA servers using the same Adapter Framework?
Note 1090516 describes a problem with credentials confusion when connecting to more servers.
It could be the cause.
With regards
Tomas

Query regarding Cluster nodes in CC

Hi Experts,
We have a query regarding the cluster nodes available in the CC monitoring.
Can two nodes of a same channel can poll at the same time?
Kindly suggest what should be done to make a specific cluster node of a CC polls at a particular time.
Thanks
Suganya.

Hi,
There is an answered thread on this
Processing in Multiple Cluster Nodes
Regards,
Manjusha

What happens if Channel started but inactive?

Hi.
While monitoring the training server, i found there are 100 channels with yellow color status "Channel started but inactive".
All these channels are not SOAP or MAIL channels. Most are FILE, JDBC, JMS channels.
I know that we can stop the channels in RWB but what happens if these channels are kept with that yellow status? If i stop these channels will it improve the performance??
Kindly let me know how it will effect the performance.
Thanks... !
Warm regards,
Praveen.

Hi,
While monitoring the training server, i found there are 100 channels with yellow color status "Channel started but inactive". - It means there was no msg for these channels so they are inactive......when there will be msgs for these channels, then those msgs will be processed by these channels and its status will change to running........
I know that we can stop the channels in RWB but what happens if these channels are kept with that yellow status? - No performance impact because of this inactive channels :).............
If i stop these channels will it improve the performance?? - Do not stop these channels but keep them started..........
Regards,
Rajeev Gupta

Iscsi target on cluster node

Hi
I'm trying to install a cluster on a lab environment, i have to physical servers, and would like to use them as cluster nodes, on one of this nodes i would like to install iscsi target server to use for sharing disk to the cluster itself is this possible?
because i did all the configurations but after installing the cluster the iscsi target server doesn't work anymore.
thanks

Hi
I'm trying to install a cluster on a lab environment, i have to physical servers, and would like to use them as cluster nodes, on one of this nodes i would like to install iscsi target server to use for sharing disk to the cluster itself is this possible?
because i did all the configurations but after installing the cluster the iscsi target server doesn't work anymore.
thanks
Bad news: You cannot do it with a Microsoft built-in solutions because you do need indeed to have physical shared storage to make Microsoft iSCSI target clustered. Something like on Robert Smit blog here:
Clustering Microsoft iSCSI Target
https://robertsmit.wordpress.com/2012/06/26/clustering-iscsi-target-on-windows-2012-step-by-step/
...or here:
MSFT iSCSI Target in HA
https://technet.microsoft.com/en-us/library/gg232621(v=ws.10).aspx
...or very detailed walk thru here:
MSFT iSCSI Target in High Availability Mode
https://techontip.wordpress.com/2011/05/03/microsoft-iscsi-target-cluster-building-walkthrough/
Good news: you can take a third-party solution from various companies (below) and create an HA iSCSI volumes on just a pair of nodes. See:
StarWind Virtual SAN
http://www.starwindsoftware.com/starwind-virtual-san-free
(this setup is FREE of charge, you just need to be an MCT, MVP or MCP to obtain your free 2-node key)
...to have a setup like this:
Also SteelEye has similar one here:
SteelEye #SANLess Clusters
http://us.sios.com/products/datakeeper-cluster/
DataCore SANsyxxxx
http://www.datacore.com/products/SANsymphony-V.aspx
You can also spawn a VMs running FreeBSD/HAST or Linux/DRBD to build a very similar setup yourself (Two-node setups should be Active-Passive to avoid brain split, Windows solutions from above all maintain own pacemaker and heartbeats to run Active-Active
on just a pair of nodes).
Good luck and happy clustering :)
StarWind Virtual SAN clusters Hyper-V without SAS, Fibre Channel, SMB 3.0 or iSCSI, uses Ethernet to mirror internally mounted SATA disks between hosts.

One cluster node shows erro

Hi,
This FTP channel Sender and in RWB i see one cluster node Red ( error ) and other as Green
The error is Unknown host exception. Files are being pulled by other node but first node shows error.
Any guess why would this happen.
Thanks!

>>>Files are being pulled by other node but first node shows error.
This is fine. If you observer the error time-stamp it might not be latest unlike the one which is processing.
AFAIK - At a given point your sender channel will point to only one instance.
So whenever (random behavior) the channel points to other instance, this should go away.
If you notice a different behavior then you should configure the advanced mode parameter "clusterSyncMode".

Cluster node reboots repeatedly

We have 2 node 10.1.0.3 cluster setup. We had a problem with a HBA card for the fibre channel to SAN and after replacing it, one of the cluster nodes keeps rebooting itself right after the Cluster processes startup.
We have had this issue once before and Support suggested the following.. Howevere the same solution is not working this time around.. Any ideas?
Check output of the unix command hostname is node1
Please rename cssnorun file in /etc/oracle/scls_scr/node1/root directory. Please issue "touch /etc/oracle/scls_scr/node1/root/crsdboot" and also change the permission and ownership of the file to match that of the node 2. Please check if there is any differences in permission, ownership, and the group for any files or directory structure under /etc/oracle between two nodes.
Please reboot node 1 after this change and see if you run into the same problem.
Please check if there is any /tmp/crsctl* files.

Well especially if you are Linux RH4 the new controler card will have cause the device names to change. Check that out. It could be that you are no longer seeing you vote and crs partitions. This can happen on other operating systems if the devices now have a new name because the controller card has changed.
For Linux try the Man pages on udev and search for udev on OTN
Regards

Cluster Node Unable to Maintain Cluster Membership

My cluster logs are very similar to the above thread... was it ever addressed?
[SV] Already protecting connection with message security level 'sign'
[FTI] Stream already exists to node: false
[Channel IP to another cluster node member] Close()
GracefuleClose(1226) because of channel to remote endpoint another cluster node
~ is closed
Cluster services stops and generates:
The Kerberos client received a KRB_AP_ERR_MODIFIED error from the server serverName$. The target name used was
serverName.
This indicates that the target server failed to decrypt the ticket provided by the client. This can occur when the target server principal name (SPN) is registered on an account other than the account the target service is using. Ensure that the target SPN
is only registered on the account used by the server.
Roderick Lyons

Hi Roderick Lyons,
Could you tell us the exact URL “above thread” I am not very sure which thread you meaning.
Please offer more information about your environment, such as, the DC server edition, the cluster node server edition.
If you are 2003 and 2012R2 mixed DC environment please restart your cluster node then do the further monitor.
The related article:
It turns out that weird things can happen when you mix Windows Server 2003 and Windows Server 2012 R2 domain controllers
http://blogs.technet.com/b/askds/archive/2014/07/23/it-turns-out-that-weird-things-can-happen-when-you-mix-windows-server-2003-and-windows-server-2012-r2-domain-controllers.aspx
Can't log on after changing machine account password in mixed Windows Server 2012 R2 and Windows Server 2003 environment
http://support.microsoft.com/kb/2989971
From the current error another possible is you never run the cluster validation before you create the cluster, please run the cluster validation first then post
the warning or error information.
If above solution not work please consider reboot your PDC at unproductive time.
More information:
Kerberos Service Principal Name on Wrong Account
https://support.microsoft.com/kb/2706695?wa=wsignin1.0
Fixing the Security-Kerberos / 4 error
http://blogs.technet.com/b/dcaro/archive/2013/07/04/fixing-the-security-kerberos-4-error.aspx
Service Principal Names (SPNs) SetSPN Syntax (Setspn.exe)
http://social.technet.microsoft.com/wiki/contents/articles/717.service-principal-names-spns-setspn-syntax-setspn-exe.aspx
I’m glad to be of help to you!
Please remember to mark the replies as answers if they help and unmark them if they provide no help. If you have feedback for TechNet Support, contact [email protected]

SCVMM losing connection to cluster nodes

Hey guys'n girls, I hope this is the right forum for this question. I already opened a ticket at MS support as well because it's impacting our production environment indirectly, but even after a week there's been no contact. Losing faith in MS support there
The problem we're having is that scvmm is that a host enters the 'needs attention' state, with a winrm error 0x80338126. I guess it has something to do with the network or with Kerberos, and I've found some info on it, but I still haven't been able to solve
it. Do you guys have any ideas?
Problem summary:
We are seeing an issue on our new hyper-v platform. The platform should have been in production last week, but this issue is delaying our project as we can't seem to get it stable.
The problem we are experiencing is that SCVMM loses the connection to some of the Hyper-V nodes. Not one
specific node. Last week it happened to two nodes, and today it happened to another node. I see issues with WinRM, and I expect something to do with kerberos. See the bottom of this post for background details and software versions.
The host gets the status 'needs attention', and if you look at the status of the machine, WinRM gives an error. The error is:
Error (2916)
VMM is unable to complete the request. The connection to the agent cc1-hyp-10.domaincloud1.local was lost.
WinRM: URL: [http://cc1-hyp-10.domaincloud1.local:5985], Verb: [ENUMERATE], Resource: [http://schemas.microsoft.com/wbem/wsman/1/wmi/root/cimv2/Win32_Service], Filter: [select * from Win32_Service where Name="WinRM"]
Unknown error (0x80338126)
Recommended Action
Ensure that the Windows Remote Management (WinRM) service and the VMM agent are installed and running and that a firewall is not blocking HTTP/HTTPS traffic. Ensure that VMM server is able to communicate with cc1-hyp-10.domaincloud1.local over WinRM by successfully
running the following command:
winrm id –r:cc1-hyp-10.domaincloud1.local
This
problem can also be caused by a Windows Management Instrumentation (WMI) service crash. If the server is running Windows Server 2008 R2, ensure that KB 982293 (http://support.microsoft.com/kb/982293)
is installed on it.
If the error persists, restart cc1-hyp-10.domaincloud1.local and then try the operation again. /nRefer to
http://support.microsoft.com/kb/2742275 for more details.
Doing a simple test from the VMM server to the problematic cluster node shows this error:
PS C:\> hostname
CC1-VMM-01
PS C:\> winrm id -r:cc1-hyp-10.domaincloud1.local
WSManFault
    Message = WinRM cannot complete the operation. Verify that the specified computer name is valid, that the computer is accessible over the network, and that a firewall exception for the WinRM service is enabled and allows access from this
computer. By default, the WinRM firewall exception for public profiles limits access to remote computers within the same local subnet.
Error number: -2144108250 0x80338126
WinRM cannot complete the operation. Verify that the specified computer name is valid, that the computer is accessible over the network, and that a firewall exception for the WinRM service is enabled and allows access from this computer. By default, the WinRM
firewall exception for public profiles limits access to remote computers within the same local subnet.
I CAN connect from other hosts to this problematic cluster node:
PS C:\> hostname
CC1-HYP-16
PS C:\> winrm id -r:cc1-hyp-10.domaincloud1.local
IdentifyResponse
    ProtocolVersion =
http://schemas.dmtf.org/wbem/wsman/1/wsman.xsd
    ProductVendor = Microsoft Corporation
    ProductVersion = OS: 6.3.9600 SP: 0.0 Stack: 3.0
    SecurityProfiles
        SecurityProfileName =
http://schemas.dmtf.org/wbem/wsman/1/wsman/secprofile/http/spnego-kerberos
And I can connect from the vmm server to all other cluster nodes:
PS C:\> hostname
CC1-VMM-01
PS C:\> winrm id -r:cc1-hyp-11.domaincloud1.local
IdentifyResponse
    ProtocolVersion =
http://schemas.dmtf.org/wbem/wsman/1/wsman.xsd
    ProductVendor = Microsoft Corporation
    ProductVersion = OS: 6.3.9600 SP: 0.0 Stack: 3.0
    SecurityProfiles
        SecurityProfileName =
http://schemas.dmtf.org/wbem/wsman/1/wsman/secprofile/http/spnego-kerberos
So at this point only the test from the cc1-vmm-01 to cc1-hyp-10 seems to be problematic.
I followed the steps in the page
https://support.microsoft.com/kb/2742275 (which is referred to above). I tried the VMMCA, but it can't really get it working the way I want, or it seems to give outdated recommendations.
I tried checking for duplicate SPN's by running setspn -x on affected machines. No results (although I do not understand
what an SPN is or how it works). I rebuilt the performance counters.
It tried setting 'sc config winrm type= own' as described in [http://blinditandnetworkadmin.blogspot.nl/2012/08/kb-how-to-troubleshoot-needs-attention.html].
If I reboot this cc1-hyp-10 machine, it will start working perfectly again. However, then I can't troubleshoot the issue, and it will happen again.
I want this problem to be solved, so vmm never loses connection to the hypervisors it's managing again!
Background information:
We've set up a platform with Hyper-V to run a VM workload. The platform consists of the following hardware:
2 Dell R620's with 32GB of RAM, running hyper-v to virtualize the cloud management layer (DC's, VMM, SQL). These machines are called cc1-hyp-01 and cc1-hyp-02. They run the management vm's like cc1-dc-01/02, cc1-sql-01, cc1-vmm-01, etc. The names are self-explanatory.
The VMM machine is NOT clustered.
8 Dell M620 blades with 320GB of RAM, running hyper-v to virtualize the customer workload. The machines are
called cc1-hyp-10 until cc1-hyp-17. They are in a cluster.
2 Equallogic units form a SAN (premium storage), and we have a Dell R515 running iscsi target (budget storage).
We have Dell Force10 switches and Cisco C3750X switches to connect everything together (mostly 10GB links).
All hosts run Windows Server 2012R2 Datacenter edition. The VMM server runs System Center Virtual Machine Manage 2012 R2.
All the latest Windows updates are installed on every host. There are no firewalls between any host (vmm and hypervisors) at this level. Windows firewalls are all disabled. No antivirus software is installed, no symantec software is installed.
The only non-standard software that is installed is the Dell Host Integration Tools 4.7.1, Dell Openmanage Server Administrator, and some small stuff like 7-zip, bginfo, net-snap, etc.
The SCVMM service is running under the domain account DOMAINCLOUD1\scvmm. This machine is in the local administrators group of each cluster node.
On top of this cloud layer we're running the tenant layer with a lot of vm's for a specific customer (although they are all off now).

I think I found the culprit, after an hour of analyzing wireshark dumps I found the vmm had jumbo frames enabled on the management interface to the hosts (and the underlying infrastructure does not).. Now my winrm commands started working again.

JMS Channel Cluster Nodes-INACTIVE

Similar Messages

Maybe you are looking for